Get documents about "Voice over Wi-Fi Test Methodology
Document Sample
Toyo and Azimuth
Voice over Wi-Fi
Test Methodology
Fanny Mlinarsky
Founder and CTO, Azimuth
May 2005
Outline
• VoWiFi - history and current status
• IEEE 802.11 overview and update
• Wi-Fi Alliance overview and update
• Azimuth VoWiFi test solution
• Azimuth demonstration VoIP
Cellular Wi-Fi
Slide 2
VoIP-Data Convergence
Traditional Enterprise Converged Telephony
Connectivity and Data Networks
First VoIP products emerged in 1996
Slide 3
VoWiFi - Legacy PBX Systems
• VoWiFi manufacturers
– Cisco, Symbol, Spectralink
– Spectralink WiFi phones
available since 1999
– 802.11b only for now
• Most IP PBXs use
proprietary protocols
– Cisco SCCP
(skinny call control protocol)
– Alcatel UA
– NEC Protims
– Avaya CCMS
– Nortel UniStim
– Siemens CorNet IP
– Mitel MiNet
– Spectralink SVP
• Standard IP telephony protocols
– Session Initiation Protocol (SIP)
– H.323
– Media Gateway Control Protocol (MGCP)
Slide 4
Cell-WiFi Convergence
Private Network
Base
Dual-Mode Station
UMA Cellular Radio Access Controller Core
Network (RAN)
Handset (BSC) Mobile
Network
IP Access
Network
UMA
Network
Unlicensed Mobile Access Controller
Network (UMAN) (UNC)
• Significant Cell-WiFi Convergence efforts throughout the
telecom industry
– 3GPP GAN (generic access network), a.k.a. UMA (unlicensed mobile access)
– 3GPP WLAN Interworking (IMS)
– SCCAN (seamless converged communication across networks)
Slide 5
Cell-WiFi Convergence Standards
• 3GPP GAN (generic access network),
a.k.a. UMA (unlicensed mobile access)
– An extension of GSM/GPRS
3GPP* to Wi-Fi
– Alcatel, AT&T Wireless,
British Telecom, Cingular,
Ericsson, Kineto Wireless,
Motorola, Nokia, Nortel Networks,
O2, Rogers Wireless, Siemens,
Sony Ericsson, T-Mobile US
• 3GPP WLAN Interworking
IMS (internet multimedia subsystem)
– VoIP, SIP-based
– SBC, Bellsouth, Cingular, Rogers (Canada), Telefonica (Mexico), Mobilkom (Austria),
GoMobile (Malta), Azaire Systems
• FOMA (freedom of mobile multimedia access)
– NTT DoCoMo N900iL dual-network 3G FOMA / WiFi handset
– SIP-based
• SCCAN (seamless converged communication across networks)
– First converged system, SIP-Based
– Proxim, Motorola, Avaya
*3rd Generation Partnership Project
Slide 6
Voice Quality
• ITU-T Voice Quality Standards
– MOS (mean opinion score)
uses a wide range of human
subjects to provide a subjective
quality score (ITU-T P.800)
– PESQ (perceptual speech quality measure)
sends a voice pattern across a network and
then compares received pattern to the original
pattern and computes the quality rating (ITU-T P.862)
– E-Model
computes Rating Factor or R-Factor as a function
of delay and packet loss;
R-Factor directly translates into MOS (ITU-T G.107)
Slide 7
ITU-T PESQ Model
Slide 8
ITU-T E-Model (G.107)
Slide 9
E-Model Parameters
Parameter Abbr. Unit Default Value Permitted Range
Send Loudness Rating SLR dB +8 0 … +18
Receive Loudness Rating RLR dB +2 -5 … +14
Sidetone Masking Rating STMR dB 15 10 … 20
Listener Sidetone Rating LSTR dB 18 13 … 23
D-Value of Telephone, Send Side Ds - 3 -3 … +3
D-Value of Telephone Receive Side Dr - 3 -3 … +3
Talker Echo Loudness Rating TELR dB 65 5 …65
Weighted Echo Path Loss WEPL dB 110 5 ... 110
Mean one-way Delay of the Echo Path T ms 0 0 … 500
Round-Trip Delay in a 4-wire Loop Tr ms 0 0 … 1000
Absolute Delay in echo-free Connections Ta ms 0 0 … 500
Number of Quantization Distortion Units qdu - 1 1 … 14
Equipment Impairment Factor Ie - 0 0 … 40
Packet-loss Robustness Factor Bpl - 1 1 … 40
Random Packet-loss Probability Ppl % 0 0 … 20
Circuit Noise referred to 0 dBr-point Nc dBmOp -70 -80 … -40
Noise Floor at the Receive Side Nfor dBmp -64 -
Room Noise at the Send Side Ps dB(A) 35 35 … 85
Room Noise at the Receive Side Pr dB(A) 35 35 … 85
Advantage Factor A - 0 0 … 20
G.107 – Default values and permitted ranges for the parameters
Slide 10
R-Factor to MOS Conversion
Slide 11
Voice over Wi-Fi Use Cases
• SOHO
– WiFi or WiFi-Cell handset, soft phone
– Service providers: Vonage, ATT CallVantage, Net2Phone
– No handoff, low capacity, basic security
– Range
• Enterprise
– WiFi or WiFi-Cell handset, soft phone
– Range
– Handoff speed
– Call capacity
– Security
• Public Access (Wi-Fi Hotspots)
– WiFi-Cell handset, soft phone
– Range
– Call Capacity
– Roaming issues
Slide 12
Voice Network Requirements
• Client
– Range
– Power-save
– Fast roaming
• Infrastructure
– Call capacity
– Delay, jitter,
packet loss
– Power-save
– Fast roaming
Slide 13
IEEE 802.11 Standards for VoWiFi
• Minimize bursty packet loss by controlling roaming time
802.11r Fast Roaming
802.11k Radio Resource Measurement (RRM)
802.11i Pre-authentication
• Manage power consumption
802.11e Power-save
• Maintain isochronous nature of voice packet
streams by controlling delay, jitter and packet loss
802.11e Quality of Service (QoS) prioritization
Slide 14
IEEE 802.11 Standards
802.11s 802.11p
Mesh WAVE
802.11r 802.11u
Fast Roam WIEN SG
802.11h
DFS & TPC 802.11e 802.11v
QoS WNM
802.11 (’99) 802.11i
MAC + Security ADS SG
802.11k
2Mbps PHY
RRM 802.11n
802.11f
High APF SG
MAC Inter AP
Throughput 802.11T
802.11m
(>100 Mbps) Test
Maint
PHY 802.11a (’99) Methods
54 Mbps
5GHz PHY Study
802.11b (’99) 802.11g Groups
11 Mbps 54 Mbps Current
2.4GHz PHY 2.4GHz PHY
Work
Published
Slide 15
QoS Protocols
• WMM (Wireless MultiMedia)
– Subset of 802.11e
– Specifies 4 priorities: voice (highest), video, background, best effort (lowest)
– Wi-Fi certification has started in September 04
• AC (Admission Control)
– Subset of 802.11e
– Enables APs to reject calls when call capacity is reached
• WMM SA (WMM Scheduled Access)
– Subset of 802.11e
– Specifies polling mechanism to optimize bandwidth utilization
– Wi-Fi certification expected in July 05
• APSD (Automatic Power Save Delivery)
– Subset of 802.11e
– Unscheduled APSD works with WMM
– Scheduled APSD works with WMM-SA
• SVP (Spectralink Voice Priority)
– Proprietary prioritization scheme that combines priority mechanism with tight synchronization
to optimize bandwidth utilization
Slide 16
Security Protocols
• WPA
– Subset of 802.11i
– TKIP, WEP
– EAP-TLS, PSK (pre-shared key)
– WPA Enterprise (TLS, PSK)
– WPA Personal (PSK)
• WPA2
– Entire 802.11i
– Superset of WPA
– AES support
– Pre-Authentication to help with 802.11r fast roaming
Slide 17
WFA and IEEE Timeline
IEEE 11e 11k 11r 11n
Standards 11p 11s
Core Wi-Fi 802.11a+b+g (basic wireless LAN IEEE Specs)
Functionality
802.11h+d
WMM (802.11e-based QoS)
WMM Scheduled Access
Multimedia
Enablers WMM Power Save
Consumer Electronics Framework phase 1
Consumer Electronics Framework phase 2
Wi-Fi Protected Access (WPA) (802.11i-based Security)
WPA2 (full 802.11i)
Security Extended EAP (Enterprise)
Public Access
Simple Configuration
Voice & Data Wi-Fi Cellular Convergence (WCC)
Convergence
Voice over Wi-Fi
Q1/05 Q2/05 Q3/05 Q4/05 Q1/06 Q2/06 Q3/06 Q4/06
Slide 18
15 minute break
Slide 19
Azimuth Automated
Test Family
AAT
Slide 20
Azimuth Automated Tests - AAT
A Systematic Approach to Automation
Precise control of physical Repeatability and real
layer to effect motion device performance
algorithms and motion
protocol
Motion
Control Isolation
Automated
User programmable Azimuth configuration and
DUT Control
multi-personality testMAC™ control and
SDKs
MAC automatic reporting
TCL testMAC™
testMAC™
Automation Programmable
Programmable
Engine Extensions
Extensions
Programmatic control of every User programmable packet-
test system and DUT function level, protocol engine
Slide 21
Azimuth Automated Tests
Automated Turnkey Test Solutions
Azimuth W-Series
Test Platform
Performance Tests Certification Tests
Data
Voice Applications Wi-Fi Alliance Cisco CCX
Application
Infra- Infra-
Client Handset Client and AP Client
structure structure
Rate Vs. Client Range Vs. Call Capacity 11a, b, g with WPA CCX Version 3
Range Capacity MOS WPA2
Roaming Assoc. Roaming WMM
Performance
Forwarding
Rate
Slide 22
VoWiFi Call Capacity Test
Test Description: Capacity vs. QoS Performance of Wi-Fi infrastructure
Features
• Emulate voice traffic from multiple
phones
• Emulate background data traffic
• Measure delay, jitter and packet
loss* vs. number of calls and
background traffic
Test Diagram Test Results
* Per IETF RFCs 2544, 2889
Slide 23
VoWiFi Call Capacity Test
Test Description: Capacity vs. QoS Performance of Wi-Fi infrastructure
Features
• Verifies the ability of infrastructure
to prioritize voice traffic and to
reject or redirect calls when
capacity is reached
• Test APs, wireless switches and
other infrastructure devices
• Measures infrastructure capacity
vs. R-Factor and MOS quality
scores computed from the
measured packet loss rate, delay
+ click to enlarge
and jitter
Test Diagram Test Results
Slide 24
VoWiFi Call Capacity Test
Test Description: Capacity vs. QoS Performance of Wi-Fi infrastructure
Features
• Verifies the ability of infrastructure
to prioritize voice traffic and to
reject or redirect calls when
capacity is reached
• Test APs, wireless switches and
other infrastructure devices
• Measures infrastructure capacity
vs. R-Factor and MOS quality
scores computed from the
measured packet loss rate, delay
+ click to enlarge
and jitter
Test Diagram Test Results
Slide 25
VoWiFi Call Capacity Test
Test Description: Capacity vs. QoS Performance of Wi-Fi infrastructure
Features
• Verifies the ability of infrastructure
to prioritize voice traffic and to
reject or redirect calls when
capacity is reached
• Test APs, wireless switches and
other infrastructure devices
• Measures infrastructure capacity
vs. R-Factor and MOS quality
scores computed from the
measured packet loss rate, delay
+ click to enlarge
and jitter
Test Diagram Test Results
Slide 26
VoWiFi Call Capacity Test
Test Description: Capacity vs. QoS Performance of Wi-Fi infrastructure
Features
• Verifies the ability of infrastructure
to prioritize voice traffic and to
reject or redirect calls when
capacity is reached
• Test APs, wireless switches and
other infrastructure devices
• Measures infrastructure capacity
vs. R-Factor and MOS quality
scores computed from the
measured packet loss rate, delay
+ click to enlarge
and jitter
Test Diagram Test Results
Slide 27
VoWiFi Call Capacity Test
Test Description: Capacity vs. QoS Performance of Wi-Fi infrastructure
Features
• Verifies the ability of infrastructure
to prioritize voice traffic and to
reject or redirect calls when
capacity is reached
• Test APs, wireless switches and
other infrastructure devices
• Measures infrastructure capacity
vs. R-Factor and MOS quality
scores computed from the
measured packet loss rate, delay
+ click to enlarge
and jitter
Test Diagram Test Results
Slide 28
Handset Range vs. Voice Quality
• Vary path loss between the
phone and the AP
• Measure packet loss, delay
and jitter of a phone-AP
link as a function of path loss
• Use ITU-T E-Model to compute
R-Factor and MOS vs. range
Slide 29
VoWiFi Handset Range Test
Test Description: Quality of Voice Transmission over Wi-Fi as a
Function of Path Loss, Roaming and Traffic Load
Features
• Automatically validates Voice over
Wi-Fi performance in minutes
• Standard benchmark test to analyze
rate adaptation of handset and AP pair
• Shows R-Factor and MOS
performance of real time voice stream
- measures packet loss, delay and
jitter to formulate MOS score
• Precise RF control allows for
measurement of true range
performance based on actual WiFi-
enabled handsets
Test Diagram Test Results
Slide 30
VoWiFi Handset Range Test
Test Description: Quality of Voice Transmission over Wi-Fi as a
Function of Path Loss, Roaming and Traffic Load
Features
• Automatically validates Voice over
Wi-Fi performance in minutes
• Standard benchmark test to analyze
rate adaptation of handset and AP pair
• Shows R-Factor and MOS
performance of real time voice stream
- measures packet loss, delay and
jitter to formulate MOS score
• Precise RF control allows for
measurement of true range
+ click to enlarge performance based on actual WiFi-
enabled handsets
Test Diagram Test Results
Slide 31
VoWiFi Handset Range Test
Test Description: Quality of Voice Transmission over Wi-Fi as a
Function of Path Loss, Roaming and Traffic Load
Features
• Automatically validates Voice over
Wi-Fi performance in minutes
• Standard benchmark test to analyze
rate adaptation of handset and AP pair
• Shows R-Factor and MOS
performance of real time voice stream
- measures packet loss, delay and
jitter to formulate MOS score
• Precise RF control allows for
measurement of true range
+ click to enlarge performance based on actual WiFi-
enabled handsets
Test Diagram Test Results
Slide 32
VoWiFi Handset Range Test
Test Description: Quality of Voice Transmission over Wi-Fi as a
Function of Path Loss, Roaming and Traffic Load
Features
• Automatically validates Voice over
Wi-Fi performance in minutes
• Standard benchmark test to analyze
rate adaptation of handset and AP pair
• Shows R-Factor and MOS
performance of real time voice stream
- measures packet loss, delay and
jitter to formulate MOS score
• Precise RF control allows for
measurement of true range
+ click to enlarge performance based on actual WiFi-
enabled handsets
Test Diagram Test Results
Slide 33
VoWiFi Handset Range Test
Test Description: Quality of Voice Transmission over Wi-Fi as a
Function of Path Loss, Roaming and Traffic Load
Features
• Automatically validates Voice over
Wi-Fi performance in minutes
• Standard benchmark test to analyze
rate adaptation of handset and AP pair
• Shows R-Factor and MOS
performance of real time voice stream
- measures packet loss, delay and
jitter to formulate MOS score
• Precise RF control allows for
measurement of true range
+ click to enlarge performance based on actual WiFi-
enabled handsets
Test Diagram Test Results
Slide 34
VoWiFi Handset Range Test
Test Description: Quality of Voice Transmission over Wi-Fi as a
Function of Path Loss, Roaming and Traffic Load
Features
• Automatically validates Voice over
Wi-Fi performance in minutes
• Standard benchmark test to analyze
rate adaptation of handset and AP pair
• Shows R-Factor and MOS
performance of real time voice stream
- measures packet loss, delay and
jitter to formulate MOS score
• Precise RF control allows for
measurement of true range
+ click to enlarge performance based on actual WiFi-
enabled handsets
Test Diagram Test Results
Slide 35
VoWiFi Roaming Performance
• Emulate motion of the phone
using programmable
attenuators
• Monitor source and destination
Wi-Fi channels and Ethernet
• Derive roaming time from data
captures
Slide 36
Roaming Process
AP1 AP2 AP1
max
Path to AP1
Atten
Path to AP2
min
Test cycle
Data rate transition
t ASSOCIATE
t DATA
t TRANSITION
t SCAN
t ROAM
Last data packet First data packet
before roam after roam
Slide 37
VoWiFi Roaming Test
Test Description: Roaming Performance of Wi-Fi Handsets
Features
• Measures the roaming time and
quality of voice vs. roam
• Detailed breakdown of the roaming
time helps optimize roaming algorithm
• Motion and fail-over roaming
emulation of real world scenarios
• Precise RF control and distributed
monitoring present a wealth of
information on handset performance
as a function of motion
Test Diagram Test Results
Slide 38
VoWiFi Roaming Test
Test Description: Roaming Performance of Wi-Fi Handsets
Features
• Measures the roaming time and
quality of voice vs. roam
• Detailed breakdown of the roaming
time helps optimize roaming algorithm
• Motion and fail-over roaming
emulation of real world scenarios
• Precise RF control and distributed
monitoring present a wealth of
information on handset performance
as a function of motion
Test Diagram Test Results
Slide 39
Sample Roaming Test Results
Fastest roaming client - test results reported by the Azimuth test script
0.2
0.18
0.16 Assoc Resp > Data
0.14 Assoc Req > Resp
Auth Resp > Assoc Req
Roam Time, Seconds
0.12
Auth Req > Resp
0.1
Probe > Auth Req
0.08 Data > First Probe
0.06
0.04
0.02
0
1 2 3 4 5 6 7
Roam Number
Slide 40
Cell-WiFi Handover Test
Complete WiFi and
VoWiFi
performance,
conformance
and certification
QoS vs. WiFi-to-cell
handover
Cellular test
Slide 41
Wi-Fi is Proliferating
• Environments
– SOHO
– Enterprise
– Public access
– Outdoor campus
– MAN/WAN
– Transportation
– Military
• Applications
– Data
– Voice
– Video
Slide 42
VoWiFi Call Capacity
Slide 43
VoWiFi Call Capacity
Slide 44
VoWiFi Call Capacity
Slide 45
VoWiFi Call Capacity
Slide 46
VoWiFi Call Capacity
Slide 47
VoWiFi Handset Range
Slide 48
VoWiFi Handset Range
Slide 49
VoWiFi Handset Range
Slide 50
VoWiFi Handset Range
Slide 51
VoWiFi Handset Range
Slide 52
