Standards Project by 25r4r3


									Telecommunications Industry Association                           TR41.3.12-11-08-006-L

                               Document Cover Sheet

Project Number    PN-3-XXXX (leave blank if unknown or not applicable)

Document Title    Evaluation of Answering machine voice quality in remote access
Source            Embarq
                  Local Access Solution Development
                  6050 Sprint Pkwy
                  Overland Park, KS-66251

Contact           Amar Nath Ray                 Phone:       913-794-6691
                  6050 Sprint Pkwy              Fax: 913-794-0412
                  Overland Park, KS-66251
                                                Phone:      913-794-6430
                  C.V. Chakravarthy             Fax: 913-794-0412
                  EMBARQ                        Email:Cadathur.Chakravarthy@mail.sp
                  6050 Sprint Parkway 
                  Overland Park, KS, 66251
Distribution      TR-41.3.12
Intended            X    For Incorporation Into TIA Publication
Purpose of               For Information
Document                 Other (describe) -
(Select one)
The document to which this cover statement is attached is submitted to a Formulating
Group or sub-element thereof of the Telecommunications Industry Association (TIA) in
accordance with the provisions of Sections 6.4.1–6.4.6 inclusive of the TIA Engineering
Manual dated March 2005, all of which provisions are hereby incorporated by reference.
Telecommunications Industry Association                            TR41.3.12-11-08-006-L

Abstract: The voice qualities of several digital answering systems connected to
      real voice network were checked remotely using a digital speech level
      analyzer. The mean opinion scores (MOS) as measured by this test were
      found to be lower than PSTN voice quality.

Test set up:
The following diagram shows the experimental set up used for this test.

      Remote access DTAM voice quality measurement set up

                                                          Send speech
                                          Telephone                         DSLA
                                           Network       Retive store
        with DTAM

Test set up sequences:
   1. DSLA calls the DTAM
   2. DTAM goes off-hook after three rings
   3. DSLA sends message after waiting a certain period of time (for DTAM to send
      OGM and get ready for recording messages)
   4. DSLA goes on-hook and waits for specified time
   5. DSLA calls DTAM again
   6. DSLA waits for a certain period of time (for DTAM to send OGM) and sends
      remote access code
   7. DSLA waits a certain period of time and sends DTMF code for retrieval of
   8. DSLA waits for some time (critical as discussed later in this document) and start
      PAMS measurements
   9. DSLA goes on-hook
Telecommunications Industry Association                       TR41.3.12-11-08-006-L

The screen shot given below is an example of the results as measured by the
Perceptual Analysis Measurement System (PAMS). The results shows that the end to
end remote access voice quality is lower than that of PSTN as shown later in this

The parameters for the measurement above are given below. The time delay of 508ms
(5ms for DSLA) was the minimum out of the measurements made.

Listening Effort – 4.11
Listening Quality – 3.63

Time Offset – 513 ms
Time Offset Confidence – 67%
The table below is showing the parameter levels for speech quality measurements with
five DTAMs from various vendors.

Sample number Listening effort Listening Quality One way delay
A                 3.7            3.2               612 ms
B                 4.0            3.5               520 ms
C                 4.1            3.6               508 ms
D                 4.0            3.4               575 ms
E                 3.7            3.4               565 ms
Telecommunications Industry Association                       TR41.3.12-11-08-006-L

The time duration after the remote access code followed by the message playback code
is crucial for good MOS score.
The table below shows the degradation of the MOS scores with the time duration for a
specific device.
Time duration      Listening Effort       Listening Quality
1.25               4.0                    3.5
1.5                4.0                    3.5
2                  4.0                    3.6
2.25               4.1                    3.7
2.5                3.9                    3.4
3.0                2.5                    1.0

The PAMS screen shot as given below was measured by the same DSLA system.

The PAMS chart as shown above is giving the best results for a PSTN call using the
same line.
Note: The signal level of the reference signal is important for measuring the MOS
scores. Please look at the message from Nick Fox of Malden, UK.


Because the test takes place in two phases, PAMS needs to know which files to
compare with the replayed speech. "Generating" speech at -99dBm is just a
Telecommunications Industry Association                        TR41.3.12-11-08-006-L

method of achieving this. If the level is increased, the resulting signal gets mixed
with the replayed speech and thus the score is degraded.

I hope this makes sense.

With best regards,


From: Ray, Amar N [LTD] []
Sent: 02 November 2006 21:31
To: Nick Fox
Cc: Ray, Amar N [LTD]; Chakravarthy, C V [LTD]
Subject: RE: DSLA

Me and CV, we have question about the sequence that you have used in the TAM
files. Why you used -99dBM signal level in that file? FYI, we have tried with
signals higher than -99dBM (-40 and -20) and found degraded PAMS scores.

Recommendation: More tests are in progress. The committee will be informed
with more results later on to make specific recommendations for setting the
requirements on DTAM’s remote access speech qualities. At this point, a MOS
level of 3.4 is suggested.

To top