Newcastle Assessment of Pitch Discrimination

Document Sample
Newcastle Assessment of Pitch Discrimination Powered By Docstoc
					Newcastle Assessment of
  Pitch Discrimination
                                 User manual
                                      Michael Drinnan
                                       6th May 2012

                      With great thanks due (in alphabetical order) to:

                                        Paul Carding



                               We hope you find NeAP useful.

For the latest version of software, advice, improvements, bug reports or anything else, please
                             contact me on one of these addresses:
Contents
Installation and preparing the audio files .................................................................................................................. 2
   Installation ............................................................................................................................................................. 2
   Checking the program has installed ...................................................................................................................... 2
Preparing the rating session....................................................................................................................................... 3
   Defining a protocol file ...................................................................................... Error! Bookmark not defined.3
   Specifying frequencies directly .......................................................................... Error! Bookmark not defined.3
   Specifying frequencies in scientific pitch notation................................................................................................ 3
   Specifying parts of semitones in scientific pitch notation ..................................................................................... 3
   Specifying the waveform ...................................................................................................................................... 3
   Specifying the amplitude ....................................................................................................................................... 3
   Specifying the duration ......................................................................................................................................... 3
   Select the protocol file........................................................................................................................................... 4
   Select the default waveform parameters................................................................................................................ 5
   Select the rating options you want to use .............................................................................................................. 5
   Type a name to identify the rating session ............................................................................................................ 5
Starting the rating session ......................................................................................................................................... 6
After the rating .......................................................................................................................................................... 7
   End the rating session ............................................................................................................................................ 7
   Look at the results ................................................................................................................................................. 7
   How does the program deal with the results file? ................................................................................................. 7
Loading the data into Excel ....................................................................................................................................... 8
The file format........................................................................................................................................................... 9
Analysing the data ....................................................................................................................................................10
   Simple measures of agreement .............................................................................................................................10
   Kappa-like measures of agreement ......................................................................................................................10
   Unrated samples ...................................................................................................................................................10
Acknowledgements ..................................................................................................................................................10




                                                                          Page 1
Installation and preparing the audio files
The Newcastle assessment of Pitch Discrimination is a mini-application for assessing a rater’s ability to
discriminate between the pitch of two tones by repeated comparison. It is very simple to use.

Installation
You should only need to install the program once, using the enclosed self-installation program. Click on the file
setup.exe and just follow the instructions. NOTE: The program uses version 3.5 of Microsoft’s .NET framework.
This comes bundled with Vista, but might not be installed if you’ve got an earlier version of Windows.

If the program doesn’t run and doesn’t offer to help you, the easiest way is to simply type: “.NET framework 3.5”
into Google, follow the links and download the .NET framework from Microsoft’s website. It is perfectly safe to
install, and there is no cost.

For obvious reasons, you also need a sound card fitted to your computer.


Checking the program has installed
Run the program. All being well, you’ll be faced with a screen like this:




                                                 Page 2
Writing the test protocol
First, you need to define the test protocol in a standard text file; use Notepad or any other text editor. Each line in
the file defines one pitch test question that will be posed to the rater. The simplest way is to type the frequencies
of the two tones, separated by a space or tab character, for example:
       440.0       466.2
Would give you frequencies of 440 Hz which is concert A, and 466.2 Hz which is the B♭ above concert A.

Specifying frequencies in pitch notation
You can also use pitch notation, where you name a musical note, followed by a digit giving the octave. You can
use lower-case b for the flat symbol, or the hash for the sharp symbol, giving the following set of 12 semitones.
    C       C# Db        D       D# Eb       E       F        F# Gb       G       G# Ab       A       A# Bb        B
Note that:
     A note in one octave is always exactly double the frequency of the same note in octave below it.
     The notes belong to an equal-tempered (Western) scale, where the proportional change in frequency
        between two consecutive notes is always the same, given by:                 . By contrast, some scales are
        arranged so that some notes are in a pleasing ratio. Typically, fifths (7 semitones) are in the ratio 3:2.
     Each octave starts with the note of C, and so for example C4 is considerably lower than B4.
        The # note is exactly the same as the following ♭ note; which you use is up to you.
        B#, Cb, E# and Fb aren’t there. If you’re not sure why, look at a piano keyboard.

The following digit specifies the octave, where C4 is middle C. You can go down to C0 which is about 16 Hz and
probably below the range of your loudspeakers, or up to B10 which is 31 kHz and only much use for dogs. To
play the same tones as the earlier example, you could use either of the following:
      A4 Bb4
      A4 A#4
Note again that each octave begins with C, and so C4 is a lower frequency than A4.

Fractions of semitones in pitch notation
When you’re using pitch notation, you can add a decimal part to the end. This decimal relates to a proportion of a
semitone. For example:
      A4 A4.1
This will play concert A, followed by the tone one-tenth of a semitone higher. You can use any number of decimal
places, but anything beyond 2 is probably needless precision. The first two decimal places will correspond to
cents, or hundredths of a semitone, which is the smallest denomination in practical use.

Using scientific pitch
There are two conventions for describing pitch: concert pitch where A4 = 440 Hz is most common, but some
workers use scientific pitch where C4 = 256 Hz. Optionally, you can add scientific or concert to the line to specify
one or the other. Otherwise, you will get the default which is set on the front page.

Specifying the waveform
Optionally, you can add one of: sine, square, triangle, sawtooth to the line. This will force the tone to be played
using the specified waveform. Otherwise, you will get the default which is set on the front page. You will
normally want sine.

Specifying the amplitude
Optionally, you can add a field of the form: XXdB, where XX is a number in the range 0 to -60. This will attenuate
the tone by the specified amount, where 0dB corresponds to the maximum loudness. Since the decibel scale is
logarithmic, each -6dB corresponds to a halving of the amplitude, and -20dB corresponds to a factor of 10.

Specifying the duration
Optionally, you can add a field of the form: XXs, where XX is a number in the range 0.1 to 5.0. This will play the
tone for the specified number of seconds. For example, to play the same tones as before using a square wave at
amplitude -20dB and for 3s duration:
      A4 Bb4 square           -20dB        3.0s




                                                   Page 3
Some example protocol files
Here are some examples. They are really just for illustration, and would be quite predictable for a rater. If you
actually used these, then you’d probably want to pick the present the pitch samples in random order option.

                     FILE CONTENTS                                                 DESCRIPTION
C4          G4
C4          F#4
C4          F4
                                                             7 pairs of tones, starting 7 semitones apart but with the
C4          E4                                               gap diminishing by one semitone after every question.
C4          D#4
C4          D4
C4          C#4

C4          C#4
                                                             4 pairs of tones, starting 1 semitone apart but with the
C4          C4.5                                             pitch spacing being halved after every question.
C4          C4.25
C4          C4.125

C2          C#2                                              5 pairs of tones, all 1 semitone apart. The frequency
C3          C#3                                              doubles after every question, and this design could be
C4          C#4                                              used to assess how pitch discrimination depends on
                                                             absolute frequency.
C5          C#5
C6          C#6

C4          C#4          0dB                                 5 pairs of tones, all 1 semitone apart. The sound level is
                                                             reduced by 10dB after every question. A design like
C4          C#4          -10dB                               this might be used to assess the effect of sound level on
C4          C#4          -20dB                               pitch discrimination.
C4          C#4          -30dB

C4          C#4          1.0s                                5 pairs of tones, all 1 semitone apart. The duration is
C4          C#4          0.5s                                reduced after every question. This idea can be used to
C4          C#4          0.3s                                assess the effect of duration on pitch discrimination.
C4          C#4          0.1s

C4          C#4          square      concert      1.0s
                                                             As before. This time, all samples will be played using a
C4          C#4          square      concert      0.5s       square waveform in concert tuning.
C4          C#4          square      concert      0.3s
C4          C#4          square      concert      0.1s

If you want to try them out, then just copy and paste the lines into a blank text file, save the file, and away you go.
Note that you can leave completely blank lines, but anything else will be interpreted.




                                                   Page 4
Starting the rating session
Select the protocol file
Using the choose test file button, select the protocol file you just created. All being well, this will be loaded and an
appropriate message will appear beneath.


Select the default waveform parameters
You can set a default waveform, amplitude and duration. These are the values used for tones where the protocol
file doesn’t specify anything different. If the protocol file specifies a waveform, amplitude or duration for a
particular pair of tones, then this will take precedence.


Select the rating options you want to use
Most of these are self–explanatory. It’s probably easier to have a trial run and just watch what happens when you
pick the various options, but here is a short explanation of each (on the next page is a picture that might help guide
you).

   If you tick the first box, then the notes are presented in scientific pitch where C4 = 256 Hz. Otherwise, the
    default is concert pitch where A4 = 440 Hz and C4 = 261.6 Hz. Note that every note is affected equally, and
    so the proportional change between pairs of notes is not affected. This option has no effect on tones that are
    specified directly by a numeric frequency.

   If you tick the first box, then the questions are presented in a different random order each time you run the
    rating test. This would be useful if you are giving the same test to the same person on multiple occasions to
    assess test-retest agreement, for example. Otherwise, the questions are presented in the order they appear in
    the protocol file. Note that unless you tick the next box, the order of the two tones in each question will not be
    affected.

   If you tick the second box, then the two tones in each question are played in a random order.

   If you tick the third box, then the rater will have a third rating option both the same. It’s up to you whether or
    not any of the tones in the protocol actually are the same.

   The fourth tick box lets the rater play the tones as many times as needed to make a judgement. If this box
    isn’t ticked, then each pair of tones is played just once.

   Under normal circumstances, the results file uses 0 = no response, 1 = sample 1 higher, 2 = sample 2 higher,
    and 3 = both the same. The fifth tick box will introduce a minus sign if the rater gets the answer wrong. For
    example, -2 means that rater thought tone 2 was higher, but this was incorrect.


Type a name to identify the rating session
In the box near the bottom of the screen, type a string of text to identify the rating session. This will appear in the
results table, to help you sort out which results came from which rater.

You CAN use the same name more than once, but you’ll be warned. If you do, then you will have the time and
date of the test to help figure out what is what.




                                                   Page 5
Starting the rating session
If you’ve done everything correctly, it should look a bit like this:




Click on the start button, and the rating window appears like in the picture below.




The rest is easy. Once you’ve pressed play, the two tones are presented and the rater just clicks the button
according to which is higher in pitch. Here’s another screen shot from a different test where the ‘allow the two
audio samples to be rated equal’ and the ‘allow the rater to play the samples again’ options are ticked. Simple.




                                                   Page 6
After the rating
End the rating session
After all the questions have been presented, the test session will finish. The results will be saved automatically,
and you can go straight on to another rating session.

NOTE: If you want to abandon a rating session before it has finished, then click the windows close button as
normal. The results to date will be saved.


Look at the results
The results are saved in a file with the same name as your protocol file, except the txt extension is replaced with
CSV. For example, results from a protocol file called ENTdata.txt would be written to a results file called
ENTdata.csv in the same folder as your original protocol file.

The CSV file can be opened straight into Microsoft Excel, or just about any text editor or statistics package.


How does the program deal with the results file?
   First, the program checks to see if the results file already exists in the same folder as your test samples. If not,
    it is created new and a header is inserted. The first row of the header reproduces the current contents of the
    protocol file, so that the software can check if the protocol file has been changed.

   If the results file DOES exist, the program reads first line of the file, which would have been written when the
    file was created as explained above. It compares this header with the current contents of the protocol file. If
    they are different then the protocol has been changed and the results file must be cleared; this is a precaution
    to make sure a single results file always relates to the same protocol.

   If the protocol has changed, the program will give you the opportunity to save the existing results under a
    different name.

   Then you perform the rating task.

   After the rating task, the program will add the new results to the end of the results file as a new row.

   Normally, that’s the end of the story but occasionally the program fails to write the new results. This is
    probably because you’ve got the file open in another program. For example, Excel will lock the file and stop
    any other program from accessing it. In this case, you are given the opportunity to save the results as a new
    file. Be aware this is a one-off; the next set of results will go back to the original file. So - if you have a
    problem, you need to sort it out immediately. Otherwise, you will have some results in one file, and some in
    another.




                                                   Page 7
Loading the data into Excel
Your results are stored in a CSV (comma separated value) file, which is a text format and can be read by any text
editor, word processor, spreadsheet or statistics package. However, it is probably best suited to Microsoft Excel
for initial viewing. Here’s how:

If you use the point (.) as your decimal separator (99.9)
This applies in most English-speaking countries. You should just be able to double-click on the file in Windows
Explorer and it will open properly in Excel.

If you use the comma (,) as your decimal separator (99,9)
This is the case in some European countries: Germany, the Netherlands, Italy, parts of Scandinavia, etc. If so, then
the comma might be misinterpreted if you open the file directly. It seems to depend on how you have your
Regional and Language settings in the control panel.

If the import doesn’t work as you expect, you will need to import the data with explicit settings to use a comma
delimiter. In Excel 2010 and I believe also Excel 2007, you do it using the Data -> From Text command like
this:




Select the file you want to import, then pick a Delimited file, and a Comma delimiter like this:




Notice that in the second box you can see how Excel has parsed your data into columns. This should give you a
good idea whether it is going to work or not. If so, the other settings can all be left as they are.




                                                 Page 8
The file format
Here’s how your data might look in Excel:

           RATER               DATE       TIME       SOURCE       D4 C4   C4.0 C4.9   C4.8 C4.1   C4.2 C4.7   C4.6 C4.3   C4.4 C4.5

    Sine 0.0dB 1.0s YnYnYn   07-May-12   11:59:26   C:\neap.txt     1        2           1           2           1           2

            Anne             07-May-12   12:00:11   C:\neap.txt     1        2           1           2           1           2

            Brian            07-May-12   12:04:53   C:\neap.txt     1        2           1           1           1           3

            Claire           07-May-12   12:23:42   C:\neap.txt    -2        -3          -2          2           1           0




      The file C:\neap.txt defined the protocol. Therefore the data we are viewing in Excel must be from the file
       C:\neap.csv.

      The first (header) row is a copy of the protocol, where each column heading from D4 C4 onwards reproduces
       one line in the protocol. In this case, the protocol is a series of tones with decreasing intervals between them.
       This row is also used by the software to determine when the protocol has changed.

      The second row gives the correct answer to each question, which of course can be worked out by the
       software. It is dated according to the time when the file was created. It uses the following code:
           1 = sample 1 higher          2 = sample 2 higher        3 = both the same

      The unusual rater name in the second row codes up the default settings when the file was created. If these
       have changed on a subsequent use of the file, then the test parameters have changed and the software will
       prompt the user.

      Three rating sessions had been conducted to date, with the dates and times as shown. The same coding is used
       to represent the responses, with the addition of 0 = no response, indicating that the test was ended before the
       question was answered.

      Anne got all the answers right.

      Brian got questions 4 and 6 wrong. He rated the samples in Q6 as both the same.

      In Claire’s data, the minus sign is used to indicate a wrong answer; that is, when her answer didn’t agree with
       the model answer in the top row. She got the first three questions wrong, the next two right, and didn’t answer
       the final question.
       NOTE: This would have been picked up by the program because the rating options have been changed. You
       get the idea though.




                                                            Page 9
Analysing the data
Simple measures of agreement
The simplest way of measuring performance is just to count the number of correct answers, which is easy in Excel
if you’ve used the minus sign to indicate a wrong answer.

Suppose your data from the previous page are in the top corner of an Excel spreadsheet, starting at cell A1. Then
to calculate how many correct answers in Claire’s row, put this formula into a nearby cell:

         =COUNTIF( E5:J5, ">0" )

Note that it won’t work on the other rows, because the minus sign hasn’t been used to indicate a wrong answer.
For these, you need to compare the actual response with the model answer at the top, which is a bit more
complicated.

You can then express the number of correct answers as a proportion of the number of questions, 6 in this case.

Kappa-like measures of agreement
The Kappa statistics correct agreement for chance, so that if you do no better than chance you get a score of 0.
The general formula is:




But in our case, we can estimate the expected chance agreement is 0.5 for the higher/lower test, or 0.333 for the
higher/equal/lower test. This would be true assuming that the rater is actually trying to do the test properly, and
doesn’t always guess the same response. Even if they did, it would still be true so long as the correct responses
were distributed evenly between the available options. So for the higher/lower test:



Or for the higher/equal/lower test:




Unrated samples
Any unrated samples will all be given the value zero. For most analysis, this is NOT the same as missing data.
The zeros might be useful in some circumstances, for example to create a histogram showing missing data.
HOWEVER in most cases you will need to replace the zeros with a blank, or whatever your own statistics
software uses to indicate missing data. This is a particular problem if your test gets more difficult as time goes on,
because then the data are not missing at random.




Acknowledgements
Thanks to the following people for helping with the development, finding bugs or suggesting improvements:

         Paul Carding.




                                                 Page 10

				
DOCUMENT INFO
Shared By:
Categories:
Tags:
Stats:
views:5
posted:9/29/2012
language:Unknown
pages:11