Audio Signal Classification
Rough-Sets based Approach
Outline
• Introduction - the research goals
• Musical instrument acoustics
• Parameters of sounds and their separability
• Preprocessing for rough set tools:
discretization (quantization) of parameters
• Automatic classification and results
• Summary
The Research Goals
• Motivation – to deal with the problem of the
automatic classification of musical data:
– database searching: there is no possibility to find
fragments performed by selected instruments inside
files, unless such information is attached to the file
?
• Aim – to check if it is possible to recognize sounds
on the basis of a limited number of parameters, and
reveal these parameters
Problems
• Amount of data in sound files
1 s, Fs=44.1kHz, 16 bits stereo, 176.4 kB
• Musical instrument sound data are
unrepeatable and inconsistent:
– the sound depends on the articulation, the
instrument itself, arrangement of microphones,
reverberation, etc.
– sounds of different instruments can be similar,
whereas sounds of one instrument may change
significantly within the scale of the instrument
Musical Instrument Acoustics
Bowed String Instruments
• articulation:
– bowed vibrato, muted/not muted,
– pizzicato (string plucked),
• sound:
– body resonances
– inharmonic partials:
where f1 - fundamental (pitch)
– pizzicato: transients only
Woodwind Instruments
• articulation - vibrato/non vibrato
• the length of the horn resonator is reduced by
holes between the mouthpiece and the end
• reed instruments – excited by vibrating reeds :
– single reed: clarinet, saxophone
– double reed: oboe, English horn, bassoon
• flute:
– blowing a stream of air across a hole in the body
Brass Instruments
• articulation: vibrato, muted/not muted
• lip-driven
• mouthpieces only help with tone production
• long narrow body and extended flaring end
- upper modes available
• mechanical valves
Processed Data
• consequent sounds in the musical scale of instruments
• source - CD: McGill University Master Samples
• stereo, sampling frequency 44.1 kHz, 16 bits
Parameterization – Frequency Domain
• Fourier analysis:
• example: oboe, 440 Hz
A partials (harmonics)
f
Calculation Points for Parameters
• The spectrum changes with time evolution
t - starting transient
qs - quasi-steady state
time envelope of an exemplary sound
Parameters of Sound
• fdm – mean frequency deviation for low partials
• hfd_max=1..5 – a partial with the greatest frequency deviation
• A1-2 [dB] – amplitude difference between 1st and 2nd partial,
• h1, h3,4, h5,6,7, h8,9,10, hrest –
energy of the selected partials
• Od, Ev – contents of odd/even partials in the spectrum
•Br – brightness of the sound:
Other Parameters
f 1 [Hz] – fundamental
|f1max– f1min| – vibrato,
dfr – fractal dimension of the spectrum envelope:
where N(r) - minimal number of
squares r covering the envelope,
f1/2 – energy of subharmonic partials in the spectrum
qs, te – proportional participation of the quasi-steady state
and the ending transient in the total sound time
rl – release velocity [dB/s]
Separability of Parameters
• criterion:
Di,j – measure of distances between classes i, j
• Hausdorff metrics
• max/min/mean distance between objects
from different classes
di – measure of dispersion in class i
• mean/max distance between class objects or
from the gravity center of the class
• set of parameters is satisfying if Q>1
Metrics
• definition:
• Euclidean
• “city”
• central
Separability as a Function of Metrics
D1 - Hausdorff metric d1/d2 - mean/max distance
D2/D3/D4 - max/min/mean between class objects
distance between objects d3/d4 - mean/max distance
from different classes from the gravity center
Quantization of Parameters
• inductive learning methods require a small
number of attribute values
• global methods: simultaneously convert all
continuous attributes – large tables
– Boolean approach (Skowron, Nguyen)
– cluster analysis (Chmielewski, Grzymala-Busse)
• local methods: restricted to simple attributes
– methods usually do not discern between points
representing different classes
Exemplary Local Methods
• equal interval width method (EIWM)
• maximum distance method (MDM)
• statistical clusterization
Separability vs. Quantization
Method
Foundations of Rough Set (RS)
Based Systems - 1
Let – a decision table
U - a universe - nonempty, finite set of objects
A - a nonempty, finite set of attributes
, the decision attribute
implies indiscernibility relation IND(B)
reduct - a minimal subset B such that IND(A)=IND(B)
Foundations of RS Based Systems – 2
– lower approximation
of X in A
– upper approximation
of X in A
rough set in A - the family of all subsets of U
having the same lower and upper approximations in A
Foundations of RS Based Systems - 3
- B positive region of A
- the generalized decision in A
B - relative reduct iff B is a minimal subset of A
such that
The relative reduct is such minimal subset of A
which preserves the positive region
Rough Set Based Systems
• generated rules
where n - length of the rule
• a rough measure m of the rule describing concept X
Y – set of all examples described by the rule
Exemplary RS Based Systems
• LERS
– allows unknown attribute values
– possibility of removing inconsistent examples
(i.e. of identical attribute values, but with
different decisions)
– priority of attributes is controlled
• DataLogic
– calculates attribute and rule strength
– quantization of data is available
A Proposed System
• implemented in Mathematica
• allows data quantization with number of
methods, both local and global
• ten-fold test included
• priority of attributes is controlled
• unnecessary attributes found by reducts and
relative calculation
• the use of produced rules available for whole
data sets, not only for singular objects
Exemplary Reducts
reduct relative reduct 1 relative reduct 2
• up to 70% correct recognition obtained in RS tests
• parameters 60,61,62 and 41,44,30,55 are the most significant
Exemplary rules
Summary (1)
• the huge amount of data contained in digital
sound representation requires parametrization as
preprocessing
• a great number of parameters is a consequence
of the variety of musical instruments and
differences in their sounds
• inconsistency of the data implies soft computing
techniques for automatic classification
• quantization is necessary as preprocessing for
RS algorithms
Summary (2)
• an appropriate choice of the quantization
requires many experiments
• rough set algorithms allow the evaluation of
the significance of parameters
• composition of parameters in RS reducts
confirms that the evolution of the sound
must be taken into account during
parametrization
• the use of learning algorithms allows
finding rules for managing classification