1. USE OF
SUPRASEGMENTAL FEATURES
PRESENT IN LP RESIDUAL
FOR
AUDIO CLIP CLASSIFICATION
- Anvita Bajpai
anvita@mailcity.com
Applied Research Group
Satyam Computer Services Ltd.
Bangalore

2. Exploding information
•Recent studies show
that most of the stored
data is in the form of
multimedia.
•Large volume of
multimedia data makes
it difficult to handle it
manually
•Need to have an 1 hr of TV broadcast across the world is 100 Petabyte.
automatic method to
Source: http://www.sims.berkeley.edu/research/projects/how-much-
organize and use it info/summary.html#tv
appropriately.
1

3. Audio indexing
Audio classification - An
Reason of choosing audio data
●
important step in building an
for study
audio indexing system
Easier to process
–
An audio indexing system
Contains significant information
–
Indexing – method of
●
organizing data for further
search and retrieval.
Example – book indexing
Audio Indexing – indexing
●
non-text data using audio
part of it
Source: J. Makhoul et. al. “Speech and language technologies for audio
indexing and retrieval”, in Proc. of the IEEE, 88(8), pp. 1338-1353, 2000.
2

4. Audio clip classification
Closed set problem
●
To classify a given audio clip in one of the following
●
predefined categories
Advertisement, Cartoon, Cricket, Football, News
–
Issues in audio clip classification
●
Feature extraction
–
Effective representation of data to capture all significant properties of audio for
●
the task
Robust under various conditions
●
Classification
–
Formulation of a distance measure and rule/models
●
Training a models for the task
–
Testing – actual classification task
–
Combining evidences from different systems
–
5

5. Levels of information in audio signal
Subsegmental information
●
Related to excitation source characteristics
–
Segmental information
●
Related to system / physiological characteristics
–
Suprasegmental information
●
Related to behavioural characteristics of audio
–
3

6. Missing component in existing
approaches and it's importance
Features derived based on spectral analysis
●
Carry significant properties of audio data at segmental level
–
Miss information present at subsegmental, suprasegmental level
–
Perceptually significant information in linear prediction
●
(LP) residual of signal
Complimentary in nature to the spectral information
–
Suprasegmental information not being used in current systems
–
4

7. Presence of audio-specific
Residual
Original
information in LP residual
Aa_res.wav
Aa1.wav
Aa1.wav
6

8. uprasegmental information in Hilbert
nvelope of LP residual of audio signal
7

9. Suprasegmental information in LP
residual for audio clip classification
8
Autocorrelation samples of Hilbert envelope of LP residual for 5 audio classes

10. Statistics of autocorrelation sequence
9
Correction – here we have statistics of autocorrelation sequence peaks of HE (not LP residual)

11. Classification results based on
suprasegmental features using SVM
# of clips correctly classified
Audio Class
(out of 20 clips for each class)
Advertisement 11
Cartoon 19
Cricket 16
Football 04
News 10
11

12. Conclusions and future work
Need to organize multimedia data because of its large volume and
●
need in real-life applications
Shown presence of audio-specific suprasegmental information in
●
LP residual, and its Hilbert envelope
Statistics of autocorrelation sequence of Hilbert envelop is shown
●
to enhance these features
Demonstrated the use of SVM to classify audio based on variance
●
of autocorrelation sequence of Hilbert envelop
Need to extend the framework for other audio indexing
●
applications
Need to explore methods to combine the suprasegmental
●
information to the systems based on segmental and subsegmental
features, for the audio clip classification task
(though little far..) Building a multimedia indexing system
●
12

13. Publications
Anvita Bajpai and B. Yegnanarayana, “Exploring Suprasegmental Features using LP Residual
1.
for Audio Clip Classification”, Workshop on Image and Signal Processing (WISP-2007), IIT
Guwahati, India, 28-29 December 2007.
Anvita Bajpai, “HTB Security Administration using UMX”, An Oracle Technical White Paper,
2.
March 2006
Anvita Bajpai and B. Yegnanarayana, “Audio Clip Classification using LP Residual and Neural
3.
Networks Models”, European Signal and Image Processing Conference (EUSIPCO-2004),
Vienna, Austria, 6-10 September 2004
Anvita Bajpai and B. Yegnanarayana, “Exploring Features for Audio Indexing using LP Residual
4.
and AANN Models”, accepted for The 17th International FLAIRS Conference (FLAIRS - 2004),
Miami Beach, Florida, 17-19 May 2004.
Anvita Bajpai and B. Yegnanarayana, “Exploring Features for Audio Clip Classification using
5.
LP Residual and Neural Networks Models”, International Conference on Intelligent Signal and
Image Processing (ICISIP-2004), Chennai, India, 4-7 January 2004
Gaurav Aggarwal, Anvita Bajpai and B. Yegnanarayana, “Exploring Features for Audio
6.
Indexing”, in Indian Research Scholar Seminar (IRIS-2002), Indian Institute of Science,
Bangalore, India, March 2002
Anvita Bajpai, “State of the art in Web Design and Content Creation for Indian Languages”, in
7.
National level Workshop on Translation Support Systems (STRANS-2001) IIT Kanpur, February
2001
Anvita Bajpai, “Web Developmental Issues – A Case Study of GITASUPERSITE”, in National
8.
level Workshop on Building Large Websites, IIT Kanpur, November 2000