SlideShare a Scribd company logo
1 of 25
Download to read offline
Rui Pedro Paiva
DEI
10 de Dezembro de 2008
Music Information Retrieval
Overview & Current Trends
2 Music Information Retrieval @ DEIUndercover
Contents
 Introduction
 Applications
 Short MIR Tale
 Disciplines
 Generic MIR System
 Techniques
3 Music Information Retrieval @ DEIUndercover
Introduction
 Music and Man
• Music expresses “that which cannot be put into words and that
which cannot remain silent” (Victor Hugo)
• We associate music with the most unique moments of our lives
and music is part of our individual and social imaginary
- By listening to music, emotions and memories, thoughts and reactions,
are awakened
• “Life has a soundtrack” (Ana Gomes, “Festivais de Verão”,
Público)
• “The history of a people is found in its songs” (George Jellinek)
4 Music Information Retrieval @ DEIUndercover
Introduction
 Music and World economy
• Music industry runs, only in the USA an amount of money in
the order of several billion US dollars per year.
• Explosion of the Electronic Music Industry (EMD)
- Widespread access to the Internet
- Bandwidth increasing in domestic and mobile accesses
- Compact audio formats with near CD quality (mp3, wma)
- Portable music devices (iPod, mp3 readers)
- Peer-to-peer networks (Napster, Kazaa, eMule)
- Online music stores (iTunes, Calabash Music, Sapo Music) 
resolution is the song, not the CD
- Music identification platforms (Shazam, 411-Song, Gracenote MusicID /
TrackID)
- Music recommendation systems (MusicSurfer)
5 Music Information Retrieval @ DEIUndercover
Introduction
 Music and World economy (cont.)
• By 2005, Apple iTunes was selling  1.25 million songs each
day
- Since the service was launched and until thee beginning of 2005  250
million songs had been sold in total
• Number and dimension of digital music archives continuously
growing
- Database size (these days, over 2 million songs)
- Genres covered
• Challenges to music providers and music librarians
- Organization, maintenance, labeling, user interaction
- Any large music database is only really useful if users can find what
they are looking for in an efficient manner!
6 Music Information Retrieval @ DEIUndercover
Introduction
 Database Organization and Music Retrieval
• Presently, databases are manually annotated  search and
retrieval is mostly textual (artist, title, album, genre)
- Service providers
– Difficulties regarding manual song labeling: subjective and time-consuming,
- Customers
– Difficulties in performing “content-based” queries
• “Music’s preeminent functions are social and psychological”, and so “the most
useful retrieval indexes are those that facilitate searching in conformity with such
social and psychological functions. Typically, such indexes will focus on stylistic,
mood, and similarity information” [Huron, 2000].
•  Music Information Retrieval (MIR) emerges from the necessity to
manage huge collections of digital music for “preservation, access,
research and other uses” [Futrelle and Downie, 2003].
7 Music Information Retrieval @ DEIUndercover
Applications
 Platforms for EMD
• Similarity-based retrieval tools
- Query-by-example
– Music identification (trackID,
Tunatic)
– Music recommendation
• Discovery of new music
– Islands of music
• Metaphor of geographic maps:
similar genres are “physically”
close together
- Automatic playlist generation
- Query-by-melody (query-by-
humming, query-by-singing)
- Plagiarism detection
- Music web crawlers
8 Music Information Retrieval @ DEIUndercover
Applications
 Music education and
training
• Automatic music
transcription
- Music composition,
analysis, performance
evaluation
 Audio software
• Intelligent audio (music)
editors  automatic
indexing
 Multimedia databases and
operating systems
© George Tzanetakis, 2002
9 Music Information Retrieval @ DEIUndercover
Applications
 Video indexing and searching
• Segmentation based on audio (music) content  detection of
scene transitions
 Advertisement and cinema
• Tools for mood-based retrieval
 Sports
• Music to induce a certain cardiac frequency
10 Music Information Retrieval @ DEIUndercover
Short MIR Tale
 Precursors of computer-based MIR: incipit and theme indexes, e.g.,
Harold Barlow and Sam Morganstern’s dictionary of musical
themes
 1966: potential of applying automatic information retrieval
techniques to music was recognized (Kassler)
 1970s and 1980s: automatic music transcription systems
 1990s: surge of interest, mostly in topics such as query-by-
humming (impulse from research on digital libraries)
 2000: 1st International Symposium on Music Information Retrieval
(ISMIR)
11 Music Information Retrieval @ DEIUndercover
Short MIR Tale
 Presently: strongly cross-disciplinary field
• My interests: content analysis and similarity assessment and
retrieval in audio song databases
- Metrics of similarity, music identification, music recommendation, audio
fingerprinting, music classification and feature extraction, tempo and
melody detection, music summarization
12 Music Information Retrieval @ DEIUndercover
Generic MIR System
Client
Internet Ordering of
Results and
Summarization
Comparison
of Signatures
Database Search
Database
Classification
Raw Audio
Recordings
Segmentation
and
Summarization
Feature
Extraction
Signature
Feature Extraction
Query Signature
Visualization
1. Tom Jobim - Corcovado
2. Dido - Thank You
3. João Gilberto - Meia Luz
4. .....
5. .....
Jazz Classical Latin
Swing Baroque Mambo Salsa
.
Interactive Edition
Query Creation
13 Music Information Retrieval @ DEIUndercover
Techniques
 Idea
• Extract semantic information from low-level data
• Feature extraction
- Physical: F0, intensity, centroid, uniformity, rolloff, flux
- Perceptual: pitch, loudness, timbre, beat
- Musicological: notes, melodies, measures, motives, themes
- Higher-level (semantic) features: emotion, genre, instruments, artist
14 Music Information Retrieval @ DEIUndercover
Techniques
 Physical features
• Frequency domain
- DFT, STFT, spectrogram
- F0 detectors
Time
Frequency
0 0.5 1 1.5 2
0
0.5
1
1.5
2
x 10
4
15 Music Information Retrieval @ DEIUndercover
Techniques
 Physical features (cont.)
• Spectral features
- MFCCs, centroid, rolloff, flux, harmonicity, high-frequency content, …
• Sub-band features
© MIR Toolbox, 2008
16 Music Information Retrieval @ DEIUndercover
Techniques
 Physical features (cont.)
• Spectral features + sub-band features (e.g., audio
fingerprinting)
© Haitsma & Kalker, 2002
17 Music Information Retrieval @ DEIUndercover
Techniques
 Physical features (cont.)
• Time domain
- Auditory model-based F0 detectors, beat detectors (energy-based)
0.21 0.22 0.23 0.24 0.25
-0.4
-0.2
0
0.2
0.4
Time (s)
x(t)
a) Sound waveform
Time (s)
Frequencychannel
b) Cochleagram frame
0.21 0.22 0.23 0.24 0.25
20
40
60
80
Time Lag (ms)
Frequncychannel
c) Correlogram frame
0 5 10 15 20
20
40
60
80
0 5 10 15 20 25
0
0.1
0.2
0.3
0.4
Time Lag (ms)
Pitchsalience
d) Summary correlogram
18 Music Information Retrieval @ DEIUndercover
Techniques
 Perceptual features
• Pitch
- Frequency
- Intensity
- Context
- Ear physiology (age)
19 Music Information Retrieval @ DEIUndercover
Techniques
 Perceptual features (cont.)
• Loudness
- Intensity
- Frequency
- Context
- Ear physiology (age)
Fletcher-Munson equal loudness contours
20 Music Information Retrieval @ DEIUndercover
Techniques
 Perceptual features (cont.)
• Timbre
- No physical correlate
- “what something sounds like”:
– Spectral content at steady-sate
• Centroid, rolloff, relative amplitudes of harmonic components, inharmonicty…
– Signal’s temporal envelope
• Attack transient
– Temporal behavior of the harmonics
• Melodic contour
- UDUEEUUD
• Rhythm contour
- LSSLEELS
• Beat
21 Music Information Retrieval @ DEIUndercover
Techniques
 Musicological features
• Notes from audio
0 100 200 300 400 500
4800
4900
5000
5100
5200
5300
5400
Time (msec)
f[k](cents)
a) Eliades Ochoa's excerpt
0 200 400 600 800
6100
6200
6300
6400
6500
6600
6700
6800
Time (msec)f[k](cents)
b) Female opera excerpt
22 Music Information Retrieval @ DEIUndercover
Techniques
 Musicological features (cont.)
• Melody
23 Music Information Retrieval @ DEIUndercover
Techniques
 Musicological features (cont.)
• Themes
© Colin Meek and William Birmingham, 2001
24 Music Information Retrieval @ DEIUndercover
Techniques
 Higher-level features  top-down information flow
© Xavier Serra, 2005
25 Music Information Retrieval @ DEIUndercover
Techniques
 Higher-level features (cont.)
• Bridge the semantic gap
• Memory, context, expectations
- Repetitions, sonic environment, modeling the individual, musicological
knowledge
• Emotion: valence (happy/anxious) and arousal (calm/energetic)
- Classification approaches resorting to low-level features

More Related Content

What's hot

Using mashup technology to improve findability
Using mashup technology to improve findabilityUsing mashup technology to improve findability
Using mashup technology to improve findabilitySten Govaerts
 
IAML Future of music in public libraries
IAML Future of music in public librariesIAML Future of music in public libraries
IAML Future of music in public librariesJohan Mijs
 
20211026 taicca 2 music generation
20211026 taicca 2 music generation20211026 taicca 2 music generation
20211026 taicca 2 music generationYi-Hsuan Yang
 
Learning to Generate Jazz & Pop Piano Music from Audio via MIR Techniques
Learning to Generate Jazz & Pop Piano Music from Audio via MIR TechniquesLearning to Generate Jazz & Pop Piano Music from Audio via MIR Techniques
Learning to Generate Jazz & Pop Piano Music from Audio via MIR TechniquesYi-Hsuan Yang
 
Research on Automatic Music Composition at the Taiwan AI Labs, April 2020
Research on Automatic Music Composition at the Taiwan AI Labs, April 2020Research on Automatic Music Composition at the Taiwan AI Labs, April 2020
Research on Automatic Music Composition at the Taiwan AI Labs, April 2020Yi-Hsuan Yang
 

What's hot (6)

Using mashup technology to improve findability
Using mashup technology to improve findabilityUsing mashup technology to improve findability
Using mashup technology to improve findability
 
IAML Future of music in public libraries
IAML Future of music in public librariesIAML Future of music in public libraries
IAML Future of music in public libraries
 
20211026 taicca 2 music generation
20211026 taicca 2 music generation20211026 taicca 2 music generation
20211026 taicca 2 music generation
 
Music Hack Day Sponsor
Music Hack Day SponsorMusic Hack Day Sponsor
Music Hack Day Sponsor
 
Learning to Generate Jazz & Pop Piano Music from Audio via MIR Techniques
Learning to Generate Jazz & Pop Piano Music from Audio via MIR TechniquesLearning to Generate Jazz & Pop Piano Music from Audio via MIR Techniques
Learning to Generate Jazz & Pop Piano Music from Audio via MIR Techniques
 
Research on Automatic Music Composition at the Taiwan AI Labs, April 2020
Research on Automatic Music Composition at the Taiwan AI Labs, April 2020Research on Automatic Music Composition at the Taiwan AI Labs, April 2020
Research on Automatic Music Composition at the Taiwan AI Labs, April 2020
 

Viewers also liked

Query By humming - Music retrieval technology
Query By humming - Music retrieval technologyQuery By humming - Music retrieval technology
Query By humming - Music retrieval technologyShital Kat
 
CS158: Final Project
CS158: Final ProjectCS158: Final Project
CS158: Final ProjectEvan Casey
 
Query By Humming - Music Retrieval Technique
Query By Humming - Music Retrieval TechniqueQuery By Humming - Music Retrieval Technique
Query By Humming - Music Retrieval TechniqueShital Kat
 
How To Do Music Recommendation
How To Do Music RecommendationHow To Do Music Recommendation
How To Do Music Recommendationmaryrosecook
 
Interactive Music Presentation - Future Music Camp 2010
Interactive Music Presentation - Future Music Camp 2010Interactive Music Presentation - Future Music Camp 2010
Interactive Music Presentation - Future Music Camp 2010digitalmusictrends
 
Writing a Big Data History of Music
Writing a Big Data History of MusicWriting a Big Data History of Music
Writing a Big Data History of MusicDigital History
 
MOODetector: A System for Mood-based Classification and Retrieval of Audio Mu...
MOODetector: A System for Mood-based Classification and Retrieval of Audio Mu...MOODetector: A System for Mood-based Classification and Retrieval of Audio Mu...
MOODetector: A System for Mood-based Classification and Retrieval of Audio Mu...Rui Pedro Paiva
 
BUAD 497 Strategic Management- Spotify
BUAD 497 Strategic Management- Spotify BUAD 497 Strategic Management- Spotify
BUAD 497 Strategic Management- Spotify Vincent Tsao
 
How Beyonce Disrupts The Music Industry
How Beyonce Disrupts The Music IndustryHow Beyonce Disrupts The Music Industry
How Beyonce Disrupts The Music IndustryPatrick Van der Pijl
 
Business model of Spotify
Business model of SpotifyBusiness model of Spotify
Business model of SpotifyAnirban Ghosh
 
The Music Industry - new models (excerpt of a keynote)
The Music Industry - new models (excerpt of a keynote)The Music Industry - new models (excerpt of a keynote)
The Music Industry - new models (excerpt of a keynote)Alexander Osterwalder
 
Tracxn Research - Music Tech Landscape, January 2017
Tracxn Research - Music Tech Landscape, January 2017Tracxn Research - Music Tech Landscape, January 2017
Tracxn Research - Music Tech Landscape, January 2017Tracxn
 
Business Models in the Music Industry
Business Models in the Music IndustryBusiness Models in the Music Industry
Business Models in the Music IndustryCamilla van den Boom
 
The future of music industry
The future of music industryThe future of music industry
The future of music industryStefanoCatracchia
 
The Evolution Of The Music Industry The Effect Of Technology And Law On Stra...
The Evolution Of The Music Industry  The Effect Of Technology And Law On Stra...The Evolution Of The Music Industry  The Effect Of Technology And Law On Stra...
The Evolution Of The Music Industry The Effect Of Technology And Law On Stra...Ben Kilmer
 
The Brief History of the Film - Albert James Burleson
The Brief History of the Film - Albert James BurlesonThe Brief History of the Film - Albert James Burleson
The Brief History of the Film - Albert James BurlesonAlbert James Burleson
 

Viewers also liked (20)

Query By humming - Music retrieval technology
Query By humming - Music retrieval technologyQuery By humming - Music retrieval technology
Query By humming - Music retrieval technology
 
CS158: Final Project
CS158: Final ProjectCS158: Final Project
CS158: Final Project
 
Query By Humming - Music Retrieval Technique
Query By Humming - Music Retrieval TechniqueQuery By Humming - Music Retrieval Technique
Query By Humming - Music Retrieval Technique
 
Music Information Retrieval
Music Information RetrievalMusic Information Retrieval
Music Information Retrieval
 
How To Do Music Recommendation
How To Do Music RecommendationHow To Do Music Recommendation
How To Do Music Recommendation
 
Interactive Music Presentation - Future Music Camp 2010
Interactive Music Presentation - Future Music Camp 2010Interactive Music Presentation - Future Music Camp 2010
Interactive Music Presentation - Future Music Camp 2010
 
Writing a Big Data History of Music
Writing a Big Data History of MusicWriting a Big Data History of Music
Writing a Big Data History of Music
 
MOODetector: A System for Mood-based Classification and Retrieval of Audio Mu...
MOODetector: A System for Mood-based Classification and Retrieval of Audio Mu...MOODetector: A System for Mood-based Classification and Retrieval of Audio Mu...
MOODetector: A System for Mood-based Classification and Retrieval of Audio Mu...
 
Spotify presentation
 Spotify presentation Spotify presentation
Spotify presentation
 
BUAD 497 Strategic Management- Spotify
BUAD 497 Strategic Management- Spotify BUAD 497 Strategic Management- Spotify
BUAD 497 Strategic Management- Spotify
 
Neo4j: Graph-like power
Neo4j: Graph-like powerNeo4j: Graph-like power
Neo4j: Graph-like power
 
Ok shazam, "la la-lalaa"!
Ok shazam, "la la-lalaa"!Ok shazam, "la la-lalaa"!
Ok shazam, "la la-lalaa"!
 
How Beyonce Disrupts The Music Industry
How Beyonce Disrupts The Music IndustryHow Beyonce Disrupts The Music Industry
How Beyonce Disrupts The Music Industry
 
Business model of Spotify
Business model of SpotifyBusiness model of Spotify
Business model of Spotify
 
The Music Industry - new models (excerpt of a keynote)
The Music Industry - new models (excerpt of a keynote)The Music Industry - new models (excerpt of a keynote)
The Music Industry - new models (excerpt of a keynote)
 
Tracxn Research - Music Tech Landscape, January 2017
Tracxn Research - Music Tech Landscape, January 2017Tracxn Research - Music Tech Landscape, January 2017
Tracxn Research - Music Tech Landscape, January 2017
 
Business Models in the Music Industry
Business Models in the Music IndustryBusiness Models in the Music Industry
Business Models in the Music Industry
 
The future of music industry
The future of music industryThe future of music industry
The future of music industry
 
The Evolution Of The Music Industry The Effect Of Technology And Law On Stra...
The Evolution Of The Music Industry  The Effect Of Technology And Law On Stra...The Evolution Of The Music Industry  The Effect Of Technology And Law On Stra...
The Evolution Of The Music Industry The Effect Of Technology And Law On Stra...
 
The Brief History of the Film - Albert James Burleson
The Brief History of the Film - Albert James BurlesonThe Brief History of the Film - Albert James Burleson
The Brief History of the Film - Albert James Burleson
 

Similar to Music Information Retrieval: Overview and Current Trends 2008

Statistics in musicology
Statistics in musicologyStatistics in musicology
Statistics in musicologyAldo Rodriguez
 
Music Objects to Social Machines
Music Objects to Social MachinesMusic Objects to Social Machines
Music Objects to Social MachinesDavid De Roure
 
Denktank 2010
Denktank 2010Denktank 2010
Denktank 2010ocor203
 
The convergence of "hard" and "soft"in music technology, Rolf Inge Godøy, UiO
The convergence of "hard" and "soft"in music technology, Rolf Inge Godøy, UiOThe convergence of "hard" and "soft"in music technology, Rolf Inge Godøy, UiO
The convergence of "hard" and "soft"in music technology, Rolf Inge Godøy, UiOThe Research Council of Norway, IKTPLUSS
 
Lorenzo Porcaro PhD Defense
Lorenzo Porcaro PhD Defense Lorenzo Porcaro PhD Defense
Lorenzo Porcaro PhD Defense Lorenzo Porcaro
 
1. paul dwyer internet radio emma
1. paul dwyer internet radio emma1. paul dwyer internet radio emma
1. paul dwyer internet radio emmaPaul Dwyer
 
2016 Press Release _rev_FINAL
2016 Press Release _rev_FINAL2016 Press Release _rev_FINAL
2016 Press Release _rev_FINALJohn Brandt
 
Human Perception and Recognition of Musical Instruments: A Review
Human Perception and Recognition of Musical Instruments: A ReviewHuman Perception and Recognition of Musical Instruments: A Review
Human Perception and Recognition of Musical Instruments: A ReviewEditor IJCATR
 
The Streams of Our Lives - Visualizing Listening Histories in Context
The Streams of Our Lives - Visualizing Listening Histories in ContextThe Streams of Our Lives - Visualizing Listening Histories in Context
The Streams of Our Lives - Visualizing Listening Histories in ContextDominikus Baur
 
Genre Classification and Analysis
Genre Classification and AnalysisGenre Classification and Analysis
Genre Classification and AnalysisAnat Gilboa
 
Are It Is Time Consuming When Analyzing The Data And It
Are It Is Time Consuming When Analyzing The Data And ItAre It Is Time Consuming When Analyzing The Data And It
Are It Is Time Consuming When Analyzing The Data And ItAmanda Hengel
 
The Listening Experience Database
The Listening Experience DatabaseThe Listening Experience Database
The Listening Experience DatabaseAlessandro Adamou
 
Parker musiclib401
Parker musiclib401Parker musiclib401
Parker musiclib401japarker12
 
Ask a Librarian: The Role of Librarians in the Music Information Retrieval Co...
Ask a Librarian: The Role of Librarians in the Music Information Retrieval Co...Ask a Librarian: The Role of Librarians in the Music Information Retrieval Co...
Ask a Librarian: The Role of Librarians in the Music Information Retrieval Co...Jenn Riley
 
Archiving and disseminating sound archives – 1. Processes and procedures in...
Archiving and disseminating  sound archives – 1. Processes and procedures  in...Archiving and disseminating  sound archives – 1. Processes and procedures  in...
Archiving and disseminating sound archives – 1. Processes and procedures in...Phonothèque MMSH
 
Music data analysis big data presentation
Music data analysis big data presentationMusic data analysis big data presentation
Music data analysis big data presentationShubhanshu Gupta
 
Sociocultultural and psychophysiological dimensions of souncscape perception
Sociocultultural and psychophysiological dimensions of souncscape perceptionSociocultultural and psychophysiological dimensions of souncscape perception
Sociocultultural and psychophysiological dimensions of souncscape perceptionmario d'andreta
 

Similar to Music Information Retrieval: Overview and Current Trends 2008 (20)

Statistics in musicology
Statistics in musicologyStatistics in musicology
Statistics in musicology
 
Music Objects to Social Machines
Music Objects to Social MachinesMusic Objects to Social Machines
Music Objects to Social Machines
 
Denktank 2010
Denktank 2010Denktank 2010
Denktank 2010
 
The convergence of "hard" and "soft"in music technology, Rolf Inge Godøy, UiO
The convergence of "hard" and "soft"in music technology, Rolf Inge Godøy, UiOThe convergence of "hard" and "soft"in music technology, Rolf Inge Godøy, UiO
The convergence of "hard" and "soft"in music technology, Rolf Inge Godøy, UiO
 
Lorenzo Porcaro PhD Defense
Lorenzo Porcaro PhD Defense Lorenzo Porcaro PhD Defense
Lorenzo Porcaro PhD Defense
 
1. paul dwyer internet radio emma
1. paul dwyer internet radio emma1. paul dwyer internet radio emma
1. paul dwyer internet radio emma
 
2016 Press Release _rev_FINAL
2016 Press Release _rev_FINAL2016 Press Release _rev_FINAL
2016 Press Release _rev_FINAL
 
Human Perception and Recognition of Musical Instruments: A Review
Human Perception and Recognition of Musical Instruments: A ReviewHuman Perception and Recognition of Musical Instruments: A Review
Human Perception and Recognition of Musical Instruments: A Review
 
The Streams of Our Lives - Visualizing Listening Histories in Context
The Streams of Our Lives - Visualizing Listening Histories in ContextThe Streams of Our Lives - Visualizing Listening Histories in Context
The Streams of Our Lives - Visualizing Listening Histories in Context
 
Genre Classification and Analysis
Genre Classification and AnalysisGenre Classification and Analysis
Genre Classification and Analysis
 
Are It Is Time Consuming When Analyzing The Data And It
Are It Is Time Consuming When Analyzing The Data And ItAre It Is Time Consuming When Analyzing The Data And It
Are It Is Time Consuming When Analyzing The Data And It
 
The Listening Experience Database
The Listening Experience DatabaseThe Listening Experience Database
The Listening Experience Database
 
Ibac2007 riede
Ibac2007 riedeIbac2007 riede
Ibac2007 riede
 
Parker musiclib401
Parker musiclib401Parker musiclib401
Parker musiclib401
 
Let the Computer Do the Work
Let the Computer Do the WorkLet the Computer Do the Work
Let the Computer Do the Work
 
Ism2011
Ism2011Ism2011
Ism2011
 
Ask a Librarian: The Role of Librarians in the Music Information Retrieval Co...
Ask a Librarian: The Role of Librarians in the Music Information Retrieval Co...Ask a Librarian: The Role of Librarians in the Music Information Retrieval Co...
Ask a Librarian: The Role of Librarians in the Music Information Retrieval Co...
 
Archiving and disseminating sound archives – 1. Processes and procedures in...
Archiving and disseminating  sound archives – 1. Processes and procedures  in...Archiving and disseminating  sound archives – 1. Processes and procedures  in...
Archiving and disseminating sound archives – 1. Processes and procedures in...
 
Music data analysis big data presentation
Music data analysis big data presentationMusic data analysis big data presentation
Music data analysis big data presentation
 
Sociocultultural and psychophysiological dimensions of souncscape perception
Sociocultultural and psychophysiological dimensions of souncscape perceptionSociocultultural and psychophysiological dimensions of souncscape perception
Sociocultultural and psychophysiological dimensions of souncscape perception
 

Recently uploaded

Upgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsUpgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsIntellect Design Arena Ltd
 
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView
 
Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)Lviv Startup Club
 
MC Heights construction company in Jhang
MC Heights construction company in JhangMC Heights construction company in Jhang
MC Heights construction company in Jhangmcgroupjeya
 
NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023Steve Rader
 
Tata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerakTata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerakEditores1
 
Anyhr.io | Presentation HR&Recruiting agency
Anyhr.io | Presentation HR&Recruiting agencyAnyhr.io | Presentation HR&Recruiting agency
Anyhr.io | Presentation HR&Recruiting agencyHanna Klim
 
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfGraham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfAnhNguyen97152
 
AMAZON SELLER VIRTUAL ASSISTANT PRODUCT RESEARCH .pdf
AMAZON SELLER VIRTUAL ASSISTANT PRODUCT RESEARCH .pdfAMAZON SELLER VIRTUAL ASSISTANT PRODUCT RESEARCH .pdf
AMAZON SELLER VIRTUAL ASSISTANT PRODUCT RESEARCH .pdfJohnCarloValencia4
 
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...Brian Solis
 
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISINGUNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISINGlokeshwarmaha
 
Cracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptxCracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptxWorkforce Group
 
Project Brief & Information Architecture Report
Project Brief & Information Architecture ReportProject Brief & Information Architecture Report
Project Brief & Information Architecture Reportamberjiles31
 
MoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor PresentationMoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor Presentationbaron83
 
Data skills for Agile Teams- Killing story points
Data skills for Agile Teams- Killing story pointsData skills for Agile Teams- Killing story points
Data skills for Agile Teams- Killing story pointsyasinnathani
 
Introduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxIntroduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxJemalSeid25
 
7movierulz.uk
7movierulz.uk7movierulz.uk
7movierulz.ukaroemirsr
 
A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.mcshagufta46
 
Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Winbusinessin
 

Recently uploaded (20)

Upgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking ApplicationsUpgrade Your Banking Experience with Advanced Core Banking Applications
Upgrade Your Banking Experience with Advanced Core Banking Applications
 
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
TalentView Webinar: Empowering the Modern Workforce_ Redefininig Success from...
 
Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)Michael Vidyakin: Introduction to PMO (UA)
Michael Vidyakin: Introduction to PMO (UA)
 
WAM Corporate Presentation Mar 25 2024.pdf
WAM Corporate Presentation Mar 25 2024.pdfWAM Corporate Presentation Mar 25 2024.pdf
WAM Corporate Presentation Mar 25 2024.pdf
 
MC Heights construction company in Jhang
MC Heights construction company in JhangMC Heights construction company in Jhang
MC Heights construction company in Jhang
 
NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023NASA CoCEI Scaling Strategy - November 2023
NASA CoCEI Scaling Strategy - November 2023
 
Tata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerakTata Kelola Bisnis perushaan yang bergerak
Tata Kelola Bisnis perushaan yang bergerak
 
Anyhr.io | Presentation HR&Recruiting agency
Anyhr.io | Presentation HR&Recruiting agencyAnyhr.io | Presentation HR&Recruiting agency
Anyhr.io | Presentation HR&Recruiting agency
 
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdfGraham and Doddsville - Issue 1 - Winter 2006 (1).pdf
Graham and Doddsville - Issue 1 - Winter 2006 (1).pdf
 
AMAZON SELLER VIRTUAL ASSISTANT PRODUCT RESEARCH .pdf
AMAZON SELLER VIRTUAL ASSISTANT PRODUCT RESEARCH .pdfAMAZON SELLER VIRTUAL ASSISTANT PRODUCT RESEARCH .pdf
AMAZON SELLER VIRTUAL ASSISTANT PRODUCT RESEARCH .pdf
 
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
The End of Business as Usual: Rewire the Way You Work to Succeed in the Consu...
 
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISINGUNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
UNLEASHING THE POWER OF PROGRAMMATIC ADVERTISING
 
Cracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptxCracking the ‘Business Process Outsourcing’ Code Main.pptx
Cracking the ‘Business Process Outsourcing’ Code Main.pptx
 
Project Brief & Information Architecture Report
Project Brief & Information Architecture ReportProject Brief & Information Architecture Report
Project Brief & Information Architecture Report
 
MoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor PresentationMoneyBridge Pitch Deck - Investor Presentation
MoneyBridge Pitch Deck - Investor Presentation
 
Data skills for Agile Teams- Killing story points
Data skills for Agile Teams- Killing story pointsData skills for Agile Teams- Killing story points
Data skills for Agile Teams- Killing story points
 
Introduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptxIntroduction to The overview of GAAP LO 1-5.pptx
Introduction to The overview of GAAP LO 1-5.pptx
 
7movierulz.uk
7movierulz.uk7movierulz.uk
7movierulz.uk
 
A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.A flour, rice and Suji company in Jhang.
A flour, rice and Suji company in Jhang.
 
Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024Ethical stalking by Mark Williams. UpliftLive 2024
Ethical stalking by Mark Williams. UpliftLive 2024
 

Music Information Retrieval: Overview and Current Trends 2008

  • 1. Rui Pedro Paiva DEI 10 de Dezembro de 2008 Music Information Retrieval Overview & Current Trends
  • 2. 2 Music Information Retrieval @ DEIUndercover Contents  Introduction  Applications  Short MIR Tale  Disciplines  Generic MIR System  Techniques
  • 3. 3 Music Information Retrieval @ DEIUndercover Introduction  Music and Man • Music expresses “that which cannot be put into words and that which cannot remain silent” (Victor Hugo) • We associate music with the most unique moments of our lives and music is part of our individual and social imaginary - By listening to music, emotions and memories, thoughts and reactions, are awakened • “Life has a soundtrack” (Ana Gomes, “Festivais de Verão”, Público) • “The history of a people is found in its songs” (George Jellinek)
  • 4. 4 Music Information Retrieval @ DEIUndercover Introduction  Music and World economy • Music industry runs, only in the USA an amount of money in the order of several billion US dollars per year. • Explosion of the Electronic Music Industry (EMD) - Widespread access to the Internet - Bandwidth increasing in domestic and mobile accesses - Compact audio formats with near CD quality (mp3, wma) - Portable music devices (iPod, mp3 readers) - Peer-to-peer networks (Napster, Kazaa, eMule) - Online music stores (iTunes, Calabash Music, Sapo Music)  resolution is the song, not the CD - Music identification platforms (Shazam, 411-Song, Gracenote MusicID / TrackID) - Music recommendation systems (MusicSurfer)
  • 5. 5 Music Information Retrieval @ DEIUndercover Introduction  Music and World economy (cont.) • By 2005, Apple iTunes was selling  1.25 million songs each day - Since the service was launched and until thee beginning of 2005  250 million songs had been sold in total • Number and dimension of digital music archives continuously growing - Database size (these days, over 2 million songs) - Genres covered • Challenges to music providers and music librarians - Organization, maintenance, labeling, user interaction - Any large music database is only really useful if users can find what they are looking for in an efficient manner!
  • 6. 6 Music Information Retrieval @ DEIUndercover Introduction  Database Organization and Music Retrieval • Presently, databases are manually annotated  search and retrieval is mostly textual (artist, title, album, genre) - Service providers – Difficulties regarding manual song labeling: subjective and time-consuming, - Customers – Difficulties in performing “content-based” queries • “Music’s preeminent functions are social and psychological”, and so “the most useful retrieval indexes are those that facilitate searching in conformity with such social and psychological functions. Typically, such indexes will focus on stylistic, mood, and similarity information” [Huron, 2000]. •  Music Information Retrieval (MIR) emerges from the necessity to manage huge collections of digital music for “preservation, access, research and other uses” [Futrelle and Downie, 2003].
  • 7. 7 Music Information Retrieval @ DEIUndercover Applications  Platforms for EMD • Similarity-based retrieval tools - Query-by-example – Music identification (trackID, Tunatic) – Music recommendation • Discovery of new music – Islands of music • Metaphor of geographic maps: similar genres are “physically” close together - Automatic playlist generation - Query-by-melody (query-by- humming, query-by-singing) - Plagiarism detection - Music web crawlers
  • 8. 8 Music Information Retrieval @ DEIUndercover Applications  Music education and training • Automatic music transcription - Music composition, analysis, performance evaluation  Audio software • Intelligent audio (music) editors  automatic indexing  Multimedia databases and operating systems © George Tzanetakis, 2002
  • 9. 9 Music Information Retrieval @ DEIUndercover Applications  Video indexing and searching • Segmentation based on audio (music) content  detection of scene transitions  Advertisement and cinema • Tools for mood-based retrieval  Sports • Music to induce a certain cardiac frequency
  • 10. 10 Music Information Retrieval @ DEIUndercover Short MIR Tale  Precursors of computer-based MIR: incipit and theme indexes, e.g., Harold Barlow and Sam Morganstern’s dictionary of musical themes  1966: potential of applying automatic information retrieval techniques to music was recognized (Kassler)  1970s and 1980s: automatic music transcription systems  1990s: surge of interest, mostly in topics such as query-by- humming (impulse from research on digital libraries)  2000: 1st International Symposium on Music Information Retrieval (ISMIR)
  • 11. 11 Music Information Retrieval @ DEIUndercover Short MIR Tale  Presently: strongly cross-disciplinary field • My interests: content analysis and similarity assessment and retrieval in audio song databases - Metrics of similarity, music identification, music recommendation, audio fingerprinting, music classification and feature extraction, tempo and melody detection, music summarization
  • 12. 12 Music Information Retrieval @ DEIUndercover Generic MIR System Client Internet Ordering of Results and Summarization Comparison of Signatures Database Search Database Classification Raw Audio Recordings Segmentation and Summarization Feature Extraction Signature Feature Extraction Query Signature Visualization 1. Tom Jobim - Corcovado 2. Dido - Thank You 3. João Gilberto - Meia Luz 4. ..... 5. ..... Jazz Classical Latin Swing Baroque Mambo Salsa . Interactive Edition Query Creation
  • 13. 13 Music Information Retrieval @ DEIUndercover Techniques  Idea • Extract semantic information from low-level data • Feature extraction - Physical: F0, intensity, centroid, uniformity, rolloff, flux - Perceptual: pitch, loudness, timbre, beat - Musicological: notes, melodies, measures, motives, themes - Higher-level (semantic) features: emotion, genre, instruments, artist
  • 14. 14 Music Information Retrieval @ DEIUndercover Techniques  Physical features • Frequency domain - DFT, STFT, spectrogram - F0 detectors Time Frequency 0 0.5 1 1.5 2 0 0.5 1 1.5 2 x 10 4
  • 15. 15 Music Information Retrieval @ DEIUndercover Techniques  Physical features (cont.) • Spectral features - MFCCs, centroid, rolloff, flux, harmonicity, high-frequency content, … • Sub-band features © MIR Toolbox, 2008
  • 16. 16 Music Information Retrieval @ DEIUndercover Techniques  Physical features (cont.) • Spectral features + sub-band features (e.g., audio fingerprinting) © Haitsma & Kalker, 2002
  • 17. 17 Music Information Retrieval @ DEIUndercover Techniques  Physical features (cont.) • Time domain - Auditory model-based F0 detectors, beat detectors (energy-based) 0.21 0.22 0.23 0.24 0.25 -0.4 -0.2 0 0.2 0.4 Time (s) x(t) a) Sound waveform Time (s) Frequencychannel b) Cochleagram frame 0.21 0.22 0.23 0.24 0.25 20 40 60 80 Time Lag (ms) Frequncychannel c) Correlogram frame 0 5 10 15 20 20 40 60 80 0 5 10 15 20 25 0 0.1 0.2 0.3 0.4 Time Lag (ms) Pitchsalience d) Summary correlogram
  • 18. 18 Music Information Retrieval @ DEIUndercover Techniques  Perceptual features • Pitch - Frequency - Intensity - Context - Ear physiology (age)
  • 19. 19 Music Information Retrieval @ DEIUndercover Techniques  Perceptual features (cont.) • Loudness - Intensity - Frequency - Context - Ear physiology (age) Fletcher-Munson equal loudness contours
  • 20. 20 Music Information Retrieval @ DEIUndercover Techniques  Perceptual features (cont.) • Timbre - No physical correlate - “what something sounds like”: – Spectral content at steady-sate • Centroid, rolloff, relative amplitudes of harmonic components, inharmonicty… – Signal’s temporal envelope • Attack transient – Temporal behavior of the harmonics • Melodic contour - UDUEEUUD • Rhythm contour - LSSLEELS • Beat
  • 21. 21 Music Information Retrieval @ DEIUndercover Techniques  Musicological features • Notes from audio 0 100 200 300 400 500 4800 4900 5000 5100 5200 5300 5400 Time (msec) f[k](cents) a) Eliades Ochoa's excerpt 0 200 400 600 800 6100 6200 6300 6400 6500 6600 6700 6800 Time (msec)f[k](cents) b) Female opera excerpt
  • 22. 22 Music Information Retrieval @ DEIUndercover Techniques  Musicological features (cont.) • Melody
  • 23. 23 Music Information Retrieval @ DEIUndercover Techniques  Musicological features (cont.) • Themes © Colin Meek and William Birmingham, 2001
  • 24. 24 Music Information Retrieval @ DEIUndercover Techniques  Higher-level features  top-down information flow © Xavier Serra, 2005
  • 25. 25 Music Information Retrieval @ DEIUndercover Techniques  Higher-level features (cont.) • Bridge the semantic gap • Memory, context, expectations - Repetitions, sonic environment, modeling the individual, musicological knowledge • Emotion: valence (happy/anxious) and arousal (calm/energetic) - Classification approaches resorting to low-level features