The document discusses the FAST-IMPACT Programme Grant, which brings together UK experts in audio/music technology, semantic web, e-science, and HCI. The program aims to pursue innovations to benefit the music industry through proof-of-concept demonstrator projects. This includes developing semantic, standardized metadata for digital music objects to enhance the experience of both music producers and consumers. Key work threads include developing ontologies and metadata, using semantic web technologies to extract meaning from music content and production processes.

1. Audio
and
seman,c
metadata:
An
overview
of
the
FAST-­‐IMPACt
Programme
Grant
Fusing
Audio
and
Seman,c
Technologies
for
Intelligent
Music
Produc,on
and
Consump,on
funded
by
EPSRC
ICT
Programme
&
Digital
Economy
Programme
Work Thread 7: Digital Music Object
Work Thread 1: Ontology and Metadata
Work Threads : Social, Inference, Workflow, etc…
Work Thread 8: Ethnography
Demonstrator projects
WT7
DMO
WT1
WT6
WT2
WT3
WT4
WT5
WT8
E1 E2
P1
P2D1
D2

2. Big
ques,ons!
(How)
Can
next
genera@on
web
technologies
(Ontologies,
Linked
Data
and
Metadata)
combined
with
music
content
analysis
(par@cularly
derived
at
source)
bring
new
value
and
func@onality
to
producers,
creators,
consumers
and
intermediaries
of
music
content?
(How)
Will
both
ends
of
this
value
chain
benefit
from
more
engaging
interac@ons
(enhanced
produc@vity,
increased
enjoyment
and
immersion)
while
crea@ng
or
consuming
music,
and
can
intermediaries
add
value
with
seman@cally
enhanced
services?
(What)
Can
other
areas
of
science
and
ICT
learn
from
end-­‐to-­‐
end
digi@sa@on
and
next
genera@on
technologies
adapted
in
the
music
industry?

3. Research
Objec,ves
• To
bring
together
the
UK's
leading
experts
in
Audio
&
Music
Technology,
Seman,c
Web,
e-­‐Science
and
HCI
• To
move
beyond
fundamental
research
and
to
pursue
radical
innova,ons
amidst
challenges
driven
by
the
prac,cal
needs
of
end-­‐users
throughout
the
music
industry
from
studios
to
sofas,
with
exci,ng
proof-­‐of-­‐concept
demonstrator
projects
that
collec,vely
form
an
intelligent
music
informa,on
infrastructure
• To
provide
a
mul,-­‐disciplinary,
trans-­‐ins,tu,onal
research
programme
to
nurture
the
next
genera,on
of
UK
researchers
with
skills
that
span
all
the
technologies
and
sciences
relevant
to
next
and
future
genera,on
music
and
audio
produc,on
and
consump,on
eco-­‐systems
• To
test
the
hypothesis
that
Seman,c
Web
technologies
combined
with
content
analysis
at
the
point
of
content
crea,on
will
deliver
real
and
las,ng
change
to
the
music
industry.
• To
examine
the
benefits
of
trea,ng
tags,
annota,ons
and
usage
data
as
forms
of
crowd-­‐sourced
meta-­‐content,
with
its
own
evolu,on
and
meaning,
both
dependent
on
and
independent
of
the
musical
essence
itself.
• To
treat
crea,ve
and
produc,on
processes
on
a
par
with
content,
with
intrinsic
value,
separate
from
but
complimen,ng
the
musical
essence

4. what
do
we
mean
by
seman,c
• logical
seman,cs:
making
sense
of
things,
implica,on
• linguis,c
seman,cs:
meaning
of
words
and
rela,ons
between
them
• extrac,ng
meaning
• represen,ng
meaning
• FAST:
seman,c
web,
linked
data,
interoperability
of
metadata
from
disparate
sources

5. 1st generation
digital music
apps
2nd generation
digital music
apps
Human
metadata, e.g.
Pandora
Content-based
metadata, e.g.
M4 TSB project
Semantic,
standardised
metadata
Consumer satisfaction
2005 2013 < 2020
0th generation
digital music
apps
for
consumers
driven
by
“content
analysis
at
the
point
of
content
crea,on”
context
metadata

6. for
producers
• new
tooling
in
the
studio
to
deliver
“seman,c,
standardised
metadata”
for
the
consumer
• hence
need
to
deliver
advantages
to
studio
prac,,oners

7. Digital Music
Object
Recording studio
computer running
FAST tools
Music content
Gold Standard
Music Metadata
End user client
from
producer
to
consumer:
the
digital
music
object

8.
consume
produce
compose/
perform/capture
distribute
compose/
perform/capture
produce
distribute
consume
from
value
chain
to
value
mesh

9. Demonstrators
• Produc'on
–
with
focus
on
the
studio
produc,on
process;
• Distribu'on
–
with
focus
on
transforming
how
music
is
distributed;
• Experience
–
with
focus
on
how
seman,c
media
will
change
music
consump,on.

10. Work Thread 7: Digital Music Object
Work Thread 1: Ontology and Metadata
Work Threads : Social, Inference, Workflow, etc…
Work Thread 8: Ethnography
Demonstrator projects
WT7
DMO
WT1
WT6
WT2
WT3
WT4
WT5
WT8
E1 E2
P1
P2D1
D2
work
threads
1. Ontologies
&
metadata
2. inference
3. signals
4. work
flow
5. interfaces
6. social
7. digital
music
objects
8. ethnography
&
design

11. Summary
of
outcomes
• Academic
papers
• Workshops
• Web
• Socware
• Standards
• Reports
and
tutorials
• Ontologies
• Repository
of
music
content
metadata
• Repository
of
test
music
• Book
• Commercial
follow-­‐up
• Concerts

12. who
is
involved
• Queen
Mary
U
of
London::
centre
for
digital
music
• U
of
Nogngham::mixed
reality
lab
• U
of
Oxford::e-­‐Research
centre
• Abbey
Road
• BBC
• Omnifone
• SSL
• Internet
Archive
• Universal
Music
Group
• Microsoc
Research
• Interna,onal
Audio
Labs,
Erlangen
• Sustrans

13. THANK
YOU
Contact:
mark.sandler@qmul.ac.uk

14. what
• EPSRC
and
Digital
Economy
funded
• 5
years
,ll
June
2019
• 9
PDRAs
• 6
PhDs
(QMUL
and
Nons
funded)

15. WT1
Ontologies
• develops
new
ontologies
for
music
applica,ons,
building
on
musicontology.com,
integra,ng
with
other
emergent
ontologies,
to
cover
an
increased
range
of
concepts
important
to
the
crea,on,
consump,on,
transmission
and
understanding
of
music.
This
is
exemplified
by
our
recent
work
[Kolozali
et
al.
2013
see
Technical
Annex]
applying
signal
processing
to
generate
music
ontologies.
• WT1
is
connected
to
every
other
WT:
to
WT2
because
the
symbols
defined
in
WT1
enable
the
inferences
of
WT2;
to
WT3
both
through
semi-­‐automa,c
ontology
genera,on
and
audio
feature
ontology
engineering;
to
WT4
because
the
defini,ons
of
workflow
must
be
supported
in
ontologies,
and
also
the
deeper
ques,ons
of
ontological
representa,on
of
process;
to
WT5
because
they
underpin
new
interfaces
that
themselves
must
reflect
the
func,ons
defined,
and
because
ontology
evalua,on
is
partly
undertaken
“in
the
wild”;
to
WT6
because
ontologies
need
to
reflect
human
prac,ce
and
capture
opera,onally
what
humans
do
communally;
to
WT7
because
the
DMOs
are
essen,ally
a
manifesta,on
of
the
ontologies
and
because
the
mapping
from
DMO
to
RDF
will
use
an
ontology,
most
likely
an
extension
of
OAI
Object
Reuse
and
Exchange;
and
to
WT8
because
ethnographic
studies
will
inform
ontology
design
and
be
an
important
aspect
of
their
evalua,on,
for
both
music
produc,on
and
consump,on.

16. WT2
Inference
• deals
with
deriva,on
of
high-­‐level
musical
knowledge
from
basic
musicological
informa,on,
and
with
understanding
of
underlying
content,
using
inference
over
RDF
representa,ons.
This
facilitates
new
search
and
recommenda,on
func,onality,
as
well
as
enhanced
learning
and
immersive,
contextual
rendering.
• As
well
as
men,oned
above
this
WT
is
related
to:
WT3
because
many
items
of
informa,on
will
have
been
obtained
from
signal
analysis;
WT4
similarly,
because
work
flow
decisions
can
be
reasoned
over,
capturing
provenance
and
drawing
inferences
as
diverse
as
well-­‐structuredness
and
record
producer
style
analysis;
WT5
because
the
interfaces
need
to
present
results
of
inference,
and
in
some
cases
infer
appropriate
parameters
for
contexualised
sonic
rendering;
WT6
in
studies
of
composi,on
and
produc,on
by
enabling
inference
and
reasoning
over
all
value-­‐chain
metadata.;
WT7
because
DMOs
are
the
raw
material
conveying
informa,on
between
applica,ons
that
use
inference;
WT8
because
the
ethnographic
evalua,on
of
demonstrators
informs
effec,veness
of
inference
structures.

17. WT3
Signals
• studies
advanced
signal
processing
techniques
to
extract
high
quality
features
from
audio,
across
a
range
of
temporal
scales
from
complete
songs
to
individual
notes.
Areas
of
interest
include
structural
segmenta,on,
chord/harmony,
rhythm/
metre,
intona,on,
vibrato
and
ornaments.
These
underpin
the
founda,ons
for
the
seman,c
representa,ons,
expressed
via
ontologies,
and
used
to
drive
seman,cally
enhanced
user
applica,ons.
• As
well
as
men,oned
above
this
WT
is
related
to:
WT4
because
all
produc,on
workflow
(and
many
in
consump,on)
depend
on
signal
processing
pipelines
to
deliver
new
data
and
content
which
should
be
captured
for
provenance
and
reasoning;
WT5
because
produc,on
interfaces
present
processing
inputs
and
outcomes,
and
abstract
the
underlying
processing,
and
because
sonic
(i.e.
listening)
interfaces
that
are
context-­‐dependant
require
the
processing
of
signals;
WT6
in
suppor,ng
‘ci,zen
musicology’
as
defined
below;
WT7
because
studio
processing
will
be
captured
in
DMOs
(via
ontologies);
WT8
because
it
empowers
demonstrators,
that
are
then
evaluated,
that
evalua,on
informing
the
relevance
and
quality
of
audio
features
extracted.

18. WT4
Workflow
• takes
established
e-­‐Science
techniques
(see
researchobject.org)
and
applies
them
to
seman,c
computa,onal
workflows
in
the
crea,on
and
processing
of
music.
These
workflows,
which
underpin
Digital
Music
Objects
(MDO)
in
WT.7,
describe
the
pipeline
of
(signal)
processing
that
deliver
new
data
and
content,
so
that
processes
can
be
repeated,
reused
and
repurposed.
• As
well
as
men,oned
above
this
WT
is
related
to:
WT5
because
interfaces
are
where
humans
meet
process
and
because
interfaces
are
needed
for
working
with
process;
WT6,
perhaps
the
deepest
intersec,on
because
music
crea,on
through
social
media
is
a
new
paradigm
to
be
described
and
replayed,
and
there
is
a
close
connec,on
to
De
Roure’s
Social
Machines
research;
WT7
because
MDOs
are
constructed,
shared
and
later
modified
as
a
consequence
of
workflows;
WT8
because
the
workflows
are
the
raw
ingredients
of
the
ethnographic
studies.

19. WT5
Interface
• researches
and
develops
new
user
interfaces
and
interac,ons
adapted
to
the
new
modali,es
for
professionals
and
consumers.
As
well
as
visual
interfaces
for
the
variety
of
human
ac,vity
this
Programme
Grant
encompasses,
we
will
conduct
research
on
context-­‐aware
sonic
interfaces.
This
combines
automa,c
audio
mixing
with
psychoacous,cs
to
render
audio
scenes
that
account
for
aspects
like
ambient
noise
and
reproduc,on
device
proper,es,
as
well
as
personalisa,on.
• As
well
as
men,oned
above
this
WT
is
related
to:
WT6
via
the
work
on
live
performance,
adap,ve
and
augmented
musical
interfaces,
and
the
sharing
of
process
descrip,ons;
WT7
because
it
is
important
to
develop
appropriate
interfaces
to
MDOs
as
an
explicit
ac,vity;
WT8
because
it
delivers
design
requirements
and
evaluates
the
interfaces,
developed
especially
in
the
demonstrators

20. WT6
Social
• examines
the
role
of
social
media
and
metadata
in
music
crea,on
and
co-­‐crea,on,
and
develops
a
concept
of
narra,ve,
based
on
ontological
principles,
that
can
be
applied
to
the
evolu,on
of
the
metadata
in
DMO
through
the
content
life-­‐
cycle.
It
also
encompasses
the
use
of
social
media
in
“ci,zen
musicology”,
combining
tags
with
signal-­‐derived
features
to
harvest
value
from
the
deepest
recesses
of
even
the
largest
music
collec,ons.
There
will
be
research
into
the
impact
of
music
on
physical
experience,
with
mul,-­‐modal
sensor
data
capture.
• As
well
as
men,oned
above
this
WT
is
related
to:
WT7
because
all
the
social
enterprises
need
to
share
metadata,
and
this
is
via
DMOs;
WT8
because
it
first
shapes
and
subsequently
evaluates
the
social
interac,ons
and
prac,ces
captured
through
the
projects

21. WT7
Digital
Music
Objects
• DMOs
can
be
shared,
edited
and
executed
by
people
or
machines
in
the
value
chain.
They
can
be
analysed
as
a
means
to
understand
the
crea,ve
process
of
music
crea,on,
form
the
raw
material
for
future
crea,ve
processes,
and
form
an
important
element
of
an
enhanced
commercial
offering
to
consumers.
They
assimilate
the
various
digital
artefacts
that
are
created
in
other
WTs
from
Signal
Processing
to
community
generated
content.
Equally,
other
WTs
should
u,lise
DMOs
to
ensure
early
provision
of
an
ini,al
specifica,on,
prototype
tools
and
a
repository
mechanism.
This
WT
is
expected
to
persist
throughout
and
interacts
with
all
the
other
WTs.

22. WT.8
Ethnography
&
Design
• encompasses
user-­‐centric
design,
enabling
the
team
to
design
appropriate
demonstrators
and
subsequently
their
formal
evalua,on
in
real
use
scenarios.
There
will
be
ethnographic
studies
of
various
scenarios
in
both
produc,on,
including
live
performance,
and
consump,on.
The
use
of
formal
ethnographic
method
in
the
requirements
capture
phase
for
ontologies
is
believed
to
be
en,rely
novel,
as
is
its
use
in
evalua,ng
applica,ons
that
those
ontologies
enable.