How we made music using Neural Networks. Live improvisation using Python, Magenta, Ableton and 3 professional musicians

1. The Sound Of AI
Aleksandr Tavgen

2. About Me
• Love machinery from childhood
• First gamewrote at 13y. old
• UFO bombed tanks (Basic)
• Last years worked at ML and Data
Science fields related to FinTech
• As a hobby different projects in
art and robotics field

3. Some Of The Past Projects
• Computer Vision
• Robotics
• New Ways Of Interactions
• Art Related Projects

4. Neuroshtein
Prepared for
conference badge
pickup party
Planned as live
improvisation
3 professional
musicians
And……

5. Recurrent Neural Networks
• Produce good results when handling
time series data with an underlying
structure
• The structure of a natural language
has a number of dimensions
• Semantic dimension is quite
difficult for machines to grasp
• Syntactic dimension is quite
manageable for machines

6. Music As A Text
Music is structured
along various
dimensions such as
rhythm, intervals,
dynamics
01
Music constitutes
well-structured
time series data
02
MIDI events could
be represented as
an alphabet
03

7. Trump Speeches Generating
• Collected Corpus of Trump Speeches
• Results were quite funny
• Completed 340 minibatches of size 32x1000 characters
Second. They left you asses, believe me, but I will have
great people.I solling us some -- and you see something
youve seen deterner to Mexico, we are building interest.
100,000. Im not going to be so bad.
It was so proud of me. Incredible and or their cities which
I kept the same wealthy. They dont want them, were the
world companies.

8. Magenta Project
• Google AI project
• Stumbled upon when started this Project
• Interface between MIDI and Tensorflow
• Ableton was a glue

9. First Steps
• MIDI signals should first be
mapped onto an artificial
alphabet
• Pretrained LSTM RNN with two
hidden layers of 128 elements
• Continued Training on bunch of
MIDI
• Call-Response approach
• Given the example Model try to
reproduce rhythm structure

11. Configuration And Problems
• Two models listens to each other in call-response mode
• There were problems with midi-clock synchronization
• Some latency were presented due to a large routes amount
between MIDI
• Latency were fixed by shifting

12. We made a video in
Playtech Tallinn’s
office
• Every attempt of improvisation
was different
• Quite funny to fill office building
with powerful bass sound

14. Vinyl Was Published
• One track was taken from one of Neural Networks
Improvisation
• Music from this vinyl were used in next large project
• Interactive Theatre with Java-Graphs and Non Linear
Plotline

16. Bringing To Live
• Connect Those Configuration With Solenoids
• 8-10 Solenoids will be used
• Main concert 07. march 2019

18. Q&A
• Medium @Atavgen
• Twitter @Atavgen
• Will send some other articles by mail