SlideShare uma empresa Scribd logo
1 de 74
How we scaled
Songkick
Friday, 8 March 13
InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
http://www.infoq.com/presentations
/songkick-scale
Presented at QCon London
www.qconlondon.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide
songkick.com
• Founded 2007
• Hundreds of thousands of upcoming
concerts
• 3.4 million past concerts
• 8 million uniques a month
• Second most visited live music website
after ticketmaster
Friday, 8 March 13
3 slides:
Overview of songkick. Pictures.
How did it start?
How big is company?
How many devs etc?
How were you architecting stuff?
What roles?
How is team structured?
What problems did this cause / did you face?
Friday, 8 March 13
We Started small
• Four people in a flat in Spitalfields
• And grew
Friday, 8 March 13
We are still small
• 30 People in an office in Hoxton
• We are divided into cross functional teams
the number and size of which change as we
need
Friday, 8 March 13
We also do
Maybe not the iphone and
android applications.
Though they use some
similar concepts and
certainly rest on some of
the same infrastructure.
In the case of the iPhone
and Android applications
the way we know which
artists you are interested
in is we look on you
device.We also use
geolocation to find where
you are and to notify you,
we use push notification.
Again this is just for
completeness we are
probably not going to
mention them much
But I’m not going to be
talking directly about
these, although they do
use a similar
architecture.
Friday, 8 March 13
The old architecture
skweb
Mysql
Friday, 8 March 13
The old architecture
skweb
Mysql
A rails application
Friday, 8 March 13
What was the problem
• Initially features were over-engineered
• To develop and ship quickly it was easier to
stick it all in one place
• But site was up, traffic growing.Trouble
brewing …
Friday, 8 March 13
What’s the problem?
• Shipping new features became difficult
• Our builds were taking hours to run
• We had complex relationships between
what were notionally separate applications
• Dependancies were hard to understand
and hard to untangle
All these things meant if you wanted to change something, if you wanted to change the copy in an emails, you had to deploy the entire app.
We had a few false starts where we broke up the functions of the application. Unfortunately the boundaries weren’t clear and it was still a single
code base so we still had to deploy everything together
Integration queue
Friday, 8 March 13
Our dependency graph
We wrote software to
tell us what the
dependancies were of
the components of our
software.
It wasn’t pretty.
Friday, 8 March 13
Why re-architect?
We can respond to
competitors and changes
in the market more
readily.
For us increasing
productivity was one of
our most important
goals.
Yes performance was
import, and, yes we had a
lot of code in the app to
make it more
performant (caching etc)
Which did make it
reasonably performant.
Friday, 8 March 13
Why re-architect?
• Scale (more users doing more things)
We can respond to
competitors and changes
in the market more
readily.
For us increasing
productivity was one of
our most important
goals.
Yes performance was
import, and, yes we had a
lot of code in the app to
make it more
performant (caching etc)
Which did make it
reasonably performant.
Friday, 8 March 13
Why re-architect?
• Scale (more users doing more things)
We can respond to
competitors and changes
in the market more
readily.
• Developer productivity (more features,
fewer bugs)
For us increasing
productivity was one of
our most important
goals.
Yes performance was
import, and, yes we had a
lot of code in the app to
make it more
performant (caching etc)
Which did make it
reasonably performant.
Friday, 8 March 13
Why re-architect?
• Scale (more users doing more things)
We can respond to
competitors and changes
in the market more
readily.
• Developer productivity (more features,
fewer bugs)
• Agility (more frequent releases, shorter
time between releases)
For us increasing
productivity was one of
our most important
goals.
Yes performance was
import, and, yes we had a
lot of code in the app to
make it more
performant (caching etc)
Which did make it
reasonably performant.
Friday, 8 March 13
Why not re-architect?
These were all real
concerns.
How do you persuade the other people in the company that
spending six months doing this is worth doing?
This is a start-up you really can’t spend six month navel gazing.
The company could go bust.
You need to have a compelling reason and a plan.
Friday, 8 March 13
Why not re-architect?
• You might never finish
These were all real
concerns.
How do you persuade the other people in the company that
spending six months doing this is worth doing?
This is a start-up you really can’t spend six month navel gazing.
The company could go bust.
You need to have a compelling reason and a plan.
Friday, 8 March 13
Why not re-architect?
• You might never finish
These were all real
concerns.
• You might not achieve the benefits
How do you persuade the other people in the company that
spending six months doing this is worth doing?
This is a start-up you really can’t spend six month navel gazing.
The company could go bust.
You need to have a compelling reason and a plan.
Friday, 8 March 13
Why not re-architect?
• You might never finish
These were all real
concerns.
• You might not achieve the benefits
• It might be easier to rewrite
How do you persuade the other people in the company that
spending six months doing this is worth doing?
This is a start-up you really can’t spend six month navel gazing.
The company could go bust.
You need to have a compelling reason and a plan.
Friday, 8 March 13
Why not re-architect?
• You might never finish
These were all real
concerns.
• You might not achieve the benefits
• It might be easier to rewrite
• The new architecture might not be better
than the old one
How do you persuade the other people in the company that
spending six months doing this is worth doing?
This is a start-up you really can’t spend six month navel gazing.
The company could go bust.
You need to have a compelling reason and a plan.
Friday, 8 March 13
Collaboration!
How did we know what
cut and what to keep.
iPhone.
Why what things are called?
Shared vocabulary, every one in the company
calls the same thing by the same name.
This will become important later on, in the
development of the application.
And actually this process of identifying, cutting or adding features,
choosing names and prioritising work is iterative. Each step of the
way this process is repeated.
Friday, 8 March 13
Collaboration!
• Software is built by a team
How did we know what
cut and what to keep.
iPhone.
Why what things are called?
Shared vocabulary, every one in the company
calls the same thing by the same name.
This will become important later on, in the
development of the application.
And actually this process of identifying, cutting or adding features,
choosing names and prioritising work is iterative. Each step of the
way this process is repeated.
Friday, 8 March 13
Collaboration!
• Software is built by a team
How did we know what
cut and what to keep.
iPhone.
• Not just a team of programmers
Why what things are called?
Shared vocabulary, every one in the company
calls the same thing by the same name.
This will become important later on, in the
development of the application.
And actually this process of identifying, cutting or adding features,
choosing names and prioritising work is iterative. Each step of the
way this process is repeated.
Friday, 8 March 13
Collaboration!
• Software is built by a team
How did we know what
cut and what to keep.
iPhone.
• Not just a team of programmers
• You need to agree on what can be cut
Why what things are called?
Shared vocabulary, every one in the company
calls the same thing by the same name.
This will become important later on, in the
development of the application.
And actually this process of identifying, cutting or adding features,
choosing names and prioritising work is iterative. Each step of the
way this process is repeated.
Friday, 8 March 13
Collaboration!
• Software is built by a team
How did we know what
cut and what to keep.
iPhone.
• Not just a team of programmers
• You need to agree on what can be cut
• What is the minimum feature set needed
Why what things are called?
Shared vocabulary, every one in the company
calls the same thing by the same name.
This will become important later on, in the
development of the application.
And actually this process of identifying, cutting or adding features,
choosing names and prioritising work is iterative. Each step of the
way this process is repeated.
Friday, 8 March 13
Collaboration!
• Software is built by a team
How did we know what
cut and what to keep.
iPhone.
• Not just a team of programmers
• You need to agree on what can be cut
• What is the minimum feature set needed
• What things are called
Why what things are called?
Shared vocabulary, every one in the company
calls the same thing by the same name.
This will become important later on, in the
development of the application.
And actually this process of identifying, cutting or adding features,
choosing names and prioritising work is iterative. Each step of the
way this process is repeated.
Friday, 8 March 13
What we settled on
Which is kind of boring, but, how you get from one to the
other is more interesting.
And doing it in a reasonable amount of time and without
breaking the existing site and not doing a big bang release is a
challenge.
We decided early on that moving to the new architecture
would be done in a stepwise fashion.With the refactoring and
splitting of the functions one page at a time.
We had a pages that were functionally quite distinct.And if
you want to do this step by step you need a unit you can use
to measure progress and divide up the work.
I should emphasise this was not handed down on graven
tablets by the development team, we arrived here by
explaining why this was the best option.
Friday, 8 March 13
What we settled on
• Services
Which is kind of boring, but, how you get from one to the
other is more interesting.
And doing it in a reasonable amount of time and without
breaking the existing site and not doing a big bang release is a
challenge.
We decided early on that moving to the new architecture
would be done in a stepwise fashion.With the refactoring and
splitting of the functions one page at a time.
We had a pages that were functionally quite distinct.And if
you want to do this step by step you need a unit you can use
to measure progress and divide up the work.
I should emphasise this was not handed down on graven
tablets by the development team, we arrived here by
explaining why this was the best option.
Friday, 8 March 13
What we settled on
• Services
• And Clients
Which is kind of boring, but, how you get from one to the
other is more interesting.
And doing it in a reasonable amount of time and without
breaking the existing site and not doing a big bang release is a
challenge.
We decided early on that moving to the new architecture
would be done in a stepwise fashion.With the refactoring and
splitting of the functions one page at a time.
We had a pages that were functionally quite distinct.And if
you want to do this step by step you need a unit you can use
to measure progress and divide up the work.
I should emphasise this was not handed down on graven
tablets by the development team, we arrived here by
explaining why this was the best option.
Friday, 8 March 13
So far, so conventional
Friday, 8 March 13
What a service
looks like We actually started with
dummy services where we
implemented the interface
to the service inside the
application.
Active record leaks up the
Worth noting our services don’t have versioning, access control or XML. And that we do not need
to maintain backwards compatibility, since we control all the clients. (at least for now)
And they are kind of REST. Not real, phd level REST, but certainly popular, Rails-developer style
REST.
Friday, 8 March 13
What a service
looks like We actually started with
dummy services where we
implemented the interface
to the service inside the
application.
Active record leaks up the
• Sinatra application
Worth noting our services don’t have versioning, access control or XML. And that we do not need
to maintain backwards compatibility, since we control all the clients. (at least for now)
And they are kind of REST. Not real, phd level REST, but certainly popular, Rails-developer style
REST.
Friday, 8 March 13
What a service
looks like We actually started with
dummy services where we
implemented the interface
to the service inside the
application.
Active record leaks up the
• Sinatra application
• Emits JSON over HTTP
Worth noting our services don’t have versioning, access control or XML. And that we do not need
to maintain backwards compatibility, since we control all the clients. (at least for now)
And they are kind of REST. Not real, phd level REST, but certainly popular, Rails-developer style
REST.
Friday, 8 March 13
What a service
looks like We actually started with
dummy services where we
implemented the interface
to the service inside the
application.
Active record leaks up the
• Sinatra application
• Emits JSON over HTTP
• Accepts form encoded or JSON data over HTTP
Worth noting our services don’t have versioning, access control or XML. And that we do not need
to maintain backwards compatibility, since we control all the clients. (at least for now)
And they are kind of REST. Not real, phd level REST, but certainly popular, Rails-developer style
REST.
Friday, 8 March 13
What a service
looks like We actually started with
dummy services where we
implemented the interface
to the service inside the
application.
Active record leaks up the
• Sinatra application
• Emits JSON over HTTP
• Accepts form encoded or JSON data over HTTP
• Completely internally encapsulated
Worth noting our services don’t have versioning, access control or XML. And that we do not need
to maintain backwards compatibility, since we control all the clients. (at least for now)
And they are kind of REST. Not real, phd level REST, but certainly popular, Rails-developer style
REST.
Friday, 8 March 13
What a client
application looks like
Pages are ruby classes
that model what a pages
behaviour should be
Friday, 8 March 13
What a client
application looks like
• Rails application (so far any way) Pages are ruby classes
that model what a pages
behaviour should be
Friday, 8 March 13
What a client
application looks like
• Rails application (so far any way)
• Has a traditional ‘MVC’ structure
Pages are ruby classes
that model what a pages
behaviour should be
Friday, 8 March 13
What a client
application looks like
• Rails application (so far any way)
• Has a traditional ‘MVC’ structure
• Gets all its data from services
Pages are ruby classes
that model what a pages
behaviour should be
Friday, 8 March 13
What a client
application looks like
• Rails application (so far any way)
• Has a traditional ‘MVC’ structure
• Gets all its data from services
• We added ‘pages’,‘components’ and
‘elements’
Pages are ruby classes
that model what a pages
behaviour should be
Friday, 8 March 13
How it fits togetherWhat is a page?
Why add pages?
What are components?
What are elements?
Benefits?
What makes a component?
A self contained unit on the page
normally you can draw a box
around it and give it a name.
Arbitrarily components cannot be
nested.
What makes an element? Are
common functionality shared
between components.
Friday, 8 March 13
How it fits together
• A Page
What is a page?
Why add pages?
What are components?
What are elements?
Benefits?
What makes a component?
A self contained unit on the page
normally you can draw a box
around it and give it a name.
Arbitrarily components cannot be
nested.
What makes an element? Are
common functionality shared
between components.
Friday, 8 March 13
How it fits together
• A Page
• Is made of components
What is a page?
Why add pages?
What are components?
What are elements?
Benefits?
What makes a component?
A self contained unit on the page
normally you can draw a box
around it and give it a name.
Arbitrarily components cannot be
nested.
What makes an element? Are
common functionality shared
between components.
Friday, 8 March 13
How it fits together
• A Page
• Is made of components
• Some components are composed of
elements
What is a page?
Why add pages?
What are components?
What are elements?
Benefits?
What makes a component?
A self contained unit on the page
normally you can draw a box
around it and give it a name.
Arbitrarily components cannot be
nested.
What makes an element? Are
common functionality shared
between components.
Friday, 8 March 13
Friday, 8 March 13
Layers
(it’s all about layers)
Event pages need a venue
and an artist and an
event
Artist pages have an
artist + calendar and
media (see etc)
Users are User +
calendar
Components
Artist Page User Page Venue PageEvent Page
Pages
Event Artist UserCalendar Venue Etc …
Models
Accounts NotificationsCaltrakEvent listings Etc …
Services
Services here are classes
in the client.They talk to
the network and handle
passing the data up to
the models and dat from
the models out to the
network
Friday, 8 March 13
So how does all this
work in practice?
Friday, 8 March 13
So how does all this
work in practice?
• The client is still a rails app with the familiar
rails layout
Friday, 8 March 13
So how does all this
work in practice?
• The client is still a rails app with the familiar
rails layout
• Anywhere a rails app might talk to a data
store, the app talks to a service instead
Friday, 8 March 13
So how does all this
work in practice?
• The client is still a rails app with the familiar
rails layout
• Anywhere a rails app might talk to a data
store, the app talks to a service instead
• And we have added some conventions
Friday, 8 March 13
Components are self
contained. Each component has
a name and takes an object.
The object contains the data
the component needs and any
decision making is provided by
methods on that object.
The name of the component is
also the name of the template
file on disc, the html class name
and the name of its
corresponding css and
javascript files.
This tight convention around
names makes understanding the
dependancy between a
The conventions
Every page having a css
file does mean you get
some repetition, but, the
confidence it gives you
about where changes
will appear makes it well
worth it.
The css file imports
smaller files with shared
styling
Friday, 8 March 13
• Every Page has a type
Components are self
contained. Each component has
a name and takes an object.
The object contains the data
the component needs and any
decision making is provided by
methods on that object.
The name of the component is
also the name of the template
file on disc, the html class name
and the name of its
corresponding css and
javascript files.
This tight convention around
names makes understanding the
dependancy between a
The conventions
Every page having a css
file does mean you get
some repetition, but, the
confidence it gives you
about where changes
will appear makes it well
worth it.
The css file imports
smaller files with shared
styling
Friday, 8 March 13
• Every Page has a type
• Every page has one CSS file
Components are self
contained. Each component has
a name and takes an object.
The object contains the data
the component needs and any
decision making is provided by
methods on that object.
The name of the component is
also the name of the template
file on disc, the html class name
and the name of its
corresponding css and
javascript files.
This tight convention around
names makes understanding the
dependancy between a
The conventions
Every page having a css
file does mean you get
some repetition, but, the
confidence it gives you
about where changes
will appear makes it well
worth it.
The css file imports
smaller files with shared
styling
Friday, 8 March 13
• Every Page has a type
• Every page has one CSS file
• The CSS file has the same name as the page
type
Components are self
contained. Each component has
a name and takes an object.
The object contains the data
the component needs and any
decision making is provided by
methods on that object.
The name of the component is
also the name of the template
file on disc, the html class name
and the name of its
corresponding css and
javascript files.
This tight convention around
names makes understanding the
dependancy between a
The conventions
Every page having a css
file does mean you get
some repetition, but, the
confidence it gives you
about where changes
will appear makes it well
worth it.
The css file imports
smaller files with shared
styling
Friday, 8 March 13
• Every Page has a type
• Every page has one CSS file
• The CSS file has the same name as the page
type
• Every component has a corresponding CSS
file
Components are self
contained. Each component has
a name and takes an object.
The object contains the data
the component needs and any
decision making is provided by
methods on that object.
The name of the component is
also the name of the template
file on disc, the html class name
and the name of its
corresponding css and
javascript files.
This tight convention around
names makes understanding the
dependancy between a
The conventions
Every page having a css
file does mean you get
some repetition, but, the
confidence it gives you
about where changes
will appear makes it well
worth it.
The css file imports
smaller files with shared
styling
Friday, 8 March 13
• Every Page has a type
• Every page has one CSS file
• The CSS file has the same name as the page
type
• Every component has a corresponding CSS
file
• If it needs it the component also has a
javascript file
Components are self
contained. Each component has
a name and takes an object.
The object contains the data
the component needs and any
decision making is provided by
methods on that object.
The name of the component is
also the name of the template
file on disc, the html class name
and the name of its
corresponding css and
javascript files.
This tight convention around
names makes understanding the
dependancy between a
The conventions
Every page having a css
file does mean you get
some repetition, but, the
confidence it gives you
about where changes
will appear makes it well
worth it.
The css file imports
smaller files with shared
styling
Friday, 8 March 13
A little bit of code
This is of cause in ruby
but that hardly matters.
Friday, 8 March 13
A little bit of code
This is of cause in ruby
but that hardly matters.
skweb/
    app/
        controllers/
            venues_controller.rb
        models/
            page_models/
                venue.rb
            skweb/
                models/
                    venue.rb
        views/
            venues/
                _brief.html.erb
                show.html.erb
    public/
        javascripts/
            songkick/
                component/
                    tickets.js
        stylesheets/
            components/
                venue-brief.css
            shared/
                components/
                    brief.css
            venue.css
Friday, 8 March 13
A little bit of code
This is of cause in ruby
but that hardly matters.
skweb/
    app/
        controllers/
            venues_controller.rb
        models/
            page_models/
                venue.rb
            skweb/
                models/
                    venue.rb
        views/
            venues/
                _brief.html.erb
                show.html.erb
    public/
        javascripts/
            songkick/
                component/
                    tickets.js
        stylesheets/
            components/
                venue-brief.css
            shared/
                components/
                    brief.css
            venue.css
class VenuesController < ApplicationController
  def show
    @page = PageModels::Venue.new(venue,
logged_in_user)
  end
end
Friday, 8 March 13
A little bit of code
This is of cause in ruby
but that hardly matters.
skweb/
    app/
        controllers/
            venues_controller.rb
        models/
            page_models/
                venue.rb
            skweb/
                models/
                    venue.rb
        views/
            venues/
                _brief.html.erb
                show.html.erb
    public/
        javascripts/
            songkick/
                component/
                    tickets.js
        stylesheets/
            components/
                venue-brief.css
            shared/
                components/
                    brief.css
            venue.css
class VenuesController < ApplicationController
  def show
    @page = PageModels::Venue.new(venue,
logged_in_user)
  end
end
module PageModels
  class Venue < PageModels::Base
    def brief
      Brief.new(@venue, upcoming_events.total_entries,
@logged_in_user)
    end
 end
end
Friday, 8 March 13
A little bit of code
This is of cause in ruby
but that hardly matters.
skweb/
    app/
        controllers/
            venues_controller.rb
        models/
            page_models/
                venue.rb
            skweb/
                models/
                    venue.rb
        views/
            venues/
                _brief.html.erb
                show.html.erb
    public/
        javascripts/
            songkick/
                component/
                    tickets.js
        stylesheets/
            components/
                venue-brief.css
            shared/
                components/
                    brief.css
            venue.css
class VenuesController < ApplicationController
  def show
    @page = PageModels::Venue.new(venue,
logged_in_user)
  end
end
module PageModels
  class Venue < PageModels::Base
    def brief
      Brief.new(@venue, upcoming_events.total_entries,
@logged_in_user)
    end
 end
end
module PageModels
  class Venue
    class Brief
      def geolocation
        @venue.geolocation
      end
    end
  end
end
Friday, 8 March 13
Moving to the view
component() and
shared_component()
are defined in
ApplicationHelper and
look like this:
Friday, 8 March 13
Moving to the view
<div class="primary col">
  <%= component('brief', @page.brief) %>
  <%= component('map', @page.brief.geolocation) %>
  <%= shared_component('calendar_summary',   @page.calendar_summary) %>
  <%= shared_component('media_summary',      @page.media_summary) %>
  <%= shared_component('media_links',        @page.media_links) %>
  <%= shared_component('gigography_summary', @page.gigography_summary) %>
</div>
component() and
shared_component()
are defined in
ApplicationHelper and
look like this:
Friday, 8 March 13
Moving to the view
<div class="primary col">
  <%= component('brief', @page.brief) %>
  <%= component('map', @page.brief.geolocation) %>
  <%= shared_component('calendar_summary',   @page.calendar_summary) %>
  <%= shared_component('media_summary',      @page.media_summary) %>
  <%= shared_component('media_links',        @page.media_links) %>
  <%= shared_component('gigography_summary', @page.gigography_summary) %>
</div>
component() and
shared_component()
are defined in
ApplicationHelper and
look like this:
def component(component_name, object)
  return '' if object.nil?
  render :partial => component_name, :object => object
end
def shared_component(component_name, object)
  component("shared/components/#{component_name}", object)
end
Friday, 8 March 13
Moving to the view
<div class="primary col">
  <%= component('brief', @page.brief) %>
  <%= component('map', @page.brief.geolocation) %>
  <%= shared_component('calendar_summary',   @page.calendar_summary) %>
  <%= shared_component('media_summary',      @page.media_summary) %>
  <%= shared_component('media_links',        @page.media_links) %>
  <%= shared_component('gigography_summary', @page.gigography_summary) %>
</div>
component() and
shared_component()
are defined in
ApplicationHelper and
look like this:
def component(component_name, object)
  return '' if object.nil?
  render :partial => component_name, :object => object
end
def shared_component(component_name, object)
  component("shared/components/#{component_name}", object)
end
@import 'shared/components/brief.css';
@import 'components/venue-brief.css';
@import 'components/venue-map.css';
@import 'shared/components/media-summary.css';
@import 'shared/components/event-listings.css';
Friday, 8 March 13
What did this give us
I’d hoped to have a graph showing
improved page response times, but
unfortunately we didn’t keep them
Many of our services can be scaled
horizontally mean at lest in the medium
term we can increase capacity by adding
nodes
The compartmentalisation of the
application.
The independence of the services means
parallelising development is simpler.
Knowing where to add functionality is
easier.
Friday, 8 March 13
What did this give us
• Developer productivity was radically improved
I’d hoped to have a graph showing
improved page response times, but
unfortunately we didn’t keep them
Many of our services can be scaled
horizontally mean at lest in the medium
term we can increase capacity by adding
nodes
The compartmentalisation of the
application.
The independence of the services means
parallelising development is simpler.
Knowing where to add functionality is
easier.
Friday, 8 March 13
What did this give us
• Developer productivity was radically improved
• Application performance was much better
I’d hoped to have a graph showing
improved page response times, but
unfortunately we didn’t keep them
Many of our services can be scaled
horizontally mean at lest in the medium
term we can increase capacity by adding
nodes
The compartmentalisation of the
application.
The independence of the services means
parallelising development is simpler.
Knowing where to add functionality is
easier.
Friday, 8 March 13
A leaner code base
• Before    After
• 3.5MB       1.4MB    ./app
• 1.8MB       744KB    ./features
• 1.2MB       724KB    ./spec
Friday, 8 March 13
Faster Builds
Before
• Over an hour
• Parallelized with 1 local and 10 ec2 instances
After
• 10 minutes
• 1 local machine
Friday, 8 March 13
Weekly visits
Visits
Songkick 2 launched
9 June 2009
Christmas
2008 2009 2010 2011 2012 2013
Started
rearchitecture
Finished
rearchitecture
A time line of how our
traffic has grown.
Traffic grew all threw
2010 we were adding
features, iterating madly
to improve conversion
and user engagement.
Friday, 8 March 13
Releases per month
0
25
50
75
100
125
150
Mar 2011 Aug 2011 Nov 2011 Feb 2012 May 2012 Aug 2012 Nov 2012
Just to prove we were
shipping more once we
finished the product.
Friday, 8 March 13

Mais conteúdo relacionado

Destaque

Historia de la economía
Historia de la economíaHistoria de la economía
Historia de la economíaErnesto Rios E.
 
Jobs in Bangalore
Jobs in  BangaloreJobs in  Bangalore
Jobs in BangaloreSME Trades
 
Eset. Дмитрий Самойленко. "Eset. Антивирусная защита для вашего бизнеса"
Eset. Дмитрий Самойленко. "Eset. Антивирусная защита для вашего бизнеса"Eset. Дмитрий Самойленко. "Eset. Антивирусная защита для вашего бизнеса"
Eset. Дмитрий Самойленко. "Eset. Антивирусная защита для вашего бизнеса"Expolink
 
Customer Experience Matters
Customer Experience MattersCustomer Experience Matters
Customer Experience MattersPiotr Merkel
 
동국대학교 중앙동아리 D.N.A 2014년도 동아리 창립제 발표 자료 - 자료구조 스터디
동국대학교 중앙동아리 D.N.A 2014년도 동아리 창립제 발표 자료 - 자료구조 스터디동국대학교 중앙동아리 D.N.A 2014년도 동아리 창립제 발표 자료 - 자료구조 스터디
동국대학교 중앙동아리 D.N.A 2014년도 동아리 창립제 발표 자료 - 자료구조 스터디dgu_DNA
 
Direitos de Autor e Propriedade Intelectual
Direitos de Autor e Propriedade IntelectualDireitos de Autor e Propriedade Intelectual
Direitos de Autor e Propriedade IntelectualBruno M.C09
 

Destaque (9)

Historia de la economía
Historia de la economíaHistoria de la economía
Historia de la economía
 
GayaThesisFinal
GayaThesisFinalGayaThesisFinal
GayaThesisFinal
 
Jobs in Bangalore
Jobs in  BangaloreJobs in  Bangalore
Jobs in Bangalore
 
Certificacion APRACOM
Certificacion APRACOMCertificacion APRACOM
Certificacion APRACOM
 
McKinneyBrianF
McKinneyBrianFMcKinneyBrianF
McKinneyBrianF
 
Eset. Дмитрий Самойленко. "Eset. Антивирусная защита для вашего бизнеса"
Eset. Дмитрий Самойленко. "Eset. Антивирусная защита для вашего бизнеса"Eset. Дмитрий Самойленко. "Eset. Антивирусная защита для вашего бизнеса"
Eset. Дмитрий Самойленко. "Eset. Антивирусная защита для вашего бизнеса"
 
Customer Experience Matters
Customer Experience MattersCustomer Experience Matters
Customer Experience Matters
 
동국대학교 중앙동아리 D.N.A 2014년도 동아리 창립제 발표 자료 - 자료구조 스터디
동국대학교 중앙동아리 D.N.A 2014년도 동아리 창립제 발표 자료 - 자료구조 스터디동국대학교 중앙동아리 D.N.A 2014년도 동아리 창립제 발표 자료 - 자료구조 스터디
동국대학교 중앙동아리 D.N.A 2014년도 동아리 창립제 발표 자료 - 자료구조 스터디
 
Direitos de Autor e Propriedade Intelectual
Direitos de Autor e Propriedade IntelectualDireitos de Autor e Propriedade Intelectual
Direitos de Autor e Propriedade Intelectual
 

Mais de C4Media

Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoStreaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoC4Media
 
Next Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileNext Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileC4Media
 
Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020C4Media
 
Understand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsUnderstand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsC4Media
 
Kafka Needs No Keeper
Kafka Needs No KeeperKafka Needs No Keeper
Kafka Needs No KeeperC4Media
 
High Performing Teams Act Like Owners
High Performing Teams Act Like OwnersHigh Performing Teams Act Like Owners
High Performing Teams Act Like OwnersC4Media
 
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaDoes Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaC4Media
 
Service Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideService Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideC4Media
 
Shifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDShifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDC4Media
 
CI/CD for Machine Learning
CI/CD for Machine LearningCI/CD for Machine Learning
CI/CD for Machine LearningC4Media
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at SpeedC4Media
 
Architectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsArchitectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsC4Media
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsC4Media
 
Build Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerBuild Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerC4Media
 
User & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleUser & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleC4Media
 
Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeScaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeC4Media
 
Make Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereMake Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereC4Media
 
The Talk You've Been Await-ing For
The Talk You've Been Await-ing ForThe Talk You've Been Await-ing For
The Talk You've Been Await-ing ForC4Media
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data EngineeringC4Media
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreC4Media
 

Mais de C4Media (20)

Streaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live VideoStreaming a Million Likes/Second: Real-Time Interactions on Live Video
Streaming a Million Likes/Second: Real-Time Interactions on Live Video
 
Next Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy MobileNext Generation Client APIs in Envoy Mobile
Next Generation Client APIs in Envoy Mobile
 
Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020Software Teams and Teamwork Trends Report Q1 2020
Software Teams and Teamwork Trends Report Q1 2020
 
Understand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java ApplicationsUnderstand the Trade-offs Using Compilers for Java Applications
Understand the Trade-offs Using Compilers for Java Applications
 
Kafka Needs No Keeper
Kafka Needs No KeeperKafka Needs No Keeper
Kafka Needs No Keeper
 
High Performing Teams Act Like Owners
High Performing Teams Act Like OwnersHigh Performing Teams Act Like Owners
High Performing Teams Act Like Owners
 
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to JavaDoes Java Need Inline Types? What Project Valhalla Can Bring to Java
Does Java Need Inline Types? What Project Valhalla Can Bring to Java
 
Service Meshes- The Ultimate Guide
Service Meshes- The Ultimate GuideService Meshes- The Ultimate Guide
Service Meshes- The Ultimate Guide
 
Shifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CDShifting Left with Cloud Native CI/CD
Shifting Left with Cloud Native CI/CD
 
CI/CD for Machine Learning
CI/CD for Machine LearningCI/CD for Machine Learning
CI/CD for Machine Learning
 
Fault Tolerance at Speed
Fault Tolerance at SpeedFault Tolerance at Speed
Fault Tolerance at Speed
 
Architectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep SystemsArchitectures That Scale Deep - Regaining Control in Deep Systems
Architectures That Scale Deep - Regaining Control in Deep Systems
 
ML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.jsML in the Browser: Interactive Experiences with Tensorflow.js
ML in the Browser: Interactive Experiences with Tensorflow.js
 
Build Your Own WebAssembly Compiler
Build Your Own WebAssembly CompilerBuild Your Own WebAssembly Compiler
Build Your Own WebAssembly Compiler
 
User & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix ScaleUser & Device Identity for Microservices @ Netflix Scale
User & Device Identity for Microservices @ Netflix Scale
 
Scaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's EdgeScaling Patterns for Netflix's Edge
Scaling Patterns for Netflix's Edge
 
Make Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home EverywhereMake Your Electron App Feel at Home Everywhere
Make Your Electron App Feel at Home Everywhere
 
The Talk You've Been Await-ing For
The Talk You've Been Await-ing ForThe Talk You've Been Await-ing For
The Talk You've Been Await-ing For
 
Future of Data Engineering
Future of Data EngineeringFuture of Data Engineering
Future of Data Engineering
 
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and MoreAutomated Testing for Terraform, Docker, Packer, Kubernetes, and More
Automated Testing for Terraform, Docker, Packer, Kubernetes, and More
 

Último

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Alan Dix
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Allon Mureinik
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitecturePixlogix Infotech
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationRidwan Fadjar
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxMalak Abu Hammad
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...HostedbyConfluent
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Paola De la Torre
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking MenDelhi Call girls
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure servicePooja Nehwal
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationRadu Cotescu
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)Gabriella Davis
 

Último (20)

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...Swan(sea) Song – personal research during my six years at Swansea ... and bey...
Swan(sea) Song – personal research during my six years at Swansea ... and bey...
 
Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)Injustice - Developers Among Us (SciFiDevCon 2024)
Injustice - Developers Among Us (SciFiDevCon 2024)
 
Understanding the Laravel MVC Architecture
Understanding the Laravel MVC ArchitectureUnderstanding the Laravel MVC Architecture
Understanding the Laravel MVC Architecture
 
My Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 PresentationMy Hashitalk Indonesia April 2024 Presentation
My Hashitalk Indonesia April 2024 Presentation
 
The Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptxThe Codex of Business Writing Software for Real-World Solutions 2.pptx
The Codex of Business Writing Software for Real-World Solutions 2.pptx
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
Transforming Data Streams with Kafka Connect: An Introduction to Single Messa...
 
Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101Salesforce Community Group Quito, Salesforce 101
Salesforce Community Group Quito, Salesforce 101
 
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men08448380779 Call Girls In Greater Kailash - I Women Seeking Men
08448380779 Call Girls In Greater Kailash - I Women Seeking Men
 
Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure serviceWhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
WhatsApp 9892124323 ✓Call Girls In Kalyan ( Mumbai ) secure service
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)A Domino Admins Adventures (Engage 2024)
A Domino Admins Adventures (Engage 2024)
 

How We Scaled Songkick for More Traffic and More Productive Development

  • 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /songkick-scale
  • 3. Presented at QCon London www.qconlondon.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
  • 4. songkick.com • Founded 2007 • Hundreds of thousands of upcoming concerts • 3.4 million past concerts • 8 million uniques a month • Second most visited live music website after ticketmaster Friday, 8 March 13
  • 5. 3 slides: Overview of songkick. Pictures. How did it start? How big is company? How many devs etc? How were you architecting stuff? What roles? How is team structured? What problems did this cause / did you face? Friday, 8 March 13
  • 6. We Started small • Four people in a flat in Spitalfields • And grew Friday, 8 March 13
  • 7. We are still small • 30 People in an office in Hoxton • We are divided into cross functional teams the number and size of which change as we need Friday, 8 March 13
  • 8. We also do Maybe not the iphone and android applications. Though they use some similar concepts and certainly rest on some of the same infrastructure. In the case of the iPhone and Android applications the way we know which artists you are interested in is we look on you device.We also use geolocation to find where you are and to notify you, we use push notification. Again this is just for completeness we are probably not going to mention them much But I’m not going to be talking directly about these, although they do use a similar architecture. Friday, 8 March 13
  • 10. The old architecture skweb Mysql A rails application Friday, 8 March 13
  • 11. What was the problem • Initially features were over-engineered • To develop and ship quickly it was easier to stick it all in one place • But site was up, traffic growing.Trouble brewing … Friday, 8 March 13
  • 12. What’s the problem? • Shipping new features became difficult • Our builds were taking hours to run • We had complex relationships between what were notionally separate applications • Dependancies were hard to understand and hard to untangle All these things meant if you wanted to change something, if you wanted to change the copy in an emails, you had to deploy the entire app. We had a few false starts where we broke up the functions of the application. Unfortunately the boundaries weren’t clear and it was still a single code base so we still had to deploy everything together Integration queue Friday, 8 March 13
  • 13. Our dependency graph We wrote software to tell us what the dependancies were of the components of our software. It wasn’t pretty. Friday, 8 March 13
  • 14. Why re-architect? We can respond to competitors and changes in the market more readily. For us increasing productivity was one of our most important goals. Yes performance was import, and, yes we had a lot of code in the app to make it more performant (caching etc) Which did make it reasonably performant. Friday, 8 March 13
  • 15. Why re-architect? • Scale (more users doing more things) We can respond to competitors and changes in the market more readily. For us increasing productivity was one of our most important goals. Yes performance was import, and, yes we had a lot of code in the app to make it more performant (caching etc) Which did make it reasonably performant. Friday, 8 March 13
  • 16. Why re-architect? • Scale (more users doing more things) We can respond to competitors and changes in the market more readily. • Developer productivity (more features, fewer bugs) For us increasing productivity was one of our most important goals. Yes performance was import, and, yes we had a lot of code in the app to make it more performant (caching etc) Which did make it reasonably performant. Friday, 8 March 13
  • 17. Why re-architect? • Scale (more users doing more things) We can respond to competitors and changes in the market more readily. • Developer productivity (more features, fewer bugs) • Agility (more frequent releases, shorter time between releases) For us increasing productivity was one of our most important goals. Yes performance was import, and, yes we had a lot of code in the app to make it more performant (caching etc) Which did make it reasonably performant. Friday, 8 March 13
  • 18. Why not re-architect? These were all real concerns. How do you persuade the other people in the company that spending six months doing this is worth doing? This is a start-up you really can’t spend six month navel gazing. The company could go bust. You need to have a compelling reason and a plan. Friday, 8 March 13
  • 19. Why not re-architect? • You might never finish These were all real concerns. How do you persuade the other people in the company that spending six months doing this is worth doing? This is a start-up you really can’t spend six month navel gazing. The company could go bust. You need to have a compelling reason and a plan. Friday, 8 March 13
  • 20. Why not re-architect? • You might never finish These were all real concerns. • You might not achieve the benefits How do you persuade the other people in the company that spending six months doing this is worth doing? This is a start-up you really can’t spend six month navel gazing. The company could go bust. You need to have a compelling reason and a plan. Friday, 8 March 13
  • 21. Why not re-architect? • You might never finish These were all real concerns. • You might not achieve the benefits • It might be easier to rewrite How do you persuade the other people in the company that spending six months doing this is worth doing? This is a start-up you really can’t spend six month navel gazing. The company could go bust. You need to have a compelling reason and a plan. Friday, 8 March 13
  • 22. Why not re-architect? • You might never finish These were all real concerns. • You might not achieve the benefits • It might be easier to rewrite • The new architecture might not be better than the old one How do you persuade the other people in the company that spending six months doing this is worth doing? This is a start-up you really can’t spend six month navel gazing. The company could go bust. You need to have a compelling reason and a plan. Friday, 8 March 13
  • 23. Collaboration! How did we know what cut and what to keep. iPhone. Why what things are called? Shared vocabulary, every one in the company calls the same thing by the same name. This will become important later on, in the development of the application. And actually this process of identifying, cutting or adding features, choosing names and prioritising work is iterative. Each step of the way this process is repeated. Friday, 8 March 13
  • 24. Collaboration! • Software is built by a team How did we know what cut and what to keep. iPhone. Why what things are called? Shared vocabulary, every one in the company calls the same thing by the same name. This will become important later on, in the development of the application. And actually this process of identifying, cutting or adding features, choosing names and prioritising work is iterative. Each step of the way this process is repeated. Friday, 8 March 13
  • 25. Collaboration! • Software is built by a team How did we know what cut and what to keep. iPhone. • Not just a team of programmers Why what things are called? Shared vocabulary, every one in the company calls the same thing by the same name. This will become important later on, in the development of the application. And actually this process of identifying, cutting or adding features, choosing names and prioritising work is iterative. Each step of the way this process is repeated. Friday, 8 March 13
  • 26. Collaboration! • Software is built by a team How did we know what cut and what to keep. iPhone. • Not just a team of programmers • You need to agree on what can be cut Why what things are called? Shared vocabulary, every one in the company calls the same thing by the same name. This will become important later on, in the development of the application. And actually this process of identifying, cutting or adding features, choosing names and prioritising work is iterative. Each step of the way this process is repeated. Friday, 8 March 13
  • 27. Collaboration! • Software is built by a team How did we know what cut and what to keep. iPhone. • Not just a team of programmers • You need to agree on what can be cut • What is the minimum feature set needed Why what things are called? Shared vocabulary, every one in the company calls the same thing by the same name. This will become important later on, in the development of the application. And actually this process of identifying, cutting or adding features, choosing names and prioritising work is iterative. Each step of the way this process is repeated. Friday, 8 March 13
  • 28. Collaboration! • Software is built by a team How did we know what cut and what to keep. iPhone. • Not just a team of programmers • You need to agree on what can be cut • What is the minimum feature set needed • What things are called Why what things are called? Shared vocabulary, every one in the company calls the same thing by the same name. This will become important later on, in the development of the application. And actually this process of identifying, cutting or adding features, choosing names and prioritising work is iterative. Each step of the way this process is repeated. Friday, 8 March 13
  • 29. What we settled on Which is kind of boring, but, how you get from one to the other is more interesting. And doing it in a reasonable amount of time and without breaking the existing site and not doing a big bang release is a challenge. We decided early on that moving to the new architecture would be done in a stepwise fashion.With the refactoring and splitting of the functions one page at a time. We had a pages that were functionally quite distinct.And if you want to do this step by step you need a unit you can use to measure progress and divide up the work. I should emphasise this was not handed down on graven tablets by the development team, we arrived here by explaining why this was the best option. Friday, 8 March 13
  • 30. What we settled on • Services Which is kind of boring, but, how you get from one to the other is more interesting. And doing it in a reasonable amount of time and without breaking the existing site and not doing a big bang release is a challenge. We decided early on that moving to the new architecture would be done in a stepwise fashion.With the refactoring and splitting of the functions one page at a time. We had a pages that were functionally quite distinct.And if you want to do this step by step you need a unit you can use to measure progress and divide up the work. I should emphasise this was not handed down on graven tablets by the development team, we arrived here by explaining why this was the best option. Friday, 8 March 13
  • 31. What we settled on • Services • And Clients Which is kind of boring, but, how you get from one to the other is more interesting. And doing it in a reasonable amount of time and without breaking the existing site and not doing a big bang release is a challenge. We decided early on that moving to the new architecture would be done in a stepwise fashion.With the refactoring and splitting of the functions one page at a time. We had a pages that were functionally quite distinct.And if you want to do this step by step you need a unit you can use to measure progress and divide up the work. I should emphasise this was not handed down on graven tablets by the development team, we arrived here by explaining why this was the best option. Friday, 8 March 13
  • 32. So far, so conventional Friday, 8 March 13
  • 33. What a service looks like We actually started with dummy services where we implemented the interface to the service inside the application. Active record leaks up the Worth noting our services don’t have versioning, access control or XML. And that we do not need to maintain backwards compatibility, since we control all the clients. (at least for now) And they are kind of REST. Not real, phd level REST, but certainly popular, Rails-developer style REST. Friday, 8 March 13
  • 34. What a service looks like We actually started with dummy services where we implemented the interface to the service inside the application. Active record leaks up the • Sinatra application Worth noting our services don’t have versioning, access control or XML. And that we do not need to maintain backwards compatibility, since we control all the clients. (at least for now) And they are kind of REST. Not real, phd level REST, but certainly popular, Rails-developer style REST. Friday, 8 March 13
  • 35. What a service looks like We actually started with dummy services where we implemented the interface to the service inside the application. Active record leaks up the • Sinatra application • Emits JSON over HTTP Worth noting our services don’t have versioning, access control or XML. And that we do not need to maintain backwards compatibility, since we control all the clients. (at least for now) And they are kind of REST. Not real, phd level REST, but certainly popular, Rails-developer style REST. Friday, 8 March 13
  • 36. What a service looks like We actually started with dummy services where we implemented the interface to the service inside the application. Active record leaks up the • Sinatra application • Emits JSON over HTTP • Accepts form encoded or JSON data over HTTP Worth noting our services don’t have versioning, access control or XML. And that we do not need to maintain backwards compatibility, since we control all the clients. (at least for now) And they are kind of REST. Not real, phd level REST, but certainly popular, Rails-developer style REST. Friday, 8 March 13
  • 37. What a service looks like We actually started with dummy services where we implemented the interface to the service inside the application. Active record leaks up the • Sinatra application • Emits JSON over HTTP • Accepts form encoded or JSON data over HTTP • Completely internally encapsulated Worth noting our services don’t have versioning, access control or XML. And that we do not need to maintain backwards compatibility, since we control all the clients. (at least for now) And they are kind of REST. Not real, phd level REST, but certainly popular, Rails-developer style REST. Friday, 8 March 13
  • 38. What a client application looks like Pages are ruby classes that model what a pages behaviour should be Friday, 8 March 13
  • 39. What a client application looks like • Rails application (so far any way) Pages are ruby classes that model what a pages behaviour should be Friday, 8 March 13
  • 40. What a client application looks like • Rails application (so far any way) • Has a traditional ‘MVC’ structure Pages are ruby classes that model what a pages behaviour should be Friday, 8 March 13
  • 41. What a client application looks like • Rails application (so far any way) • Has a traditional ‘MVC’ structure • Gets all its data from services Pages are ruby classes that model what a pages behaviour should be Friday, 8 March 13
  • 42. What a client application looks like • Rails application (so far any way) • Has a traditional ‘MVC’ structure • Gets all its data from services • We added ‘pages’,‘components’ and ‘elements’ Pages are ruby classes that model what a pages behaviour should be Friday, 8 March 13
  • 43. How it fits togetherWhat is a page? Why add pages? What are components? What are elements? Benefits? What makes a component? A self contained unit on the page normally you can draw a box around it and give it a name. Arbitrarily components cannot be nested. What makes an element? Are common functionality shared between components. Friday, 8 March 13
  • 44. How it fits together • A Page What is a page? Why add pages? What are components? What are elements? Benefits? What makes a component? A self contained unit on the page normally you can draw a box around it and give it a name. Arbitrarily components cannot be nested. What makes an element? Are common functionality shared between components. Friday, 8 March 13
  • 45. How it fits together • A Page • Is made of components What is a page? Why add pages? What are components? What are elements? Benefits? What makes a component? A self contained unit on the page normally you can draw a box around it and give it a name. Arbitrarily components cannot be nested. What makes an element? Are common functionality shared between components. Friday, 8 March 13
  • 46. How it fits together • A Page • Is made of components • Some components are composed of elements What is a page? Why add pages? What are components? What are elements? Benefits? What makes a component? A self contained unit on the page normally you can draw a box around it and give it a name. Arbitrarily components cannot be nested. What makes an element? Are common functionality shared between components. Friday, 8 March 13
  • 48. Layers (it’s all about layers) Event pages need a venue and an artist and an event Artist pages have an artist + calendar and media (see etc) Users are User + calendar Components Artist Page User Page Venue PageEvent Page Pages Event Artist UserCalendar Venue Etc … Models Accounts NotificationsCaltrakEvent listings Etc … Services Services here are classes in the client.They talk to the network and handle passing the data up to the models and dat from the models out to the network Friday, 8 March 13
  • 49. So how does all this work in practice? Friday, 8 March 13
  • 50. So how does all this work in practice? • The client is still a rails app with the familiar rails layout Friday, 8 March 13
  • 51. So how does all this work in practice? • The client is still a rails app with the familiar rails layout • Anywhere a rails app might talk to a data store, the app talks to a service instead Friday, 8 March 13
  • 52. So how does all this work in practice? • The client is still a rails app with the familiar rails layout • Anywhere a rails app might talk to a data store, the app talks to a service instead • And we have added some conventions Friday, 8 March 13
  • 53. Components are self contained. Each component has a name and takes an object. The object contains the data the component needs and any decision making is provided by methods on that object. The name of the component is also the name of the template file on disc, the html class name and the name of its corresponding css and javascript files. This tight convention around names makes understanding the dependancy between a The conventions Every page having a css file does mean you get some repetition, but, the confidence it gives you about where changes will appear makes it well worth it. The css file imports smaller files with shared styling Friday, 8 March 13
  • 54. • Every Page has a type Components are self contained. Each component has a name and takes an object. The object contains the data the component needs and any decision making is provided by methods on that object. The name of the component is also the name of the template file on disc, the html class name and the name of its corresponding css and javascript files. This tight convention around names makes understanding the dependancy between a The conventions Every page having a css file does mean you get some repetition, but, the confidence it gives you about where changes will appear makes it well worth it. The css file imports smaller files with shared styling Friday, 8 March 13
  • 55. • Every Page has a type • Every page has one CSS file Components are self contained. Each component has a name and takes an object. The object contains the data the component needs and any decision making is provided by methods on that object. The name of the component is also the name of the template file on disc, the html class name and the name of its corresponding css and javascript files. This tight convention around names makes understanding the dependancy between a The conventions Every page having a css file does mean you get some repetition, but, the confidence it gives you about where changes will appear makes it well worth it. The css file imports smaller files with shared styling Friday, 8 March 13
  • 56. • Every Page has a type • Every page has one CSS file • The CSS file has the same name as the page type Components are self contained. Each component has a name and takes an object. The object contains the data the component needs and any decision making is provided by methods on that object. The name of the component is also the name of the template file on disc, the html class name and the name of its corresponding css and javascript files. This tight convention around names makes understanding the dependancy between a The conventions Every page having a css file does mean you get some repetition, but, the confidence it gives you about where changes will appear makes it well worth it. The css file imports smaller files with shared styling Friday, 8 March 13
  • 57. • Every Page has a type • Every page has one CSS file • The CSS file has the same name as the page type • Every component has a corresponding CSS file Components are self contained. Each component has a name and takes an object. The object contains the data the component needs and any decision making is provided by methods on that object. The name of the component is also the name of the template file on disc, the html class name and the name of its corresponding css and javascript files. This tight convention around names makes understanding the dependancy between a The conventions Every page having a css file does mean you get some repetition, but, the confidence it gives you about where changes will appear makes it well worth it. The css file imports smaller files with shared styling Friday, 8 March 13
  • 58. • Every Page has a type • Every page has one CSS file • The CSS file has the same name as the page type • Every component has a corresponding CSS file • If it needs it the component also has a javascript file Components are self contained. Each component has a name and takes an object. The object contains the data the component needs and any decision making is provided by methods on that object. The name of the component is also the name of the template file on disc, the html class name and the name of its corresponding css and javascript files. This tight convention around names makes understanding the dependancy between a The conventions Every page having a css file does mean you get some repetition, but, the confidence it gives you about where changes will appear makes it well worth it. The css file imports smaller files with shared styling Friday, 8 March 13
  • 59. A little bit of code This is of cause in ruby but that hardly matters. Friday, 8 March 13
  • 60. A little bit of code This is of cause in ruby but that hardly matters. skweb/     app/         controllers/             venues_controller.rb         models/             page_models/                 venue.rb             skweb/                 models/                     venue.rb         views/             venues/                 _brief.html.erb                 show.html.erb     public/         javascripts/             songkick/                 component/                     tickets.js         stylesheets/             components/                 venue-brief.css             shared/                 components/                     brief.css             venue.css Friday, 8 March 13
  • 61. A little bit of code This is of cause in ruby but that hardly matters. skweb/     app/         controllers/             venues_controller.rb         models/             page_models/                 venue.rb             skweb/                 models/                     venue.rb         views/             venues/                 _brief.html.erb                 show.html.erb     public/         javascripts/             songkick/                 component/                     tickets.js         stylesheets/             components/                 venue-brief.css             shared/                 components/                     brief.css             venue.css class VenuesController < ApplicationController   def show     @page = PageModels::Venue.new(venue, logged_in_user)   end end Friday, 8 March 13
  • 62. A little bit of code This is of cause in ruby but that hardly matters. skweb/     app/         controllers/             venues_controller.rb         models/             page_models/                 venue.rb             skweb/                 models/                     venue.rb         views/             venues/                 _brief.html.erb                 show.html.erb     public/         javascripts/             songkick/                 component/                     tickets.js         stylesheets/             components/                 venue-brief.css             shared/                 components/                     brief.css             venue.css class VenuesController < ApplicationController   def show     @page = PageModels::Venue.new(venue, logged_in_user)   end end module PageModels   class Venue < PageModels::Base     def brief       Brief.new(@venue, upcoming_events.total_entries, @logged_in_user)     end  end end Friday, 8 March 13
  • 63. A little bit of code This is of cause in ruby but that hardly matters. skweb/     app/         controllers/             venues_controller.rb         models/             page_models/                 venue.rb             skweb/                 models/                     venue.rb         views/             venues/                 _brief.html.erb                 show.html.erb     public/         javascripts/             songkick/                 component/                     tickets.js         stylesheets/             components/                 venue-brief.css             shared/                 components/                     brief.css             venue.css class VenuesController < ApplicationController   def show     @page = PageModels::Venue.new(venue, logged_in_user)   end end module PageModels   class Venue < PageModels::Base     def brief       Brief.new(@venue, upcoming_events.total_entries, @logged_in_user)     end  end end module PageModels   class Venue     class Brief       def geolocation         @venue.geolocation       end     end   end end Friday, 8 March 13
  • 64. Moving to the view component() and shared_component() are defined in ApplicationHelper and look like this: Friday, 8 March 13
  • 65. Moving to the view <div class="primary col">   <%= component('brief', @page.brief) %>   <%= component('map', @page.brief.geolocation) %>   <%= shared_component('calendar_summary',   @page.calendar_summary) %>   <%= shared_component('media_summary',      @page.media_summary) %>   <%= shared_component('media_links',        @page.media_links) %>   <%= shared_component('gigography_summary', @page.gigography_summary) %> </div> component() and shared_component() are defined in ApplicationHelper and look like this: Friday, 8 March 13
  • 66. Moving to the view <div class="primary col">   <%= component('brief', @page.brief) %>   <%= component('map', @page.brief.geolocation) %>   <%= shared_component('calendar_summary',   @page.calendar_summary) %>   <%= shared_component('media_summary',      @page.media_summary) %>   <%= shared_component('media_links',        @page.media_links) %>   <%= shared_component('gigography_summary', @page.gigography_summary) %> </div> component() and shared_component() are defined in ApplicationHelper and look like this: def component(component_name, object)   return '' if object.nil?   render :partial => component_name, :object => object end def shared_component(component_name, object)   component("shared/components/#{component_name}", object) end Friday, 8 March 13
  • 67. Moving to the view <div class="primary col">   <%= component('brief', @page.brief) %>   <%= component('map', @page.brief.geolocation) %>   <%= shared_component('calendar_summary',   @page.calendar_summary) %>   <%= shared_component('media_summary',      @page.media_summary) %>   <%= shared_component('media_links',        @page.media_links) %>   <%= shared_component('gigography_summary', @page.gigography_summary) %> </div> component() and shared_component() are defined in ApplicationHelper and look like this: def component(component_name, object)   return '' if object.nil?   render :partial => component_name, :object => object end def shared_component(component_name, object)   component("shared/components/#{component_name}", object) end @import 'shared/components/brief.css'; @import 'components/venue-brief.css'; @import 'components/venue-map.css'; @import 'shared/components/media-summary.css'; @import 'shared/components/event-listings.css'; Friday, 8 March 13
  • 68. What did this give us I’d hoped to have a graph showing improved page response times, but unfortunately we didn’t keep them Many of our services can be scaled horizontally mean at lest in the medium term we can increase capacity by adding nodes The compartmentalisation of the application. The independence of the services means parallelising development is simpler. Knowing where to add functionality is easier. Friday, 8 March 13
  • 69. What did this give us • Developer productivity was radically improved I’d hoped to have a graph showing improved page response times, but unfortunately we didn’t keep them Many of our services can be scaled horizontally mean at lest in the medium term we can increase capacity by adding nodes The compartmentalisation of the application. The independence of the services means parallelising development is simpler. Knowing where to add functionality is easier. Friday, 8 March 13
  • 70. What did this give us • Developer productivity was radically improved • Application performance was much better I’d hoped to have a graph showing improved page response times, but unfortunately we didn’t keep them Many of our services can be scaled horizontally mean at lest in the medium term we can increase capacity by adding nodes The compartmentalisation of the application. The independence of the services means parallelising development is simpler. Knowing where to add functionality is easier. Friday, 8 March 13
  • 71. A leaner code base • Before    After • 3.5MB       1.4MB    ./app • 1.8MB       744KB    ./features • 1.2MB       724KB    ./spec Friday, 8 March 13
  • 72. Faster Builds Before • Over an hour • Parallelized with 1 local and 10 ec2 instances After • 10 minutes • 1 local machine Friday, 8 March 13
  • 73. Weekly visits Visits Songkick 2 launched 9 June 2009 Christmas 2008 2009 2010 2011 2012 2013 Started rearchitecture Finished rearchitecture A time line of how our traffic has grown. Traffic grew all threw 2010 we were adding features, iterating madly to improve conversion and user engagement. Friday, 8 March 13
  • 74. Releases per month 0 25 50 75 100 125 150 Mar 2011 Aug 2011 Nov 2011 Feb 2012 May 2012 Aug 2012 Nov 2012 Just to prove we were shipping more once we finished the product. Friday, 8 March 13