At BlaBlaCar, we’ve always tried to be at the forefront of continuous delivery of the application. However our infrastructure was initially manually provisioned, so we decided to address that, inspired by our software colleagues. Leveraging Chef and The Foreman, we’ve been able to build an automated infrastructure provisioning process that could cope with our geographical development and members growth. This had to be done without affecting an application delivery process consistently pulling out 10 builds a day. This talk will cover the challenges encountered along the way, where we are today, and where we want to be tomorrow.

1. Continuous Delivery of
Infrastructure
The case of the growing startup

2. Nicolas Blanc - BlaBlArchitect
SinfomicSinfomic
(1999)
@thewhitegeek
(2001)
(2005)
(2008)
(2012)

3. 3/49
Summary
● Introduction
● Why Continuous Delivery of Infrastructure
● Automate Configuration
● Hardware Industrialization
● Automate OS Install
● Hardware availability
● Continuous delivery

4. Introduction

5. 80€
20€
20€
20€
@BlaBlaCar_FR
20€
Trusted Ridesharing

6. People Powered Travel
A travel
search
engine
A trusted
community
@BlaBlaCar_FR

7. Fast growth
@BlaBlaCar_FR

8. 8/49
Why Infrastructure Automation ?

9. 9/49
Few numbers of BlaBlaCar today
2 Data Centers in Paris
● 242 servers
● 109 virtual servers (VMWare)
● 110 EC2 instances during working day
– 20 instances the rest of the time

10. Our Continuous Delivery at the top.
Jealous from the development team !

11. 11/49
We have a strong agility culture
● Push our main app on production 10x a day
● Developers make the push themselves
● App can be pushed only after a Bamboo's merge
● Bamboo merge only after thousands of tests

12. 12/49
tools to manage them all

13. Old Infrastructure Overview

14. 14/49
Physical Server
● 3 racks on a single DC
● 12 servers per rack
● More than 20 differents
hardware configurations

15. 15/49
Cloud (Amazon EC2)
● Only 10 to 20 instances always running
● Used for elasticity
● 10 to 20 instances running for test (Bamboo)

16. 16/49
Network
● Strong Team who designed a
very strong network
– Rémi (@shtouff)
● Not a problem for us to scale
the network

17. Automate the Configuration

18. 18/49
Choosing the right tool

19. 19/49
Improve our Service Deployment
● Started to work on a front web server
– Ease configuration deployment
– Ease a new installation
● Then worked on all servers
– First just for configuration
– Then all service deployment

20. Hardware Industrialization

21. 21/49
Haute Couture
● Before our automation process:
– One server configuration by service
– Multiple services

22. 22/49
Haute Couture - Caveats
● Tailored server are complicated to design
● Hard to upgrade

23. 23/49
Next Step - Virtualization
● Create a virtualization platform
– 4 big servers
– A big disk array

24. 24/49
Virtualization - Caveats
● I/Os Performances compare to physical server
● Network Performances

25. 25/49
Cloud: Why on premise in 2015 ?
● Data privacy concerns
● Limited elasticity required
● Cost efficiency
● History & Culture

26. 26/49
Now – Standard Servers
● Define a single 1U server
– Dual-CPU (12 cores)
– 128 Go of RAM
– Disk configuration vary on 3 flavors:
● Only 2 SSD mirrored (RAID)
● Fast disk (SSD) needed
● Very big capacity (lot of disks)

27. Automate OS Installation

28. 28/49
Automate debian installation
● All our infra is debian based
● PXE
● ipmi
● preseed
– (thx to @jbfavre)

29. 29/49
Virtualization => Template
● Work well on EC2 (only option for instance
install)
● Never used in VMWare
– Same install as physical server debian based
(thanks to Chef)

30. 30/49
Define our needs to choose a tool
● IP allocation
● Bootstrap chef-client
● PXE preseed debian
● Can install either a physical, virtual or cloud server
● Configure the BMC interface
– HP (iLo)
– Dell (DRAC)
– SuperMicro

31. 31/49
● Because Simon (@s_ourea) chose it !
– More likely to be maintained
– Big community
● Fits our needs

32. Hardware Availability

33. 33/49
We need more
● Chef / Foreman, great tools to:
– Install OS
– Install all services and configure them
● No easy solution when you need a specific
hardware on never used server

34. 34/49
Collins !
● Nice tool to manage our hardware inventory.
● A lot of good reasons to choose it:
– Simon (@s_ourea) chose it
– Fit our needs
● Linked with
Foreman

35. Hey ? What about the initial setup of
a physical server ?

36. 36/49
Install servers into DC
● Limited added value in the team racking servers
● Time consuming
● DCs can be far away
● Simon(@s_ourea) & Rémi(@shtouff) pushed a new idea
– Buy servers only when we can use a full rack
– Write an installation documentation
– Choose a company to rack

37. 37/49
Our procedure to have a new rack
● Buy a new rack into the DC
● Our network team installs 2 switchs in the rack.
● They add links to our core network equipments
● Hey! Let's ask to our provider to rack and setup all
servers, following Simon's documentation

38. 38/49
The typical life of a server
● Discovery, and go into Collins
● Another team takes it and installs it via Foreman
● Chef is called via Foreman
● Chef is always running

39. Tests, and Continuous delivery
really ?

40. 40/49
Chef and tests
● Yes, we test all of our
infrastructure's services !
● Thanks to Chef and all of it's community:
KitchenCI + vagrant + serverspec, chefspec

41. 41/49
Chef - Cookbook life
● Each cookbook in it's own git repository
● First test and merge with Bamboo
● Push in production with a new version number
● All servers converge to the desired state

42. 42/49
What do you test ?
● Everything! (one day perhaps)
– Server install
– Bootstrapping
– Convergence
– Clustering /! (not automatic)
– Version upgrade ( one day !)
– Coffee quality

43. 43/49
Continuous delivery, really ?
● New cookbook versions pushed 10x times / day
● Provision several new servers / VM / instances by
day
● Buy a rack full of servers, and install them in less
than a day

44. So cool ! But...
What will you do in the future ?

45. 45/49
Container
CoreOS

46. 46/49
What we will continue to do
● Test our infra (with or without container)
● Industrialize our processes
● Spread knowledge to all other teams
● Dream solutions, and finally build them
● Listen to Simon (@s_ourea)

47. Conclusion

48. 48/49
What to keep in mind
● Involve overall company
● Support company growth
● Don't get attached to tools
● Keep a “small team” spirit

49. That's all folks!
Thanks you!
Nicolas Blanc
(@thewhitegeek)