Presentation of CI Process at Spectos Asia Ltd., slides from my talk at FOSSASIA 2015

1. Continuous Integration
with Jenkins, Github & Grunt

3. - Integrate any changes
within a matter of hours
into the product
- Collection of tools and
practices to make that
possible
- Originating from XP,
now an essential
practice in Agile SD
What is CI?

4. RPM/Spectos Cloud

5. ● LAMPP
● MEAN
● Ubuntu
● AWS, EC2
● Jenkins
● JIRA
● Github
Technology Stack

6. Motivation
- RPM: Frontend + 10 backend services
- New features usually affect at least frontend
+ one (maybe 2 or 3) backends
- How to know everything works well with
each other?
- Who determines when we can ship to the
customer?

7. Practices
● Single Source Repository
● Automate the build
● Self-Testing
● Commit often
● Build every commit

8. Practices (cont.)
● Fix broken builds ASAP
● Keep builds fast
● Easy access to latest version
● Everybody can see what’s happening
● Automate deployment

9. Source Code Repository
- Not one “single”
repo but one per
project
- It’s all on https:
//github.
com/Spectos
(private, sorry)

10. Source Code Repository
- Mainline branch:

11. Source Code Repository
Sources
Docs
Depende
ncies
Tests Cronjobs
IDE
Settings
Build
Script
Config

12. Build Automation
- Using Grunt: http:
//gruntjs.com/
- Javascript taskrunner,
based on Node.js
- Simple syntax (JS)
- Easy to wrap around
other tools (Cake Shell,
bash)
- Lot of plugins

13. Build Automation
- Database backups,
migrations
- Generate locales (ng)
- Run tests (mocha.js or
Cake shell)
- Deploy via shipit.js (like
Capistrano)
- Init work queues, clear
caches, etc.

14. Build Automation
// start development server
grunt s
// run all tests
grunt test
// deploy to stage
grunt shipit:stage deploy

15. Testing

16. Testing
- Automated tests for each functionality added
(TDD is recommended)
- Using PHPUnit for PHP or mocha for Node.
js apps
- Part of every build
- Automated frontend tests with Selenium
(WIP)

17. Commit Often
- On your local: multiple times per day, easy
to experiment and undo changes (git
reset --hard ftw.)
- Remote: Pull & rebase often to avoid merge
conflicts
- Push every feature when you consider it
done so it can be integrated & tested

18. Fix Broken Builds ASAP
- Emails are not very
effective
- Desktop notifications,
sounds
- Integrations (Slack)
- Broken build is top
priority!

19. Speed
- Full commit build
not > 10 min
- Currently: ~6 min
for full ecosystem
- Component builds
(one per service)

20. Speed - How To?
- Avoid expensive
stuff in tests (I/O,
large datasets)
- Run things parallel
- Consider build
pipeline (not
necessary yet)

21. Build Triggers
- Jenkins
- Using Github hooks
- Every commit pushed to
dev/master will trigger a
build
- Production deploy: manual

22. Easy Access
github
master
development
production
FDB-123 rpm-dev.spectos.com/FDB-123/current
rpm-dev.spectos.com/dev/current
rpm-stage.spectos.com
rpm.spectos.com
Feature Branch

23. Branch Builds (Frontend only)

24. Improvements
- Travis style builds (.travis.yml)
- Dynamic provisioning for build workers
- Containers (Docker, …)
- Frontend tests (Selenium, Robot)
- Test reports, coverage reports, Sonar
- Blue/Green deployment

25. Benefits
- Reduced risk: find problems early, no unpredictable
integration phase (“Blind Spot”)
- Helps to ensure code & functional quality
- Debugging: smaller changesets, bugs don’t pile up
- (Almost) no manual tasks, repeatable, less error-prone
- Precondition for Continuous Deployment => rapid
feedback cycles, more focused development
- Bottom line: less bugs, more features = happier
developers & happier customers

26. References/Further Reading
- Martin Fowler: http://www.
martinfowler.
com/articles/continuousIntegrati
on.html
- Duvall, Matyas, Glover:
“Continuous Integration:
Improving Software Quality and
Reducing Risk”

27. http://spectos.com
http://jhenning.me
Thank You!