SmartNews's journey into microservices Livesnse TechNight http://made.livesense.co.jp/blogs/431 Takumi Sakamoto @ SmartNews

1. SmartNews's
journey into microservices
Takumi Sakamoto
Livesense TechNight
2015.09.29

2. Takumi Sakamoto
@takus

4. News Delivery Pipelines

5. 36 months ago
The first system is developed
Crawler
Analyzer
Indexer
On memory DB
API
Monolithic App
Social
Media
Article
Lists

6. 12 months ago
Split crawlers
Crawler
Queue
Monolithic App
Analyzer
Indexer
On memory DB
API

7. https://medium.com/@takus/velocity-conference-2015-wrap-up-velocityconf-f3811e05f1e8
4 months ago
At Velocity Conference

8. 2 months ago
Dive into the microservices world
Crawler
Kinesis
IndexerAnalyzer
Kinesis CloudSearch

9. Why microservices?
• A lot of different types of services are required for us
• crawler, image analyzer, content extractor ...
• Adapt to technical progress / market change
• every system should be replaceable
• Make developers happy :)
• avoid to maintain "legacy" "large" "monolithic"
• enable us to develop any languages

10. A part of microservices
Source A
Analyzer Indexer Search API
A
PhotoqualCaffe
Reporting
API
Gateway
Realtime
feedback
Image
Analysis
Extractor
Content
extraction
Source B
Search API
B
Ranking
algorithm
A/B test
Language
detecter
Crawling
Language
detection
Spark
Streaming
KPI
Dashboard
Admin
Operation

11. References (Japanese only)
Stream 処理と Offline 処理の統合
http://bit.ly/1VnQQao
スマートニュースの世界進出を支えるログ解析基盤
http://bit.ly/1MCOyBX
SmartNews の Webmining を支えるプラットフォーム
http://bit.ly/1RcVGqk

12. Journey into microservices
When you use microservices you have to work on
automated deployment,
monitoring,
dealing with failure,
eventual consistency,
and other factors that a distributed system introduces.
http://martinfowler.com/bliki/MicroservicePremium.html

13. Automated Deployment
with AWS CodeDeploy
Today's topic

14. Deployment is not easy
• Manage complexity
• multiple roles
• multiple environments
• auto scaling
• Avoid downtime
• JVM based applications doesn't support hot deploy
• Monitoring
• stop deployment if something wrong
• which instances are succeeded or failed

15. AWS CodeDeploy
http://www.slideshare.net/AmazonWebServices/aws-codedeploy-manage-deployment-complexity

16. 2W1H configurations
http://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-steps.html
Where?How?What?

17. Deployment Flow
http://docs.aws.amazon.com/codedeploy/latest/userguide/deployment-steps.html
What?
How?
Where?

18. What to Deploy?
• Prepare your "revision" archive
• appspec.yml
• application (WAR/JAR etc...)
• (optional) scripts
• register to ELB / deregister from ELB
• pre-compile assets
• waiting cache warmup

19. App Spec

20. How to Deploy?

21. Where to Deploy?
EC2 Tag / Auto Scaling Group

22. My favorites
• Simple, Fast, Flexible, Free
• Require small changes for current system
• Integration w/ AWS Auto Scaling
• deploy last successful revision to a new instance
• wait for registering ELB until finishing deployment

23. Pitfall
Exclusive lock is required :P
Deployment is skipped during scale-out

24. Automated Deployment
at SmartNews

25. Deployment pipelines

26. Slack notification
Role / Env / Version
Deployment status
Commits

27. Datadog monitoring

28. Datadog alerts

29. Manage alerts by DSL
Most of alerts are created automatically
Powered by airbnb/interferon

30. Docker for C++ apps

31. Towards future...

32. We’re hiring!!!
Join our journey into microservices
Site reliability engineer
Productivity engineer
Web application engineer
iOS/Android engineer
ML/NLP engineer
Ad engineer
http://about.smartnews.com/en/careers/

33. Wait for you @ SmartKitchen