O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Scaling the Netflix API - From Atlassian Dev Den

8.965 visualizações

Publicada em

The term "scale" for engineering often is used to discuss systems and their ability to grow with the needs of its users. This is clearly an important aspect of scaling, but there are many other areas in which an engineering organization needs to scale to be successful in the long term. This presentation discusses some of those other areas and details how Netflix (and specifically the API team) addresses them.

Publicada em: Tecnologia, Diversão e humor
  • Seja o primeiro a comentar

Scaling the Netflix API - From Atlassian Dev Den

  1. Scaling the Netflix API Daniel Jacobson @daniel_jacobson http://www.linkedin.com/in/danieljacobson http://www.slideshare.net/danieljacobson
  2. Please read the notes associated with each slide for the full context of the presentation
  3. What do I mean by “scale”?
  4. But There Are Many Ways to Scale! Organization Systems Devices Development Testing
  5. But first, some background…
  6. Global Streaming Video for TV Shows and Movies
  7. More than 36 Million Subscribers More than 40 Countries
  8. Netflix Accounts for 33% of Peak Internet Traffic in North America Netflix subscribers are watching more than 1 billion hours a month
  9. 2007
  10. Netflix REST API: One-Size-Fits-All (OSFA) Solution
  11. Image courtesy of Jay Mac 3 on Flickr
  12. Netflix API Requests by Audience At Launch In 2008 External Developers
  13. Image courtesy of Jay Mac 3 on Flickr
  14. Netflix API Requests by Audience From 2011 External Developers
  15. Global Streaming Product Three aspects of the Streaming Product: • Discovery • Sign-Up • Streaming
  16. Member Sign-Up
  17. Discovery
  18. Discovery
  19. Today, Netflix API Supports Discovery and Sign-Up
  20. But Soon, Will Support Streaming
  21. Scaling… Organization Systems Devices Development Testing
  22. Distributed Architecture
  23. 1000+ Device Types
  24. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies Reviews A/B Test Engine Dozens of Dependencies
  25. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  26. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  27. http://www.slideshare.net/reed2001/culture-1798664
  28. Scaling… Organization Systems Devices Development Testing
  29. System Resiliency
  30. Distributed Architecture
  31. Dependency Relationships
  32. 2,000,000,000 Requests Per Day to the Netflix API
  33. 30 Distinct Dependent Services for the Netflix API
  34. 14,000,000,000 Netflix API Calls Per Day to those Dependent Services
  35. 0 Dependent Services with 100% SLA
  36. 99.99% = 99.7%30 0.3% of 2B = 6M failures per day 2+ Hours of Downtime Per Month
  37. 99.99% = 99.7%30 0.3% of 2B = 6M failures per day 2+ Hours of Downtime Per Month
  38. 99.9% = 97%30 3% of 2B = 60M failures per day 20+ Hours of Downtime Per Month
  39. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  40. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  41. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  42. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  43. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  44. Circuit Breaker Dashboard
  45. Call Volume and Health / Last 10 Seconds
  46. Call Volume / Last 2 Minutes
  47. Successful Requests
  48. Successful, But Slower Than Expected
  49. Short-Circuited Requests, Delivering Fallbacks
  50. Timeouts, Delivering Fallbacks
  51. Thread Pool & Task Queue Full, Delivering Fallbacks
  52. Exceptions, Delivering Fallbacks
  53. Error Rate # + # + # + # / (# + # + # + # + #) = Error Rate
  54. Status of Fallback Circuit
  55. Requests per Second, Over Last 10 Seconds
  56. SLA Information
  57. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  58. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  59. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine
  60. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine Fallback
  61. Personaliz ation Engine User Info Movie Metadata Movie Ratings Similar Movies API Reviews A/B Test Engine Fallback
  62. System Infrastructure
  63. AWS Cloud
  64. Autoscaling
  65. Autoscaling
  66. Forced Failure
  67. Global System
  68. More than 36 Million Subscribers More than 40 Countries
  69. Zuul Gatekeeper for the Netflix Streaming Application
  70. Zuul • Multi-Region Resiliency • Insights • Stress Testing • Canary Testing • Dynamic Routing • Load Shedding • Security • Static Response Handling • Authentication
  71. Isthmus
  72. Scaling… Organization Systems Devices Development Testing
  73. Screen Real Estate
  74. Controller
  75. Technical Capabilities
  76. One-Size-Fits-All API Request Request Request
  77. Scaling… Organization Systems Devices Development Testing
  78. Courtesy of South Florida Classical Review
  79. Resource-Based API vs. Experience-Based API
  80. Resource-Based Requests • /users/<id>/ratings/title • /users/<id>/queues • /users/<id>/queues/instant • /users/<id>/recommendations • /catalog/titles/movie • /catalog/titles/series • /catalog/people
  81. REST API RECOMME NDATIONS MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS Network Border Network Border
  82. RECOMME NDATIONS MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS OSFA API Network Border Network Border SERVER CODE CLIENT CODE
  83. RECOMME NDATIONS MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS OSFA API Network Border Network Border DATA GATHERING, FORMATTING, AND DELIVERY USER INTERFACE RENDERING
  84. Experience-Based Requests • /ps3/homescreen
  85. JAVA API Network Border Network Border RECOMME NDATIONS MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS Groovy Layer
  86. RECOMME NDATIONSA ZXSXX C CCC MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS JAVA API SERVER CODE CLIENT CODE CLIENT ADAPTER CODE (WRITTEN BY CLIENT TEAMS, DYNAMICALLY UPLOADED TO SERVER) Network Border Network Border
  87. RECOMME NDATIONSA ZXSXX C CCC MOVIE DATA SIMILAR MOVIES AUTH MEMBER DATA A/B TESTS START- UP RATINGS JAVA API DATA GATHERING DATA FORMATTING AND DELIVERY USER INTERFACE RENDERING Network Border Network Border
  88. Scaling… Organization Systems Devices Development Testing
  89. Dependency Relationships
  90. Testing Philosophy: Act Fast, React Fast
  91. That Doesn’t Mean We Don’t Test • Unit tests • Functional tests • Regression scripts • Continuous integration • Capacity planning • Load / Performance tests
  92. Cloud-Based Deployment Techniques
  93. Current Code In Production API Requests from the Internet
  94. Single Canary Instance To Test New Code with Production Traffic (around 1% or less of traffic) Current Code In Production API Requests from the Internet Error!
  95. Current Code In Production API Requests from the Internet
  96. Current Code In Production API Requests from the Internet Perfect!
  97. Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  98. Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  99. Error! Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  100. Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  101. Current Code In Production API Requests from the Internet Perfect!
  102. Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  103. Current Code In Production API Requests from the Internet New Code Getting Prepared for Production
  104. API Requests from the Internet New Code Getting Prepared for Production
  105. https://www.github.com/Netflix
  106. Scaling the Netflix API Daniel Jacobson @daniel_jacobson http://www.linkedin.com/in/danieljacobson http://www.slideshare.net/danieljacobson

×