Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2d5hFq5. Manny Pelarinos talks about the tension of optimizing APIs for different experiences, while supporting hundreds of endpoints and many web and mobile applications at scale. In the last few years ESPN has made many iterations on its API platform, eventually settling on developing a product centric architecture along with a new platform to build APIs. Filmed at qconnewyork.com. Manny Pelarinos is the Senior Director of APIs at ESPN. He is a highly experienced technology leader with 15 years of extensive experience building progressive systems within media, healthcare, financial, and retail sectors, and a skilled problem-solver with expertise in the complete software development life-cycle.

1. Next Generation APIs
Powering Web, Mobile, TV
Manny Pelarinos
Sr. Director of Distribution Platforms

2. InfoQ.com: News & Community Site
• 750,000 unique visitors/month
• Published in 4 languages (English, Chinese, Japanese and Brazilian
Portuguese)
• Post content from our QCon conferences
• News 15-20 / week
• Articles 3-4 / week
• Presentations (videos) 12-15 / week
• Interviews 2-3 / week
• Books 1 / month
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
espn-api

3. Presented at QCon New York
www.qconnewyork.com
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practitioner-driven conference designed for YOU: influencers of
change and innovation in your teams
- speakers and topics driving the evolution and innovation
- connecting and catalyzing the influencers and innovators
Highlights
- attended by more than 12,000 delegates since 2007
- held in 9 cities worldwide

4. 2
1. Back Story
2. Next Gen API Platform
3. Real time consumer messaging…
at extreme scale
4. Q&A
QCON 2016
Agenda

5. 3
• Manny Pelarinos - Sr Director of Distribution
Platforms
• Second time at QCon
• 9 year career with ESPN
• We’re hiring! - jobs.espncareers.com
About me

6. Backstory

7. 5
Headline:
• Bullet 1
• Bullet 2
• Bullet 3
How
• Personalization
• Globalization
• Product-first design

8. 6
ESPN Facts and Figures
• > 400 Million API calls per day
• Peaks of over 20K RPS
• 1 Million WS Updates to Fans in
< 100MS

9. 7
Product Galore So Many Screens

10. 8
• Network latency and multiple requests are
bad
• Data plan restrictions and bandwidth
and Mobile
The Good, The Bad

11. 9
v2v1v0
Focus on
Partners
Custom Feeds
REST
API Reuse
Product Agnostic
BIG APIs
Focus on
Partners
Best of both
worlds
API Evolution

12. 10

13. Next Generation API
Platform

14. 12
Say Hello to Binder
Core APIs Product APIs

15. 13
Baseball Football … Sports CMS Audio/Video Fan Pref.
Core APIs
Business logic
Service tier
Lightweight
Data
References
Product APIs
Composition
Trimming
Custom DSL
Efficient
Baseball
Core APIs
…Sports
Core APIs
CMS Core
APIs
Audio/
Video
Fan Core
APIs
Caching and Routing
Football
Core APIs
Caching and Routing
ESPN.com
Product APIs
App Product
APIs
TV Product
APIs
Partner
Product APIs
Products &
Screens
Architecture

16. 14

17. 15

18. 16

19. 17

20. 18
• Actual Fan preference data is small - IDs
• Content is big - News, Scores, Etc.
• Solution - Product API that conflates Fan
Preferences with sports data
Personalization

21. 19
ESPN Now
• Many disparate sources of
content - CMS, Blogs,
Twitter, Facebook, etc.
• Search API alone doesn’t
get you all the details
• Solution - Product API that
conflates search and all
our various content APIs

22. 20
• Caching
• Asynchronous
• DSL + Groovy
• Tools & Dashboards
Key Components

23. 21
Caching
Local Core API Cache
(Countdown TTL)
Product APIs
Core
APIs
Database
Distributed
Cache

24. 22
• Make requests in parallel –
Product API should only be
as slow as its slowest core
API call
• But, mixing synchronous
programming with
asynchronous is hard
• Enforce asynchronous all
the time but needed
convenience and rails
• Ported JS Promises to
Java (With RxJava)
Promises

25. 23
Groovy
• Powerful Dynamic
Language
• Java++
• Performance issues
mitigated by caching

26. 24
Tools and Dashboards - Trace

27. 25
HAR (HTTP Archive)

28. 26
Grafana and OpenTSDB

29. 27
Technology
RxJav

30. Real Time Messaging

31. 29
Massive Scale
• How do we enable real time data with REST
APIs?
• How can we scale to millions of concurrently
connected Fans?
Real-Time Data at Sports Scale

32. 30
• FastCast is a real-time high-scale multicast
publishing platform that uses web sockets
as its underlying technology, allowing it to
work in both web and mobile spaces
• Client sets up a “Topic” via Admin Service
• Client publishes to Topic via Admin Service
• Fans subscribe to Topic via FastCast Core
• Fans receive messages as they are
published
Say Hello to FastCast

33. 31
The Details
• Reliable core node replication handled thru Redis
• Built-in checkpoint/snapshot feature
• Uses generic JsonDiff/Patch + payload compression
• Client/server healthCheck + Better analytics captures
• AutoScaling in AWS with multi-region latency based routing

34. 32
FastCast Architecture
API Delta
Capture
FastCast
Admin
TopicsLog
Replication
Traffic
Manager
Product APIs

35. Summary

36. 34
What Did We Learn?
• One size does NOT fit all - Need
to separate Core from Product
• Caching is Key
• Operational Tools are awesome
• REST APIs + WebSockets =
COOL

37. 35
• Error handling is complicated…
• What to do when circuit breaker
trips and nothing is in cache
• 50x vs. NULL
Playbook for Errors

38. 36
• Binder as an in-memory application side
‘join engine’
• On going discussion around pre-expanding
more
NoSQL = Joins

40. Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
espn-api