1. 500Startups

2. Giving a talk about ou r APIs and our
sc ale to the visitin g 500Startups
class.
28 Mar via Twitter for iPhone
from Twitter
795 Folsom Street
San Francisco, CA
View Tweets at this place

3. >750K client applications +
Officially Owned and Operated

4. Twitter is just three things:
Tweets,relationships between those three
And the users, anD Timelines

5. What is the TwitterAPI?
REST API
- provides basic Twitter functionality
- Read / Write (Tweet, Follow, DM, etc.)
Search API
- real-time search index
Streaming API
- HTTP long poll connection
- Tweets in real-time

6. Using the system
10B API calls 115,000 calls
per day ≈ per second

7. Three main objects
Status Objects - the Tweet (text, author, and metadata)
User objects - username, screen name, avatar
Timelines - orderings of Tweets

8. Quick note on timelines
user - a the tweets a given user has authored
home - the main timeline you would see in a client
Mentions - all the tweets that @mention a user

9. Causing change
Tweeting - a POST to status/update
Following - a POST to friendships/create
DM-ing - a POST to direct_messages/new

10. Authentication
OAuth 1.0a
- signature based requests
- user driven access to the API
- client differentiated access to the API
Applications don’t have passwords
- applications store tokens for their users
- users can change passwords, but tokens still work
OAuth 2.0
- coming soon!

11. Limits
350 OAuth calls / user / hour / IP
- authenticated calls goes against calling user
- unauthenticated calls goes against calling IP
“Natural” limits
- limits on number of Tweets / DMs sent
- limits on number of follows / unfollows a day
Status Limits
- can’t send “duplicate” tweets

12. Streaming API
Persistent connections
- get pushed a tweet, in real-time, that matches your predicate
- “push” version of search
- read-only
User streams / Site streams
- re-create the client experience using streams
- great for “client” experiences

13. Latency
200ms
100ms
0ms

14. Streaming API’s Track and follow
Track
- watch a particular keyword
- up to 200 can be sent
Follow
- get all the tweets (RTs, etc.) from a particular user
- up to 400 can be issued

15. Tools of the Trade
dev.twitter.com
- documentation central
twurl
- OAuth enabled version of curl
- allows you to manually test authenticated and unauthenticated REST
Twitter for Mac
- built in “developer console”

16. Is that it? ... YES! ... sorta ...

17. things get more complicated at scale
And scaling “real-time” is hard

18. How many Tweets are there?

19. How many Tweets are there? 140M!
off the chart

20. 140M tweets 1600 tweets
per day ≈ per second

25. Tweets multiply
8,080 people
need to see my
tweet

26. @ladygaga
mother mons†er
8.9 million followers
@justinbieber
Justin Bieber
8.3 million followers
@BarackObama
44th President of the United States
7.1 million followers
@raffi
me!
8.5 thousand followers

27. @ladygaga
mother mons†er
8.9 million followers
@raffi
me!
8.5 thousand followers
30,000 tweets
1 tweet from
≈ 10
scaling for delivered
1000x more followers
@ladygaga
6than me
per second

28. A System is only as strong as its
weakest link

29. 200M registered users
2006 2008 2010 2011

30. Lots of deliveries

31. MySQL
Can’t generate IDs fast enough
Centralized and a single point of failure
snowflake
Highly available and uncoordinated (10kqps)
Compatible with the ecosystem
http://github.com/twitter/snowflake

32. Photo used under Creative Commons from jurvetson
Distributed graph database
flockdb High rate of CRUD operations
Complex set arithmetic queries
http://github.com/twitter/flockdb

33. Follow me at
Questions? twitter.com/raffi