1. Realtime at Lore
Using Node.js and Socket.io
Dan Getelman — May 9, 2012
Tuesday, May 15, 12

2. Why am I here?
• CTO/Co-founder at Lore (formerly
Coursekit)
• We built a system where real-time
updates are global. It’s been happily
running in production for about a year.
Tuesday, May 15, 12

3. Outline
• What we have
• How we ended up there
• How it works
• Lessons learned
• Miscellaneous Thoughts
Tuesday, May 15, 12

4. What’s our stack?
• Rich front-end framework
• node.js + socket.io for real-time
communications
• Python API
Tuesday, May 15, 12

5. How did we end up
here?
• Build best experience possible
• Strong vision for what we wanted —
solving general case way less work
than tons of specifics
• Able to leverage infrastructure both for
speed gains and real-time updating
Tuesday, May 15, 12

6. How did we end up
here?
• Sat down last summer, built it
• Pleasantly surprised by ease of
socket.io + Redis
Tuesday, May 15, 12

7. Goals
• Build awesome experience for
teachers/students
• Performance
• Secure
Tuesday, May 15, 12

8. Data Model
Restrictions
• Permission Groups let us simplify
updates significantly
• Individual models being pushed to front-
end
• Originally, circular dependencies not
allowed
Tuesday, May 15, 12

9. Two Browser
Connections
API Push
HTTPS
WebSocket
/ Fallback
Tuesday, May 15, 12

10. Beauty of Push
Server
• Fills two roles:
• Lets us bypass browser concurrent
requests limits
• Lets us push any changes relevant to
a user down the pipe
• Socket.io generally lets us do this
without worrying about browsers
Tuesday, May 15, 12

11. Redis — Wonderchild
• Used in three very different ways,
generally happy with all
• Message queue
• Cache
• Pubsub
Tuesday, May 15, 12

12. How It Fits Together
• Subscribe to models (message via
socket.io)
• On write, determine which models care
about the change / addition
• Publish to their channel in Redis
Tuesday, May 15, 12

13. Front-end
Touchpoints
• Receiving and dispatching messages
(chat, notifications, updates)
• Fetching models
• Batch calls to improve performance
Tuesday, May 15, 12

14. Running in
production
• Handling HTTPS
• Load balancing
• Monit
• Metrics
Tuesday, May 15, 12

15. HTTPS
https = require 'https'
privateKey = fs.readFileSync("key.key").toString()
certificate = fs.readFileSync("ssl.pem").toString()
ca1 = fs.readFileSync("blah.pem").toString()
https.createServer {key:privateKey,cert:certificate, ca:
[ca1]}
Tuesday, May 15, 12

16. Lessons Learned
• Log everything
• Measure everything
• Unstable standards are unstable
• socket.io can be a bit opaque if things
aren’t going well
Tuesday, May 15, 12

17. Thanks!
• Follow me on Twitter: @dget
• We’re hiring!
Tuesday, May 15, 12