Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014
•
2 likes•1,477 views
Glynn Bird – Cloudant – Building applications for success. All too often, web applications are built to work in development but are not capable of scaling when success arrives. Whether the application is a log aggregator that can't deal with the throughput, a blog that can't handle traffic when it hits the heights of Google's rankings or a mobile game that goes viral, an application can become the victim of its own success. By building with Cloudant from the outset, and architecting the application to scale by design, we can build apps that scale as the traffic, data-volumes and users arrive. Using several real-life use cases, this talk will detail how Cloudant can solve an application's data storage, search and retrieval needs, scaling easily with success!
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014
Similar to Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014 (20)
More from NoSQLmatters
More from NoSQLmatters (20)
Recently uploaded
Recently uploaded (20)
Glynn Bird – Cloudant – Building applications for success.- NoSQL matters Barcelona 2014
- 1. © 2014 IBM Corporation Building Applications for Success Glynn Bird, Developer Advocate
- 2. Agenda • Application Development • Scaling • Real-world examples & how can Cloudant help?
- 3. Introduction Glynn Bird Developer Advocate Cloudant http://glynnbird.com @glynn_bird glynn@cloudant.com
- 4. Development - Go-to tools Basic 1980s 1990s 2000s 2010s Pascal C C++ Forth PHP4 Java ObjC PHP5 Node.js File System NoSQL SQL
- 5. Scalability? • Pick your tools of choice • PHP + MySQL, Ruby + Mongo • Build on your machine • Deploy! • Scalability is an ops problem? • Hint: “no”
- 6. Scaling tip #1 - Keep things separate • Separate • load balancing • app • database • cache • Separation of layers allows independent scaling Google’s first server rack http://bit.ly/10xGe2Y
- 7. Scaling tip #2 - Keep app servers stateless • Store all persistent data in a database • Don’t be tempted to store sessions on local disk • Stateless app servers can be created and destroyed at will IBM 305 RAMAC 5Mb Disk - 1957 http://bit.ly/10xGe2Y
- 8. Scaling tip #3 - queue-driven workflow • Store tasks in queues • Workers servers consume the queue • Queues allow workers to be scaled at will http://bit.ly/1Aycjse
- 9. Scaling tip #4 - Automate everything • No need to manually configure servers anymore • Workers servers consume the queue • It should be as easy to deploy N servers as 1 server http://bit.ly/1Aycjse
- 10. What does failure to scale look like? http://bit.ly/1Aycjse
- 12. What does failure to scale look like? • increased latency • downtime • high maintenance costs • data loss • disgruntled customers http://bit.ly/1Aycjse
- 13. Cloudant Highlights • Founded in 2009 by data scientists: Hoffman, Miller & Kocoloski • 20,000+ freemium and 225+ paying customers • Key collaboration & investment agreement with Samsung Electronics • HQ’d in Boston with offices/presence in Seattle, San Francisco, Bristol, UK & Seoul, SK • 34 global locations on 4 hosting providers • Acquired by IBM in February 2014 into the Information Management software group
- 14. Cloudant NoSQL DBaaS App Layer / Browser / Mobile App REST (HTTPS) Cloudant JSON API Incremental MapReduce Security Monitoring Geo-Load Balancing Indexed JSON Storage Physical Storage @ IBM SoftLayer, Rackspace, Microsoft Azure, AWS JSON CRUD Multi- Master Replication Device Sync Lucene Search 2D, 3D Geospatial IO Control Managed for you 24x7 by Cloudant developer-operators
- 15. How can Cloudant help you to scale? Cloudant empowers clients to get their applications to market faster in a cost-effective, hassle-free delivery model with guaranteed performance and support. • Scales massively & elastically • Handle millions of daily active users • Guaranteed performance & up time • Removes risk of project & SLA delivery failure • More agile development for web & mobile • No rigid schemas to slow development • Managed for you 24x7 • Stay focused on new development, no DB administration
- 16. Example 1 - IoT Data Logging • Temperature sensor £1.50 • Raspberry Pi £25 • Network attached thermometer DS18B20
- 17. DS18B20 Software • Bash script • Where should we put the data ? sensorid=`find /sys/bus/w1/devices/* -type l | grep -o '[0-9]*-.*$' | head -n 1` filename="/sys/bus/w1/devices/$sensorid/w1_slave" T=`tail -n 1 "$filename" | grep -o "[0-9]*$"` t=${T:0:2}.${T:2:3} unix=`date "+%s"` d=`date "+%Y-%m-%d %H:%M:%S %Z"` json="{ "ts": $unix, "date": "$d", "temperature": $t, "sensor_id": "$sensorid" }"
- 18. DS18B20 Storage • Local CouchDB • Installed with one line sensorid=`find /sys/bus/w1/devices/* -type l | grep -o '[0-9]*-.*$' | head -n 1` filename="/sys/bus/w1/devices/$sensorid/w1_slave" T=`tail -n 1 "$filename" | grep -o "[0-9]*$"` t=${T:0:2}.${T:2:3} unix=`date "+%s"` d=`date "+%Y-%m-%d %H:%M:%S %Z"` json="{ "ts": $unix, "date": "$d", "temperature": $t, "sensor_id": "$sensorid" }" curl -X POST -H "Content-Type: application/json" -d "$json" "http://localhost: 5984/logger/"
- 19. Where does Cloudant fit in? • Replicate to Cloudant • Scale by adding more sensors • Data collection can be offline • Sync when connected
- 21. Why is Cloudant a good fit? • Replication • collect data at the edge, replicate when online • Cloudant can store data at massive scale with high availability • Time-series data is a great fit for Cloudant
- 22. Time-series data in Cloudant • MapReduce indexes on date field • Allows range querying with grouping e.g. find min/max/avg between two dates, grouped by year/month/day/hour etc. { "_id": "00339bc7", "_rev": "1-0ef01b53", "ts": 1411723490, "date": "2014-09-26 09:24:50 UTC", "temperature": 20.937, "sensor_id": "28-000006746cde" } [2014,9,26,9,24,50] ---> 20.937
- 23. Example #2 - Mobile applications • Mobile game • Stores games state, usage stats, achievements etc. locally • Needs to be cross-platform
- 24. Option 1 - PouchDB + HTML5 • In-browser, cross-platform datastore • CouchDB compatible • Syncs with Cloudant http://pouchdb.com/
- 25. Option 2 - CloudantSync • iOS and Android native libraries • Local storage + query • Syncs to and from Cloudant https://cloudant.com/product/cloudant-features/sync/
- 26. One database per user
- 27. One database per user • Each user’s data has it’s own access control • Each database is relatively small; small enough to fit on phone • Scales massively; hundreds of thousands of concurrent users https://cloudant.com/product/cloudant-features/sync/
- 28. Demo • Business Card collection app • Native Android App • Companion Web App https://cloudant.com/product/cloudant-features/sync/
- 29. How did that work? Android Phone App pouchdb IBM BlueMix Web App
- 30. What about the code? var db = new PouchDB("myfirstdatabase"); db.post( { a:1, b: "2", c: true } ); db.sync(URL, { live: true }); https://cloudant.com/product/cloudant-features/sync/
- 31. Why is Cloudant a good fit? • Replication is a hard problem. PouchDB + Cloudant solve it • One database per-user is a great way to scale mobile apps
- 32. Conclusion • Building with Cloudant as your data layer has become a “go to” development solution • Building with Cloudant builds in scale from day one • large data sets • lots of smaller data sets • Cloudant syncs! https://cloudant.com/product/cloudant-features/sync/