Using git as a data store to create a cache for data which: * Is versioned * Supports common file operations (diff, log) * Is very fast Great for slow backend systems where the metadata is valuable by itself...

1. Using Git as a Database
Solving slow API access and other problems
Kirsten Hunter
@synedra
http://www.princesspolymath.com
store

2. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Introduction
Kirsten Hunter
Kirsten Hunter
@synedra
http://www.princesspolymath.com

3. Kirsten Hunter
@synedra
http://www.princesspolymath.com
What’s the trouble?
API Evangelist for Akamai
Powerful, write-based API for configuration
Very slow for reads
Amazing available data

4. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Crawling the Data
• Account Groups (Akamai DevRel)
-> 1 call
• Group Properties (Kirsten Hunter)
-> 1 call / group
(tens/hundreds)

5. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Still Crawling
• Properties (www.princesspolymath.com)
-> 1 call / property
(tens/hundreds/thousands)
• Versions (1)
-> 3 calls/version
(tens/hundreds/thousands)

6. Kirsten Hunter
@synedra
http://www.princesspolymath.com
API Goals
• Writing configuration changes
• Creating new properties/versions
• Updating metadata
• Internal data IDs, not user friendly
names

7. Kirsten Hunter
@synedra
http://www.princesspolymath.com
API Read Behavior
• Very, very restful
• No way to expand data
• Several seconds/call

8. Kirsten Hunter
@synedra
http://www.princesspolymath.com
What do Customers
Want?
• Diff between properties and versions
• Properties with particular billing codes
• Properties with particular behaviors
• Changes since date

9. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Why Not a Database?
• Need to have infrastructure
• Not designed for diff
• Complicated to query on multiple things ad
hoc
• Maintenance becomes an issue

10. Kirsten Hunter
@synedra
http://www.princesspolymath.com

11. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Why Git?
• Lightweight system ideal for storing text
• Portable data store
• Designed with tools for manipulating text files

12. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Structure
• Property name -> Branch
• Version -> Commit
• Rules file
• Hostnames file
• Metadata file

13. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Tags
• Property versions
• princesspolymath.com@2
• Easy to interface
• Also LATEST, STAGING, PRODUCTION

14. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Useful Git Commands
• git diff princesspolymath.com@2
princesspolymath.com@LATEST
• git branch -a | tr -d * | xargs git grep
'"<cpcode>'
• git diff $(git rev-list -n1 --after=06-01-2015
princesspolymath.com)

15. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Git Tools
• gitk - graphical diff and interface
• git desktop applications
• gitweb

16. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Process
• Run once to grab all the versions
-> Can be very long
• Run again to grab new versions
-> Quite quick
• Can schedule to run every 5 minutes

17. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Other Use Cases

18. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Family Tree
• master -> you
• branch -> surname
• commit -> individual
• sub-branch -> surname…

19. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Gollum
• git-backed wiki
• Runs git pages
• Simple to use and setup
• Again, text files… git is designed for them

20. Kirsten Hunter
@synedra
http://www.princesspolymath.com
Questions? Thoughts?
Ideas?http://www.princesspolymath.com
Kirsten Hunter
@synedra
http://www.princesspolymath.com