O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

NoSQL meets Microservices - Michael Hackstein

457 visualizações

Publicada em

Just a few years ago all software systems were designed to be monoliths running on a single big and powerful machine. But nowadays most companies desire to scale out instead of scaling up, because it is much easier to buy or rent a large cluster of commodity hardware then to get a single machine that is powerful enough. In the database area scaling out is realized by utilizing a combination of polyglot persistence and sharding of data. On the application level scaling out is realized by microservices. In this talk I will briefly introduce the concepts and ideas of microservices and discuss their benefits and drawbacks. Afterwards I will focus on the point of intersection of a microservice based application talking to one or many NoSQL databases. We will try and find answers to these questions: Are the differences to a monolithic application? How to scale the whole system properly? What about polyglot persistence? Is there a data-centric way to split microservices?

Publicada em: Dados e análise
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

NoSQL meets Microservices - Michael Hackstein

  1. 1. www.arangodb.com NoSQL meets Microservices Michael Hackstein @mchacki
  2. 2. Michael Hackstein ‣ ArangoDB Core Team ‣ Web Frontend ‣ Graph visualisation ‣ Graph features ‣ Host of cologne.js ‣ Master’s Degree
 (spec. Databases and
 Information Systems) 2
  3. 3. Classical Setup 3
  4. 4. Scaling 4 LoadBalancer
  5. 5. Responsibilities 5 LoadBalancer Application Code Consistency Joins
  6. 6. app Monolith breakdown 6 server.route({ method: "GET", path: "/app", handler: function (req, reply) { var sum = cpuWaste(); reply(dataLookup["key" + sum]); } }); var cpuWaste = function() { var sum = Math.floor(Math.random() * 10000); for (var j = 0; j < 1000000; ++j) { sum += j; sum %= 10000; } return sum; }; var dataLookup = {}; for (var j = 0; j < 500000; ++j) { dataLookup["key" + j] = "This is test text number " + j + "."; }
  7. 7. memService cpuService app Monolith breakdown 6 server.route({ method: "GET", path: "/app", handler: function (req, reply) { var sum = cpuWaste(); reply(dataLookup["key" + sum]); } }); var cpuWaste = function() { var sum = Math.floor(Math.random() * 10000); for (var j = 0; j < 1000000; ++j) { sum += j; sum %= 10000; } return sum; }; var dataLookup = {}; for (var j = 0; j < 500000; ++j) { dataLookup["key" + j] = "This is test text number " + j + "."; }
  8. 8. memService cpuService app Monolith breakdown 6 server.route({ method: "GET", path: "/app", handler: function (req, reply) { var sum = cpuWaste(); reply(dataLookup["key" + sum]); } }); var cpuWaste = function() { var sum = Math.floor(Math.random() * 10000); for (var j = 0; j < 1000000; ++j) { sum += j; sum %= 10000; } return sum; }; var dataLookup = {}; for (var j = 0; j < 500000; ++j) { dataLookup["key" + j] = "This is test text number " + j + "."; } server.route({ method: "GET", path: "/mem/{id}", handler: function (request, reply) { reply(dataLookup["key" + request.params.id]); } });
  9. 9. memService cpuService app Monolith breakdown 6 server.route({ method: "GET", path: "/app", handler: function (req, reply) { var sum = cpuWaste(); reply(dataLookup["key" + sum]); } }); var cpuWaste = function() { var sum = Math.floor(Math.random() * 10000); for (var j = 0; j < 1000000; ++j) { sum += j; sum %= 10000; } return sum; }; var dataLookup = {}; for (var j = 0; j < 500000; ++j) { dataLookup["key" + j] = "This is test text number " + j + "."; } server.route({ method: "GET", path: "/mem/{id}", handler: function (request, reply) { reply(dataLookup["key" + request.params.id]); } }); server.route({ method: "GET", path: "/cpu", handler: function (request, reply) { reply(cpuWaste()); } });
  10. 10. memService cpuService app Monolith breakdown 6 var cpuWaste = function() { var sum = Math.floor(Math.random() * 10000); for (var j = 0; j < 1000000; ++j) { sum += j; sum %= 10000; } return sum; }; var dataLookup = {}; for (var j = 0; j < 500000; ++j) { dataLookup["key" + j] = "This is test text number " + j + "."; } server.route({ method: "GET", path: "/mem/{id}", handler: function (request, reply) { reply(dataLookup["key" + request.params.id]); } }); server.route({ method: "GET", path: "/cpu", handler: function (request, reply) { reply(cpuWaste()); } }); server.route({ method: "GET", path: "/app", handler: function (req, reply) { request('http://127.0.0.1:9000/cpu', function (error, response, body) { if (!error && response.statusCode == 200) { request('http://127.0.0.1:9000/mem/' + body, function (error, response, body) { if (!error && response.statusCode == 200) { reply(body); } else { reply({error: error}); } }); } else { reply({error: error}); } }); } });
  11. 11. Introduction to NoSQL 7 NoSQL World Documents - JSON Graphs Key Value { “type“: "pants", “waist": 32, “length”: 34, “color": "blue", “material”: “cotton" } { “type“: "television", “diagonal screen size": 46, “hdmi inputs": 3, “wall mountable": true, “built-in digital tuner": true, “dynamic contrast ratio”: “50,000:1”, Resolution”: “1920x1080” } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } { “type": "sweater", “color": "blue", “size": “M”, “material”: “wool”, “form”: “turtleneck" } ‣ Map value data to unique string keys (identifiers) ‣ Treat data as opaque (data has no schema) ‣ Can implement scaling and partitioning easily ‣ Focussed on m-to-n relations between entities ‣ Stores property graphs: entities and edges can have attributes ‣ Easily query paths of variable length ‣ Normally based on key-value stores (each document still has a unique key) ‣ Allow to save documents with logical similarity in “collections” ‣ Treat data records as attribute-structured documents (data is no more opaque) ‣ Often allow querying and indexing document attributes
  12. 12. Polyglot Modeling ‣ If you have structured data ➡ Use a document store ‣ If you have relations between entities and want to efficiently query them by arbitrary steps in between ➡ Use a graph database ‣ If you manage the data structure yourself and do not need complex queries ➡ Use a key-value store ‣ If you have structured data in graph format, or data model might change ➡ Use a multi-model database 8 Use the right tool for the job
  13. 13. Using different databases 9 LoadBalancer
  14. 14. Responsibilities 10 LoadBalancer Application Code Consistency Joins
  15. 15. Responsibilities 10 LoadBalancer Application Code Consistency Joins
  16. 16. Solution 1: Writes only to one master 11
  17. 17. Solution 2: Microservices 12
  18. 18. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  19. 19. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  20. 20. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  21. 21. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  22. 22. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  23. 23. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  24. 24. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload})
  25. 25. neo4jService mongoDBService productService Polyglot Setup 13 MATCH (me:Users {id: "' + id + '"}) -[:Friend]-> (:Users) -[:Bought]-> (p) return p.id as id products.find({$or: req.payload}) 6 network hops + 2 queries
  26. 26. Live Demo Neo4J + MongoDB 14 https://github.com/mchacki/ms-meets-nosql-examples
  27. 27. Multi Model Database ‣ Can natively store several kinds of data models: ‣ Key-value pairs ‣ Documents ‣ Graphs ‣ Delivers query mechanisms for all data models ‣ ACID transactions ‣ Cross collection joins 15
  28. 28. Using a Multi-Model Database 16 LoadBalancer Application Code Consistency Joins
  29. 29. Using a Multi-Model Database 16 LoadBalancer Application Code Consistency Joins
  30. 30. Foxx ‣ Customized REST API on top of ArangoDB ‣ Microservice framework ‣ Integrate with other microservices ‣ Reuse your Node.js code and NPM modules ‣ Built-in authentication using OAuth2.0 or HTTP-Basic Auth ‣ Operations are encapsulated in the database ‣ low network traffic, direct data access ‣ increases data privacy / (~( ) ) /_/ ( _-----_(@ @) ( / /|/--| V " " " " 17
  31. 31. foxx productService Foxx Setup 18 RETURN GRAPH_NEIGHBORS('ecom', @id, {direction: 'outbound', minDepth: 2, maxDepth: 2, neighborCollectionRestriction: 'Products', includeData: true }) 4 network hops + 1 query
  32. 32. foxx productService Foxx Setup 18 RETURN GRAPH_NEIGHBORS('ecom', @id, {direction: 'outbound', minDepth: 2, maxDepth: 2, neighborCollectionRestriction: 'Products', includeData: true }) 4 network hops + 1 query
  33. 33. foxx productService Foxx Setup 18 RETURN GRAPH_NEIGHBORS('ecom', @id, {direction: 'outbound', minDepth: 2, maxDepth: 2, neighborCollectionRestriction: 'Products', includeData: true }) 4 network hops + 1 query
  34. 34. foxx productService Foxx Setup 18 RETURN GRAPH_NEIGHBORS('ecom', @id, {direction: 'outbound', minDepth: 2, maxDepth: 2, neighborCollectionRestriction: 'Products', includeData: true }) 4 network hops + 1 query
  35. 35. foxx productService Foxx Setup 18 RETURN GRAPH_NEIGHBORS('ecom', @id, {direction: 'outbound', minDepth: 2, maxDepth: 2, neighborCollectionRestriction: 'Products', includeData: true }) 4 network hops + 1 query
  36. 36. Live Demo Foxx 19 https://github.com/mchacki/ms-meets-nosql-examples
  37. 37. ‣ open source and free (Apache 2 license) ‣ sharding & replication ‣ JavaScript throughout (V8 built into server) ‣ drivers for a wide range of languages ‣ web frontend ‣ good & complete documentation ‣ professional as well as community support 20 An overview of other features
  38. 38. Thank you ‣ Further questions? ‣ Follow me on twitter/github: @mchacki ‣ Write me a mail: mchacki@arangodb.com ‣ Join or google group: https://groups.google.com/forum/#!forum/arangodb ‣ Visit us at our booth ‣ Free T-shirts 21

×