O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Building a Microservices-based ERP System

Speaker: Jerry Reghunadh, Architect, CAPIOT Software Pvt. Ltd.
Level: 200 (Intermediate)
Track: Microservices
One of the leading assisted e-commerce players in India approached CAPIOT to rebuild their ERP system from the ground up. Their existing PHP-MySQL setup, while rich in functionality and having served them well for under half a decade, would not scale to meet future demands due to the exponential grown they were experiencing.

We built the entire system using a microservices architecture. To develop APIs we used Node.js, Express, Swagger and Mongoose, and MongoDB was used as the active data store. During the development phase, we solved several problems ranging from cross-service calls, data consistency, service discovery, and security.

One of the issues that we faced is how to effectively design and make cross-service calls. Should we implement a cross-service call for every document that we require or should we duplicate and distribute the data, reducing cross-service calls? We found a balance between these two and engineered a solution that gave us good performance.

In addition, our current system has 36 independent services. We enabled services to auto-discover and make secure calls.

We used Swagger to define our APIs first and enforce request and response validations and Mongoose as our ODM for schema validation. We also heavily depend on pre-save hooks to validate data and post-save hooks to trigger changes in other systems. This API-driven approach vastly enabled our frontend and backend teams to scrum together on a single API spec without worrying about the repercussions of changing API schemas.

What You Will Learn:
- How we used Swagger and Mongoose to off-load validations and schema enforcements. We used Swagger to define our APIs first and enforce request and response validations and Mongoose as our ODM for schema validation. We also heavily depend on pre-save hooks to validate data and post-save hooks to trigger changes in other systems. This API-driven approach vastly enabled our frontend and backend teams to scrum together on a single API spec without worrying about the repercussions of changing API schemas.
- How microservices and cross-service calls work. One of the issues that we faced is how to effectively design and make cross-service calls. Should we implement a cross-service call for every document that we require or should we duplicate and distribute the data, reducing cross-service calls? We found a balance between these two and engineered a solution that gave us good performance.
- How we implemented microservice auto discovery: Our current system has 36 independent services, so we enabled services to auto-discover and make secure calls.

  • Entre para ver os comentários

Building a Microservices-based ERP System

  1. 1. Building a Micro-services based ERP system Using Node.js and MongoDB Jerry M. Reghunadh Senior Architect http://capiot.com
  2. 2. { “title” :“Topics” } • Building our services - Decision process, Modeling, Referential Integrity • Enabling service auto-discovery • Validating request, response and data • Performance and Throughput • Our learnings – What we did right and what we did not.
  3. 3. {“short_intro”: “ERP”} Core-Masters Category Brand Product Deal Inventory and Warehouse Warehouses Racks BinsStock Adjustments Finance and Commissions AccountsPayments ReceiptsCommissions Third-party integrations Wallet Recharge Money Transfer Bill pay Billing and Order Management Orders Promotions User Management OTPAccess control Logistics and Returns RoutingReturns Servicing
  4. 4. APIs are for the greater good! “APIs are so overrated” President Frank Underwood #FU2016
  5. 5. {“apis”:“very chatty!”} Deal Brand Category Product 1 2 3 4 1 2 3 4
  6. 6. { “topic”:“Engineering Decisions”}
  7. 7. Low resource footprint
  8. 8. Low resource footprint Elasticity • pm2 • Docker
  9. 9. Low resource footprint Elasticity • pm2 • Docker Monolingual .js & json
  10. 10. Documents JSON
  11. 11. Documents JSON Isn’t my data relational? Why not SQL?
  12. 12. Documents JSON Flexible-Schema
  13. 13. Documents JSON Flexible-Schema Clustering + Sharding
  14. 14. { “topic”:“Architecture”} Cache Micro-services Logger Big data store Gateway
  15. 15. { “topic”:“Architecture”} Redis Node.js + Mongo Filebeat Elasticsearch Node.js Portal (Angular) API Gateway
  16. 16. { “topic”:“APIs”} Service 1 Service 2 Service N ... Node.js Express.js Swagger.io Mongoose.js Passport.js
  17. 17. { “api.framework”: “Swagger.io”} Refer Open API Specification (OAS) https://github.com/OAI/OpenAPI-Specification
  18. 18. { “odm”:“Mongoosejs”} mongoosejs.com Application layer • Enforces schema • Pre- and Post- hooks • Promises
  19. 19. {“library”: “Swagger-mongoose-crud} https://github.com/capiotsoftware/swagger-mongoose-crud Create Update Delete List Get Bulk Update (and more)
  20. 20. MongoDB features as APIs Create Update Delete List Get GET http://localhost:8080/api/category/v1 PUT http://localhost:8080/api/category/v1/C2 GET http://localhost:8080/api/category/v1/C2 DELETE http://localhost:8080/api/category/v1/C2 POST http://localhost:8080/api/category/v1 db.categories .find({name:{$regex: /as/}},{id:1, name:1}) .limit(10) .skip(0) .sort({name:-1})
  21. 21. {“question”: “How to auto-discover services?”} Gateway Alice Bob
  22. 22. {“question”: “How to auto-discover services?”} Gateway Bob? Alice Bob I want Alice Alice?
  23. 23. {“solution”: “IP-based config”} Gateway Alice Bob I want Alice Problem: Non-scalable!
  24. 24. {“solution”: “Look-up service”} Gateway Alice Bob I want Alice • Dynamic lookup solution • Lightweight service • Fast fetch Cache https://github.com/capiotsoftware/puttu-redis
  25. 25. {“issue”: “Security”} ? • Censorship • Privileged access
  26. 26. {“issue”: “Security”} ✓ • Censorship • Privileged access HTTPs RBAC: Role Based Access Control
  27. 27. {“rbac.define”: “How to set permissions”} Service name Attributes Roles
  28. 28. ERP Revisited APIs Adding features Security Scaling and Performance Logging and tracingCache Micro-services Logger Big data store Gateways
  29. 29. {“role”: “MongoDB”} • Graphs $graphLookup • Finding parent child relationships • Finding optimal logistics route from route definitions, limited by hop count
  30. 30. {“role”: “MongoDB”} • Geospatial queries • Finding nearby hops to create routes. • Offloaded most operations on the DB. • Counters using $inc() • Aggregate • Graph lookup • Auto expiring documents.
  31. 31. {“topic”:“Throughput and performance”}
  32. 32. {“topic”:“Throughput and performance”} What we achieved: 330 tps o 4Core 28GB o 42 services on one machine …
  33. 33. {“topic”:“Throughput and performance”} What we achieved: 330 tps o 4Core 28GB o 42 services on one machine … Isn’t that low? Not really! Existing system: 150 tps Projected: 300 tps in 2 years
  34. 34. {“topic”:“Throughput and performance”} What we achieved: 330 tps o 4Core 28GB o 42 services on one machine … Isn’t that low? Not really! Existing system: 150 tps Projected: 300 tps in 2 years Consider other matrices § Talent acquisition § Feature turn around time § Agility of platform
  35. 35. {“question”: “What if I’m looking for more?”} Might not be the right tool
  36. 36. {“question”: “What if I’m looking for more?”} Might not be the right tool Alternatives
  37. 37. {“question”: “What if I’m looking for more?”} Might not be the right tool Alternatives Request: GET / Response: Test response 9k tps 15k tps
  38. 38. {“question”: “What if I’m looking for more?”} Might not be the right tool Alternatives Request: GET / Response: Test response 9k tps 15k tps What about DB?
  39. 39. {“question”: “What if I’m looking for more?”} Might not be the right tool Alternatives Request: GET / Response: Test response 9k tps 15k tps What about DB? + 5000 writes per secondHFT Solution
  40. 40. {“reflections”: “What we did right”} ü Consistent with separation of collections ü Know your DB ü Create re-usable libraries/frameworks ü Handle errors early
  41. 41. {“reflections”: “If time travel was possible…”} o Understand: Write or Read intensive o HTTP calls are expensive. o Think of transactional data
  42. 42. Thankyou

×