O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

I Love APIs 2015 : Zero to Thousands TPS Private Cloud Operations Workshop

I Love APIs 2015 workshop

I Love APIs 2015 : Zero to Thousands TPS Private Cloud Operations Workshop

  1. 1. Zero to Thousands TPS: ! Private Cloud Operations Workshop! Maudrit Martinez Apigee Paul Mibus Apigee
  2. 2. Agenda 9:30 – 10:00 Apigee Overview 10:00 – 11:00 Apigee Private Cloud Architecture BREAK (15 MINS) 11:15 – 12:00 Apigee Private Cloud Architecture (cont) BREAK – LUNCH (60 MINS) 13:00 – 15:45 Apigee Edge Install BREAK (15 MINS) 16:00 – 17:30 Platform Operations ©2015 Apigee. All Rights Reserved.
  3. 3. Apigee Edge Overview
  4. 4. Intelligent API platform enables the digital value chain INTELLIGENT API PLATFORM Intelligent API Platform User Connected Experience Developer API API Team Backend ©2015 Apigee. All Rights Reserved.
  5. 5. Apigee Edge Design" Design first. Document Smart. Full support for Swagger 2.0 Develop Configuration: Over 30 ready-to-use and " configurable policies Code: Built-in support for Node, JavaScript and Java extensibility BaaS Secure End-to-end security Threat protection Access control Simple OAuth implementation for your APIs PCI and HIPAA compliance Publish Turnkey developer portal Monitor
 Centralized control, decentralized development Multi-tenant architecture Billions of API calls, including large spikes Analyze Complete visibility– from app end to backend Automatically and continuously collect " all API-traffic data out of the box Monetize Flexible rate plans Internationalization support Usage tracking Limits and notifications Scale Self-service State @ scale Flexible deployment ©2015 Apigee. All Rights Reserved.
  6. 6. Apigee Edge – API management API Services API Team Developer Services Monitor, manage, and measure success Analytics Services Increase adoption and value of APIs Build, manage, scale, and secure APIs and Apps API" Gateway OAuth & Security Policies & Programmability Versioning & Governance API Services Developer" Portal Smart" Docs API" BaaS Monetization Developer Services Ops & Business Metrics Developer" Analytics App" Performance Custom" Reports Analytics Services Flexible Deployment Self-Service State @ Scale Connected Experience Business User Developer ©2015 Apigee. All Rights Reserved.
  7. 7. Config - Over 30 ready-to-use and configurable policies Code - Built-in support for Node, JavaScript, Java and Python extensibility Apigee Edge policies – Build APIs faster Manage interactions with API consumers and optimize performance Secure APIs and protect back-end systems from attack Transform, translate and reformat data for easy consumption Extend with programming when you need it ©2015 Apigee. All Rights Reserved.
  8. 8. Public Cloud = Private Cloud Flexible deployment options Apigee Cloud Multi-Datacenter Deployment 1+ Billion " calls / month 7 regions >99.9%" API availability 19 availability zones 3,000+" servers ~1/2 billion" transactions / day ~20 ms" average latency Private Cloud" leading Telco ©2015 Apigee. All Rights Reserved.
  9. 9. Agenda 09:30 – 10:00 Apigee Overview 10:00 – 11:00 Apigee Private Cloud Architecture BREAK (15 MINS) 11:15 – 12:00 Apigee Private Cloud Architecture (cont) BREAK – LUNCH (60 MINS) 13:00 – 15:45 Apigee Edge Install BREAK (15 MINS) 16:00 – 17:30 Platform Operations ©2015 Apigee. All Rights Reserved.
  10. 10. Apigee Private Cloud Architecture
  11. 11. Apigee Edge is comprised of several stateless components that use infrastructure services to persist data •  Gateway: Routing and processing API calls •  Apigee UIs: Enterprise UI, Developer Portal •  Infrastructure Services: Persistence and queuing of run time data •  Management Server: Provider of REST APIs for all configuration tasks Note: Monetization is part of Developer Services and leverages Gateway, Analytics Services and Management Server Apigee Edge architecture – High level ANALYTICS SERVICES API SERVICES BaaS" Backend DEVELOPER SERVICES Analytics Backend Management Server Gateway Apigee " UIs Infrastructure Services Developer Portal Client Backend Apigee Open Source Legend: ©2015 Apigee. All Rights Reserved.
  12. 12. Apigee Edge architecture – High level Load Balancer Infrastructure Services BaaS Backend Developer Portal Analytics Backend Management Server Gateway Infrastructure Services Apigee " UIs Gateway Infrastructure Services Infrastructure Services BaaS Backend Developer Portal Analytics Backend Management Server Gateway Infrastructure Services Apigee " UIs Gateway Infrastructure Services DATAREPLICATION DC-1 DC-2 Load Balancer Client Backends Apigee Open Source Legend: ©2015 Apigee. All Rights Reserved.
  13. 13. Apigee Edge architecture – Component view Apigee Open Source Legend: •  Router handles all incoming API traffic and dispatches it. The Router terminates the HTTP request, handles the SSL traffic, and uses the virtual host name, port, and URI to steer requests to the appropriate node. •  Message Processor handles API traffic for a specific organization and environment and which executes all policies. •  Apache Cassandra stores application configurations, distributed quota counters, API keys, and OAuth tokens for applications running on the gateway. •  Apache ZooKeeper contains configuration data about all the services of the zone and which notifies the different servers of configuration changes. •  OpenLDAP contains organization user and roles. •  Management Server offers an API that is used by the Central Services server to communicate with the servers in each on- premises installation. •  QPID Server manages queuing system for analytics data. •  Postgres Server manages analytics database. API SERVICES DEVELOPER SERVICES Router Enterprise " UI Qpid / Ingest Server postgreSQL Postgres Server Developer Portal mySQL BaaS Stack BaaS UI Management Server cassandra openldap zookeeper qpidd BAAS ANALYTICS SERVICES Message Processor Client Backend ©2015 Apigee. All Rights Reserved.
  14. 14. Apigee Edge architecture – Technology Stack Management ZookeeperConsistency Availability Partition Tolerance Our customers value " the ability to have highly available API expanded across geographically dispersed sites. Our customers value accurate metrics to " help drive data driven business decisions. Our customers value the ability to centralized management of distributed components. Apigee Edge CAP Theorem In theoretical computer science, the CAP theorem, also known as Brewer's theorem, states that it is impossible for a distributed computer system to simultaneously provide all three of the following guarantees: •  Consistency - all nodes see the same data at the same time. •  Availability - a guarantee that every request receives a response about whether it succeeded or failed. •  Partition tolerance - the system continues to operate despite arbitrary message loss or failure of part of the system. Text Credits: Wikipedia ©2015 Apigee. All Rights Reserved.
  15. 15. Apache Cassandra 15 ©2015 Apigee. All Rights Reserved. DC 1 CS CS CS DC 2 CS CS CS “Apache Cassandra is an open source distributed database management system. It is an Apache Software Foundation top-level project designed to handle very large amounts of data spread out across many commodity servers while providing a highly available service with no single point of failure.” -- Wikipedia http://cassandra.apache.org/ Cassandra characteristics: •  All nodes are equal. Not master/slave or primary/secondary. •  An application can read/write data from any node. •  Data replication. Apigee Edge uses replication factor 3. •  Consistency managed by application. Apigee Edge uses one and local quorum. Cassandra is used by Apigee for a variety of purposes, including: •  Storage of developer, application and API Product data •  Storage of access and refresh tokens •  Storage of key-value map data •  Audit logs •  Custom analytics report models QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: ©2015 Apigee. All Rights Reserved.
  16. 16. Apache Zookeeper 16 ©2015 Apigee. All Rights Reserved. DC 1 ZKV ZKV ZKV DC 2 ZKV ZKV ZKO “Apache ZooKeeper is a software project of the Apache Software Foundation, providing an open source distributed configuration service, synchronization service, and naming registry for large distributed systems.” " – Wikipedia Zookeeper is used by Apigee as a distributed configuration registry, tracking component location, configuration and status data. With some exceptions, it is NOT required to process API requests. •  Leader: The node that controls coordination of writes across distributed Zookeeper nodes •  Voters: Nodes that can vote on change proposal made by the Leader •  Observers: Do not vote on change proposals and must forward all writes to the Leader http://zookeeper.apache.org/ QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: ©2015 Apigee. All Rights Reserved.
  17. 17. Apache Qpid 17 ©2015 Apigee. All Rights Reserved. DC 1 QD QD DC 2 QD QD “Apache Qpid, an open-source (Apache 2.0 licensed) messaging system, implements the Advanced Message Queuing Protocol. It provides transaction management, queuing, distribution, security, management, clustering, federation and heterogeneous multi-platform support.” " – Wikipedia Qpid is used by Apigee Edge as messaging system for analytics and monetization data. http://qpid.apache.org/ QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: ©2015 Apigee. All Rights Reserved.
  18. 18. •  Apigee Edge architecture offers great flexibility when it comes to deployment options. •  Edge can be deployed from a single VM to a multi-datacenter active/active configuration. •  Edge has been designed from the ground up to be a true cloud scale solution, capable of running on both virtualized (including AWS) and physical hardware. Apigee Edge architecture – Deployment options QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: Node 1 MS UI R MP ZK CS AD QIS PS QD PG Demo Non-Production Node 1 Node 2 Node 3 Node 4 Node 5 Node 7 Node 8 Node 9 Node 10 R R QISMP MP PS QIS ZKV CS ZKV CS CS QD QD PGM MS UI OL Node 6 R MP ZKO Production Node 1 Node 2 Node 3 Node 4 Node 5 MS UI R R QISMP MP PS QIS PS ZK CS ZK CS CS ZK OL QD QD PGM PGS ©2015 Apigee. All Rights Reserved.
  19. 19. 1. Know your business Topology selection – Key considerations Business SLAs Infrastructure 2. Understand traffic patterns 3. Plan for growth (2X, 5X, 10X?) 4. Know your API proxies 5. Remember everything fails 6. API Traffic, Analytics and Management, Developer Service components can and should scale independently. R R MP MP CS CS CS PSQISQD PG MSUI OL Analytics Management DP MYDeveloper R MPAPI Traffic CS ZK QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: ©2015 Apigee. All Rights Reserved.
  20. 20. •  Horizontally scalable. Components can be installed and configured to run from a single node " (all in one setup) all the way to a multi DC centers, active/active, globally distributed setup. •  Ability to stack components. •  Distributed data replicated using eventual consistency. •  Asynchronous analytics data capture and processing. •  Centralized configuration for distributed components. •  Multitenant by design. •  Management via APIs and UI console. Apigee Edge architecture – Architecture characteristics DC2 DC1 Node 1 Node 2 Node 3 Node 4 Node 5 Node 7 Node 8 Node 9 Node 10 R R QISMP MP PS QIS ZKV CS ZKV CS CS QD QD PGM MS UI ELB OL ELB Load balancer Node 6 R MP Node 11 Node 12 Node 13 Node 14 Node 15 Node 17 R R QISMP MP PS QIS ZKV CS ZKV CS CS QD QD PGS MS UI OL Node 16 R MP ZKO ZKV Node 18 Node 19 Node 20 QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: ©2015 Apigee. All Rights Reserved.
  21. 21. •  Planet footprint is driven by customer requirements, transaction volumes, availability and reliability among others drive component stacking and number of nodes. •  R, MP and CS are critical components to handle live API traffic. •  Management and analytics components are not required on every DC. •  For high availability, ZK and QD should be available in all DCs. •  Analytics data replication via Master/Slave or Publish/Subscribe. Apigee Edge architecture – Horizontal scalability QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: DC2 DC2 DC2DC1 Node 1 Node 2 Node 3 Node 4 Node 5 Node 6 Node 7 Node 8 Node 9 R R QISMP MP PS QIS ZKV CS ZKV CS CS QD QD PGM MS UI OL Node 10 Node 11 Node 12 Node 13 Node 14 Node 15 Node 16 Node 17 Node 18 R R QISMP MP PS QIS ZKV CS ZKV CS CS ZKV QD QD PGS MS UI OL DC<N> Node 11 Node 12 Node 13 Node 14 Node 15 Node 16 Node 17 R R QISMP MP QIS ZKV CS ZKV CS CS ZKV QD QD ZKV ©2015 Apigee. All Rights Reserved.
  22. 22. •  Given the responsibility and capabilities offered by each component, scalability requirements and how they are implemented may vary. •  In most scenarios, scaling to accommodate higher number of TPS or API calls may impact only components serving live API traffic. •  Analytics and management components may grow in number mostly driven by demanding high availability requirements for those capabilities provided by analytics and management components. Apigee Edge architecture – Scaling by capability QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: QIS PSQIS QD QD PGM R R MP MP CS CS CS MSUI OL ZK ZK ZK API Traffic Analytics Management DP MYDeveloper ©2015 Apigee. All Rights Reserved.
  23. 23. Need correct heart 15 mins break
  24. 24. Agenda 09:30 – 10:00 Apigee Overview 10:00 – 11:00 Apigee Private Cloud Architecture BREAK (15 MINS) 11:15 – 12:00 Apigee Private Cloud Architecture (cont) BREAK – LUNCH (60 MINS) 13:00 – 15:45 Apigee Edge Install BREAK (15 MINS) 16:00 – 17:30 Platform Operations ©2015 Apigee. All Rights Reserved.
  25. 25. Apigee Edge architecture – Multitenant by design DC2 DC1 Node 1 Node 2 Node 3 Node 4 Node 5 Node 7 Node 8 Node 9 Node 10 R R QISMP MP PS QIS ZKV CS ZKV CS CS QD QD PGM MS UI ELB OL ELB Load balancer Node 6 R MP Node 11 Node 12 Node 13 Node 14 Node 15 Node 17 R R QISMP MP PS QIS ZKV CS ZKV CS CS QD QD PGS MS UI OL Node 16 R MP ZKO ZKV Node 18 Node 19 Node 20 QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: ©2015 Apigee. All Rights Reserved.
  26. 26. Multitenancy “Multitenancy is a reference to the mode of operation of software where multiple independent instances of one or multiple applications operate in a shared environment. The instances (tenants) are logically isolated, but physically integrated. The degree of logical isolation must be complete, but the degree of physical integration will vary. The more physical integration, the harder it is to preserve the logical isolation. The tenants (application instances) can be representations of organizations that obtained access to the multitenant application (this is the scenario of an ISV offering services of an application to multiple customer organizations). The tenants may also be multiple applications competing for shared underlying resources (this is the scenario of a private or public cloud where multiple applications are offered in a common cloud environment).” - Gartner Multitenancy = Software capabilities to support shared infrastructure " while providing tenants with data and processing partitioning. ©2015 Apigee. All Rights Reserved.
  27. 27. Apigee Edge Architecture – Multitenancy Region   Pod   Org   Virtual  Host   Environment   Regional Services are shared by many pods across a single geographical region and maintain state and provide an API that works across the pods in a region Pods are a collection of servers that share logical functions such as a Gateway Pod or an Analytics Pod. Orgs provide logical grouping to secure access to API management services. Orgs are associated with pods for servers – they can either share hardware or be isolated Environments are virtual routes that allow API bundles to be deployed and tested within and Org. Environments can be associated with pods for servers independently of the Org. Virtual Hosts are similar to Apache Virtual hosts and route traffic to environments based on ports and domain names. Planet Planets represent an entire physical environment and can encompass multiple regions and pods ©2015 Apigee. All Rights Reserved.
  28. 28. Apigee Edge Architecture – Multitenancy Data Center 1 Data Center 2 Data Center N Planet   Org  1   Environment  1   Environment  N   Org  N   Environment  1   Environment  N   •  A Planet can expand multiple DCs. Organization and Environment can expand across the planet. •  Data partitioning by Organization and Environment. Processing partitioning by Org+Env can be configured. ©2015 Apigee. All Rights Reserved.
  29. 29. Apigee Edge Architecture – Organizational structure Organiza3on   Environment   Cache  Resource   KVM  Resource   Target  Server   User   Role   Permission   Permission   Extension  Resource   Key  /  Trust  Store   Vault   API  Product   Company   Developer   Applica3on   Key  /  Token   KVM  Resource   Virtual  Host   API  Proxy   Extension  Resource   Vault   Deployment   ©2015 Apigee. All Rights Reserved.
  30. 30. Apigee Edge Architecture – Physical partitioning Tenant A Tenant B Tenant C Tenant A Tenant B Tenant C Tenant D Tenant E Tenant D Tenant E Tenant A Tenant B Tenant C Tenant D Tenant E Inbound Routing API Call Processing Data Storage ©2015 Apigee. All Rights Reserved.
  31. 31. Apigee Edge Architecture – Organizational structure DC2 DC1 Node 1 Node 2 Node 3 Node 4 Node 5 Node 7 Node 8 Node 9 Node 10 R R QISMP MP PS QIS ZKV CS ZKV CS CS QD QD PGM MS UI OL Node 6 R MP Node 11 Node 12 Node 13 Node 14 Node 15 Node 17 R R QISMP MP PS QIS ZKV CS ZKV CS CS QD QD PGS MS UI OL Node 16 R MP ZKO ZKV Node 18 Node 19 Node 20 Organization Environment Central POD Gateway POD Analytics POD Planet QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: ©2015 Apigee. All Rights Reserved.
  32. 32. Apigee Edge Architecture – API traffic data flow 32 ©2015 Apigee. All Rights Reserved. Legend: •  Routers send requests to Message Processors in their Gateway pod. •  If there are two or more gateway pods in a region, then routers will ignore message processors in the other gateway pods. •  Message Processors respect the region as their scope. •  For two data center the same rules apply as for one datacenter. •  All Apigee components are configured to only use the Cassandra nodes in their region / data center. Region / DC 1 R MP R MP Load balancer Client CS CSCS Backend GatewayPod1 R MP R MP GatewayPod2 QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: API call flow Analytics flow ©2015 Apigee. All Rights Reserved.
  33. 33. Apigee Edge Architecture – API traffic data flow 33 ©2015 Apigee. All Rights Reserved. R+MP Configuration options R MP R MP Load balancer Tenant Aware Routing •  Rs direct traffic to appropriate MPs, load balancing between them. •  Default behavior. •  Health check heartbeat allows R to automatically take MP out of/into rotation if unresponsiveness is detected. R MP R MP Load balancer Server Affinity •  R can also be configured to connect to a dedicated MP. Using server affinity, all traffic handled by a R is exclusively sent to its corresponding MP. •  This configuration option offers customer the ability to isolate R/MP for dedicated use cases without impacting API traffic flowing across other R/MP within the same pod. QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: API call flow Analytics flow ©2015 Apigee. All Rights Reserved.
  34. 34. Apigee Edge Architecture – Analytics data flow 34 ©2015 Apigee. All Rights Reserved. •  Ingest services in Qpid Ingest Server will collect Analytics data from all queues and store in PostgreSQL. •  Postgres Server aggregates analytics data asynchronously. •  Message Processors respect the region as their scope and will offload analytics data to all Apache Qpid queues in their region / data center. R MP R MP Load balancer GatewayPod QD QD CentralPod PGM QIS QIS PS QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: API call flow Analytics flow ©2015 Apigee. All Rights Reserved.
  35. 35. DC 2 DC 2 DC 1 DC 1 Apigee Edge Architecture – Analytics data flow 35 ©2015 Apigee. All Rights Reserved. QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: API call flow Analytics flow QIS PS PS QD PGM PGS MP Writes Writes Reads Writes QISQD MP Writes Writes Reads QIS PS QD PGM MP Writes Reads Writes QD MP Writes QIS PS PGM Writes Reads Reads Reads Reads/Writes Reads/Writes Reads/Writes •  Analytics data is partitioned per Environment. •  Analytics data size generated by MPs per transaction is about 1kb. •  PostgreSQL contains Raw Data Tables and Aggregated Data Tables. Raw Data Tables grow as analytics data is collected. Appropriate data retention policy and purge processes are required. •  Custom reports allow customers to define user-defined queries which run against raw data. ©2015 Apigee. All Rights Reserved.
  36. 36. Apigee Edge Architecture – Network Zoning 36 ©2015 Apigee. All Rights Reserved. QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: API call flow Analytics flow Data Data DMZ App DMZ Core R R QIS MP MP PS QIS ZKV CS ZKV CS CS QD QD PGM MS UI OL R MP ZKV DP MY QIS MP MP QIS QD QD MS UI OL MP R RR PS ZKV CS ZKV CS CS PGM ZKV DP MY QIS MP MP QIS QD QDMS UI OL MP R RR PS ZKV CS ZKV CS CS PGM ZKV DP MY Single Zone Two Zones Three Zones •  Edge architecture does not impose network zone requirements. Network zoning will be driven by customer operation and security requirements. •  Firewalls and security appliances between zones should accommodate the connectivity " and traffic characteristics of Apigee components without adding " latency overhead. •  Keep API traffic as direct as possible. Keep MP dependencies close. •  Pay attention to cross-zone connections. ©2015 Apigee. All Rights Reserved.
  37. 37. Apigee Edge Architecture – Components Connectivity 37 ©2015 Apigee. All Rights Reserved. QD QIS CS ZK CS ZK CS ZK AD / OL PG-standby PS Target Backend PG: 5432 QD: 5672 customer specific 10389 ZK: 2181 CS: 9160, 7199 PG: 5432 PG-master PS ZK: 2181 CS: 9160, 7199 UI MS QIS: 4529 PS: 4530 PG: 5432 PS PS MP MS MS MS MS QIS MP QIS MS ZK: 2181 2888, 3888 CS: 7000 7199, 9160 PG: 5432* PG PG QIS: 8083 1102 PS: 8084 1103 8081 1100 MS: 8080 UI: 9000 4528 M M M MP 8082 1101 M 8998, 4528 R customer specific MP MS 4527 MYDP 3306 MS: 8080 DP DP client / load balancer R R Router MP Message Processor CS Cassandra MS Management Server OL OpenLDAP ZK Zookeeper QD Apache Qpid QS Qpid Server PG PostgreSQL PS Postgres Server DP Developer Portal MY MySQL APItraffic Configuration Analytics Dev UI Management UI Legend: from svc:port 1-way 2-way to management calls HTTP or JMX) M Notes: •  For CS all nodes talk to all other CS nodes •  For ZK all nodes talk to all other ZK nodes •  * both directions to enable PostgreSQL failover QD: 5672 R Single DC View ©2015 Apigee. All Rights Reserved.
  38. 38. Apigee Edge Architecture – Components Connectivity 38 ©2015 Apigee. All Rights Reserved. R Router MP Message Processor CS Cassandra MS Management Server OL OpenLDAP ZK Zookeeper QD Apache Qpid QS Qpid Server PG PostgreSQL PS Postgres Server DP Developer Portal MY MySQL APItraffic Configuration Analytics Dev UI Management UI Legend: from svc:port 1-way 2-way to management calls HTTP or JMX) M Notes: •  For CS all nodes talk to all other CS nodes •  For ZK all nodes talk to all other ZK nodes •  * both directions to enable PostgreSQL failover Multi DC View ^ Data Center 1 Data Center 2 MP Target Backend R QD QIS CS ZK CS ZK CS ZK OL PG-master PS UI MS CS ZK CS ZK CS ZK OL QD QIS PG-standby PS R MP UI MS PG: 5432 10389 PG: 5432 4527 1100 4528 1101 4527 1100 4528 1101 ZK: 2181, 2888, 3888 CS: 7000, 7199, 9160 PG QIS MS MS ^Target Backend ©2015 Apigee. All Rights Reserved.
  39. 39. Need correct heart 60 mins break
  40. 40. Agenda 09:30 – 10:00 Apigee Overview 10:00 – 11:00 Apigee Private Cloud Architecture BREAK (15 MINS) 11:15 – 12:00 Apigee Private Cloud Architecture (cont) BREAK – LUNCH (60 MINS) 13:00 – 15:45 Apigee Edge Install BREAK (15 MINS) 16:00 – 17:30 Platform Operations ©2015 Apigee. All Rights Reserved.
  41. 41. Apigee Edge Install
  42. 42. System Requirements – Hardware Specification •  Cassandra –  Local storage with SSD or fast HDD supporting 2000 IOPS •  PostgreSQL –  Local storage with SSD or fast HDD supporting 1000 – 8000 IOPS –  CPU, Memory and Size the storage according to the data retention requirements. Implement archive / purge process and keep less data for best performance. –  Operations Guide provides formula to estimate storage needs. Type C ZKV CS Type G QIS QD Type H PS PGM Type A MS UI Type B OL Type   Components   CPU   Memory   Disk   A   UI,  MS   2  core   4  GB   60  GB   B   OL   2  core   4  GB   60  GB   C   ZK,  CS   8  core   16  GB   250  GB  local  storage  with  SSD  or  fast  HDD  supporOng  2000  IOPS   D   R,  MP   4/8  core   8/16  GB   100  GB   E   R   4  core   8  GB   60  GB   F   MP   8  core   16  GB   100  GB   G   QIS,  QD   8  core   16  GB   500  GB  local  storage  with  SSD  or  fast  HDD  supporOng  1000  IOPS   H   PS,  PG   8  core   16  GB   500  GB  to1  TB  local  storage  with  SSD  or  fast  HDD  supporOng  4000-­‐8000   IOPS   Type F MP Type D R MP Type E R QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: ©2015 Apigee. All Rights Reserved.
  43. 43. System Requirements – Software Dependencies https://apigee.com/docs/api-services/reference/supported-software Operating System •  Red Hat Enterprise Linux (64-bit): o  6.3, 6.4, 6.5, 6.6, 7.0 •  CentOS (64-bit): o  6.3, 6.4, 6.5, 6.6, 7.0 •  Oracle Linux (64-bit): o  6.5 JDK •  Oracle JDK 1.7 •  OpenJDK 7 SSL/TLS •  1.0 •  1.2 Other software •  Cassandra 2.0.15 •  Zookeeper 3.4.5 •  QPID 0.14 •  PostgreSQL 9.3 •  Play (UI) 2.3.4 •  OpenLDAP 2.4 ©2015 Apigee. All Rights Reserved.
  44. 44. Private Cloud Installer – Software Dependencies •  awk •  basename •  bash •  chkconfig •  curl •  date •  dirname •  echo •  expr •  grep •  hostname •  id •  ls •  perl •  pgrep (from procps) •  ps •  pwd •  python •  rpm •  rpm2cpio •  sed •  sudo •  tar •  tr •  uname •  unzip •  useradd •  wc •  yum Complete list of prerequisites can be found in Apigee Edge Install and Configuration Guide, Page 19. In addition to the tools above, some nodes require the installation of additional software components such as: •  ntp (all nodes) •  rsync (PostgreSQL nodes) •  openldap-clients openldap-servers (Management Server node) ©2015 Apigee. All Rights Reserved.
  45. 45. Apigee Edge Installation Process DC1 Node 1 Node 2 Node 3 Node 4 Node 5 Node 7 Node 8 Node 9 Node 10 R R QISMP MP PS QIS ZKV CS ZKV CS CS QD QD PGM MS UI OL Node 6 R MP ZKV QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: 1.  Install data store hosts 2.  Install management hosts •  Install LDAP first if using standalone LDAP hosts 3.  Install router, message processor 4.  Qpid hosts 5.  PostgreSQL hosts 6.  Configure PostgreSQL replication, if needed 7.  Create organization(s) and environment(s) If applicable, install: 7.  Developer portal 8.  Install monetization 9.  App services (BaaS) 1 2 3 4 5 ©2015 Apigee. All Rights Reserved.
  46. 46. Private Cloud – Software Install Apigee Private Cloud (apigee-edge-4.15.07.00.zip) has everything required to install and configure Apigee Edge and BaaS components except for the Developer Portal. The Developer Portal is distributed using a different package (DeveloperServices-4.15.07.00.tar). Apigee Edge installation steps: 1.  Acquire a license key and copy it to all nodes. 2.  Download the Apigee Private Cloud from ftp.apigee.com and copy it to all nodes. 3.  Installation downloads and installs required system software via Yum. It requires Internet connection or local repository. 4.  Unzip apigee-edge-4.<YY>.<MM>.<V>.zip 5.  Run the primary installation script: /<unzip-­‐location>/apigee-­‐edge-­‐4.<YY>.<MM>.<V>/apigee-­‐install.sh  -­‐j  /usr/java/default   -­‐r  <inst-­‐root>  -­‐d  <data-­‐root>   ©2015 Apigee. All Rights Reserved.
  47. 47. Private Cloud – Software Setup 1.  Once Apigee binaries are installed, run: /<inst-­‐root>/apigee4/share/installer/apigee-­‐setup.sh   2.  Setup script requires a profile type and information about the system, provided interactively or via response file (silent install). 3.  Create an organization, environment(s) and organization administrator, run: /<inst-­‐root>/apigee4/bin/setup-­‐org.sh   ©2015 Apigee. All Rights Reserved.
  48. 48. Private Cloud – Silent Install •  Silent install provides a way for specifying, in advance, all necessary values used by apigee-setup.sh •  Silent install provides a way for specifying, in advance, all necessary values used by apigee- setup.sh /<inst-­‐root>/apigee4/share/installer/ apigee-­‐setup.sh  -­‐p  ds  -­‐f  <response-­‐ file-­‐name>   •  The response files contain a number of variables definitions to be used by apigee-setup.sh. A response file per DC/Region will be needed since some values are unique per DC/Region. Single machine setups: •  aio = All In One (Gateway and Analytics Standalone) Cluster node setup for ZooKeeper and Cassandra (min 3 nodes): •  ds = Datastore Cluster Node LDAP setup for OpenLDAP: •  ld = LDAP Node Separate components setup: •  ld = LDAP Node •  ms = Gateway Management Server •  r = Gateway Router •  mp = Gateway Message Processor •  rmp = Gateway Router and Message Processor •  qs = Analytics Qpid Server •  ps = Analytics Postgres Server •  mo = Monetization Server Note: The list above doesn’t show all the available options. Refers to Apigee Edge Install and Configuration Guide, section Basic Host Installation, page 35 for details. ©2015 Apigee. All Rights Reserved.
  49. 49. Private Cloud – Silent Response File •  IP1= •  IP2= •  IP3= •  IP4= •  IP5= •  HOSTIP=$(hostname -i) •  MSIP=$IP1 •  ADMIN_EMAIL= •  APIGEE_ADMINPW= •  LICENSE_FILE=/root/opdk/license.txt •  USE_LDAP_REMOTE_HOST=n •  LDAP_TYPE=1 •  APIGEE_LDAPPW= •  ENABLE_AX=y •  MP_POD=gateway •  REGION=dc-1 •  USE_ZK_CLUSTER=y •  ZK_HOSTS="$IP1 $IP2 $IP3" •  ZK_CLIENT_HOSTS="$IP1 $IP2 $IP3" •  USE_CASS_CLUSTER=y •  CASS_HOSTS="$IP1:1,1 $IP2:1,1 $IP3:1,1" •  CASS_USERNAME= •  CASS_PASSWORD= •  SKIP_SMTP=y •  SMTPHOST= •  SMTPPORT=25 •  SMTPUSER= •  SMTPPASSWORD= •  SMTPSSL=n •  BIND_ON_ALL_INTERFACES=y ©2015 Apigee. All Rights Reserved.
  50. 50. Apigee Edge Install Lab
  51. 51. Apigee Edge Install Lab Scope •  5 nodes Apigee Edge Install •  VMs instances running in Amazon AWS •  Installation and setup using silent install process. •  Setup order DS, MS, RMP, SAX AWS Instances IP •  http://iloveapis.com/workshops/ops/ Documentation •  ApigeeEdgePrivateCloud-Install-Config-Guide.pdf o  Page 35 - Basic Host Installation o  Page 47 – Onboarding o  Page 120 - Appendix A: Silent Installation o  Page 121 - 5-host Clustered Installation Node 1 Node 2 Node 3 Node 4 Node 5 MS UI R R QISMP MP PS QIS PS ZK CS ZK CS CS ZK OL QD QD PGM PGS 2: MS 1: DS 3: RMP 4: SAX ©2015 Apigee. All Rights Reserved.
  52. 52. Apigee Edge Install Lab - Prerequisites Preparation A.  Software, sample install file and JDK are located in /root/opdk/ B.  Install prerequisites o  In all nodes: §  Install  JDK (rmp -ivh <rmp>)   §  yum  -­‐y  install  ntp   o  In node 1: §  yum  -­‐y  install  openldap-­‐clients  openldap-­‐servers   o  In node 4 and 5: §  yum  -­‐y  install  rsync   C.  In all nodes, unzip  apigee-­‐edge-­‐4.15.07.00.zip   D.  Update silent install file with private IP addresses corresponding to your AWS instances. Copy silent install files to all nodes. Additional considerations: •  For the purpose of the lab, we took care of some prerequisites. When planning for real world installation, prerequisites described in Apigee Edge Install and Configuration Guide, Page 19-35 should be completed. ©2015 Apigee. All Rights Reserved.
  53. 53. Apigee Edge Install Lab – Installation and Setup Install Steps A.  Unzip apigee-edge-4.15.07.00.zip B.  ./apigee-­‐install.sh  -­‐j  /usr/java/default  -­‐r  /opt  -­‐d  /opt   Setup Steps 1.  /opt/apigee4/share/installer/apigee-­‐setup.sh  -­‐p  ds  -­‐f  /root/opdk/response.txt   2.  /opt/apigee4/share/installer/apigee-­‐setup.sh  -­‐p  ms  -­‐f  /root/opdk/response.txt   3.  /opt/apigee4/share/installer/apigee-­‐setup.sh  -­‐p  rmp  -­‐f  /root/opdk/response.txt   4.  /opt/apigee4/share/installer/apigee-­‐setup.sh  -­‐p  sax  -­‐f  /root/opdk/response.txt   Node 1 Node 2 Node 3 Node 4 Node 5 MS UI R R QISMP MP PS QIS PS ZK CS ZK CS CS ZK OL QD QD PGM PGS 2: MS 1: DS 3: RMP 4: SAX ©2015 Apigee. All Rights Reserved.
  54. 54. Apigee Edge Install Lab – Installation and Setup Setup Master-Standby Replication for PostgreSQL •  Apigee Edge Install and Configuration Guide, Page 59 Onboarding •  Apigee Edge Install and Configuration Guide, Page 47 •  /opt/apigee4/bin/setup-org.sh Access Apigee Edge •  http://<management-server-public-ip>:9000 ©2015 Apigee. All Rights Reserved.
  55. 55. Platform Operations
  56. 56. Agenda 09:30 – 10:00 Apigee Overview 10:00 – 11:00 Apigee Private Cloud Architecture BREAK (15 MINS) 11:15 – 12:00 Apigee Private Cloud Architecture (cont) BREAK – LUNCH (60 MINS) 13:00 – 15:45 Apigee Edge Install Lab BREAK (15 MINS) 16:00 – 17:30 Platform Operations ©2015 Apigee. All Rights Reserved.
  57. 57. Platform Operations - Walkthrough •  Directory Structure o  /<inst-­‐root>/apigee4   •  Start / Stop / Status o  /<inst-­‐root>/apigee4/bin/all-­‐start.sh   o  /<inst-­‐root>/apigee4/bin/all-­‐status.sh   o  /<inst-­‐root>/apigee4/bin/all-­‐stop.sh   •  Logs file o  /<inst-­‐root>/apigee4/var/log   •  Management UI o  http://<management-­‐server-­‐host>:9000   •  Management API o  http://apigee.com/docs/management/apis   o  http://<management-­‐server-­‐host>:8080   ©2015 Apigee. All Rights Reserved.
  58. 58. Platform Operations – Backup and Restore •  Backup/restore o  Edge includes backup scripts for: §  LDAP §  Cassandra §  ZooKeeper §  PostgreSQL §  UI (custom reports) §  Component UUIDs o  <inst-root>/apigee4/bin/backup.sh •  High Availability and Disaster Recovery o  Multiple copies of the data o  Eventual consistency o  Horizontal scalability and resiliency DC1 Node 1 Node 2 Node 3 Node 4 Node 5 Node 7 Node 8 Node 9 Node 10 R R QISMP MP PS QIS ZKV CS ZKV CS CS QD QD PGM MS UI OL Node 6 R MP ZKV ©2015 Apigee. All Rights Reserved.
  59. 59. Platform Operations – Recurring Maintenance •  Cassandra nodes repair /opt/apigee4/share/apache-­‐cassandra/bin/nodetool  repair  -­‐pr   •  Periodically prune detailed analytics data based on retention requirements /opt/apigee4/bin/pg-­‐data-­‐purge.sh   ©2015 Apigee. All Rights Reserved.
  60. 60. Platform Operations – Component Level Monitoring JVM Statistics – Threads, heap, GC. Platform APIs. System Statistics – CPU, Memory, Disk, Network. DC1 Node 1 Node 2 Node 3 Node 4 Node 5 Node 7 Node 8 Node 9 Node 10 R R QISMP MP PS QIS ZKV CS ZKV CS CS QD QD PGM MS UI OL Node 6 R MP ZKV Monitoring Tool QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: •  System-Level Checks –  CPU, Memory, Disk, Load, Network •  Process/Application Checks –  Thread statistics, Memory utilization •  Components monitoring –  JMX –  Metrics –  Management API (http://<host>:<port>/v1/ servers/self/up) –  Router health check of the Message Processer –  Log monitoring •  HeartBeat •  Mark down/mark up events •  Logging Policy ©2015 Apigee. All Rights Reserved.
  61. 61. •  API-Level Checks –  API calls 20X responses –  API call error within Apigee –  API call error on backend –  JMX –  Metrics •  Message Flow Checks –  Number of active clients –  Number of responses (10X, 20X, 30X, 40X and 50X) –  Connect failures •  API Health - https://pages.apigee.com/api-health Platform Operations – APIs Monitoring QD PG OL ZK CS MY UI R MP MS QIS PS DP BA Developer Portal BaaS Server Router Message Processor Enterprise UI Management Server Postgres Server Qpid/Ingest Server Openldap PostgreSQL Apache Qpid MySQL Zookeeper Cassandra Server/Virtual Machine POD Legend: LB Backend System(s) R+MP R MP R MP Monitoring Tool Monitoring Tool Edge Analytics ©2015 Apigee. All Rights Reserved.
  62. 62. Takeaway Thank you! •  Public Cloud = Private Cloud •  Architecture purpose built and proven •  Horizontally scalable, components can be scaled independently of each other •  Multi-tenant by design ©2015 Apigee. All Rights Reserved.

×