O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

Cloudify workshop at CCCEU 2014

Cloudify workshop at CCCEU 2014

  • Seja o primeiro a comentar

Cloudify workshop at CCCEU 2014

  1. 1. Cloudify 3 Workshop #ccceu Budapest 2014 Uri Cohen - @uri1803
  2. 2. INTRODUCTION (OR WHY WE NEED THIS AT ALL…)
  3. 3. The World Is about Services & Apps
  4. 4. Automation Is Key for: • Testability • Consistency • Agility • Stability
  5. 5. The Automation Continuum Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
  6. 6. The Automation Continuum Environment Creation • VMs / Bare Metal Servers • Network (Firewall, NAT, VPN, etc.) • LB Groups • Storage (Block / Blob)
  7. 7. The Automation Continuum SW Infrastructure Setup & Configuration • OS Configuration (e.g. ulimit, useradd, permissions, etc.) • Installation of packages and middleware components • Startup orchestration • Update (not very frequent)
  8. 8. The Automation Continuum Code Push • User code installation on software infrastructure – e.g. jar, war, rails sources, PHP sources, DB scripts, etc. – After setup – On going - can be very frequent • Push policies (canary, red/black, a/b)
  9. 9. The Automation Continuum Monitoring & Alarming • What should be monitored? – Availability: are my services up or not? – Performance (OS &services level metrics). How are my services doing? – State: What’s running where, state of system resources (e.g. quota utilization) • Alarms upon failure or when reaching certain thresholds – Simple (a-la CloudWatch) or complex (CEP driven)
  10. 10. The Automation Continuum Repairing • Various types of failures – Service level – service not responding, process failed – VM / Node failure – Infrastructure failure (disk, network) • Automation tools can go a certain way – Resiliency should also be part of the SW stack
  11. 11. The Automation Continuum Scaling • Manually or Automatically (alarm triggered) • Scaling by – Adding / removing instances (AKA out / in) – Adding / removing resources to / from an existing instance (AKA up / down) • For both, it needs to be supported by the SW stack
  12. 12. Let’s Look at Some Tools
  13. 13. Orchestration Tools Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
  14. 14. CM Tools Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
  15. 15. Monitoring Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
  16. 16. Tying The Pieces Together Usually Looks Like This
  17. 17. A Way to tie all the pieces on the Automation Continuum together • Use what works best, not reinvent each piece from scratch
  18. 18. Open Source Platform for Deploying, Managing and Scaling Complex Multi-Tier Applications on the Cloud
  19. 19. So You Can Have This Environment Creation SW Infra. Setup & Config Code Push Monitoring & Alarming Repairing Scaling
  20. 20. Main Functions • Deployment modeling using TOSCA • Automated resource creation, placement, configuration and startup • Metric and log collection • Monitoring • Auto-repairing • Auto-scaling • Deployment updates (infra and app code)
  21. 21. Cloudify in the Devops Toolchain API Orchestration CI Monitoring & Alarming CM Infrastructure
  22. 22. Cloudify in the Devops Toolchain API Orchestration CI Monitoring & Alarming TOSCA CM Infrastructure
  23. 23. Architecture
  24. 24. Architecture
  25. 25. Architecture
  26. 26. Demo 1 Setup up a Manager on CloudStack
  27. 27. CLOUDIFY 3 BLUEPRINTS – DOING IT THE TOSCA WAY
  28. 28. Cloudify 3 Blueprints • A Blueprint is an Orchestration Plan for a single* application • It has 3 parts: • Topology: Declarative written in YAML DSL • Workflows: Imperative written in Python* • Policies: Declarative, written mostly in YAML • Conforms to TOSCA (next slide)
  29. 29. What is TOSCA? • Topology & Orchestration Specification of Cloud Application • By OASIS – Sponsored by IBM, CA, RH, Huawei and others • The goal is to allow for a cross cloud, cross tools orchestration of applications on the Cloud • Status: • Version 1 approved (XML). • Version 2 (also YAML) – in design
  30. 30. TOSCA-Inspired Application Blueprints Application Topologies Workflows Policies
  31. 31. Why TOSCA? • Standard • Can Describe • Any Topology • Any Automation Process • Portable between Clouds and Tools
  32. 32. So here’s an application
  33. 33. What do we see here? Host Middleware App module connection
  34. 34. What Have We Seen? • An application topology • 3 levels of components: • Infrastructure (Cloud or DC objects) • Platform or Middleware (App containers) • Application modules, schemas and configurations • Relationships between components (dependencies): • What’s hosted on what or installed on what • What’s connected to what
  35. 35. What’s in TOSCA Topology? • Inputs and outputs • Types and nodes • Relationships • Requirements and capabilities
  36. 36. Input and Outputs • A way to parameterize blueprints and let them declare runtime computed values (URLs, passwords, etc.)
  37. 37. Inputs and Outputs inputs: cloudstack_api_url: default: '' type: string cloudstack_key: default: '' type: string cloudstack_secret: default: '' type: string ... outputs: endpoint: description: Web application endpoint value: ip_address: { get_attribute: [ nodejs_vm, ip ] } port: { get_property: [ nodecellar_app, base_port ] }
  38. 38. Types & Nodes • Each component in the topology is a node: • For example, a VM is a node, a Webserver is a Node • The node holds the configuration (properties) and the relationships to other nodes • A node has a type • The type is where the lifecycle interface operations are defined • The type specified the properties schema • Default lifecycle operations are: • create • configure • start • stop • delete 38
  39. 39. Type Example 39 Can be scripts or references to Python functions implemented by plugins
  40. 40. Relationships • There are 3 types of relationships: • depends_on – which is the base type • conataind_in – a component is hosted / contained / deployed within nother • connected_to – a component needs to establish a connection to another and therefore this needs to be configured • The relationship can define operation to be applied on the source of the target instances • Possible operations on each: • preconfigure – before node configure is called • postconfigure – after node configure is called but before start • establish – after start when connection needs to be established • unlink – remove the connection 40
  41. 41. Relationship Example Can be scripts or references to Python functions implemented 41 by plugins
  42. 42. Requirements and Capabilities
  43. 43. One Way of Putting This Nodecellar_app IS CONNECTED to mongod
  44. 44. Another Way of Putting This Nodecellar_app NEEDS a database of type ‘MongoDB’
  45. 45. Requirements and Capabilities • Relationships will soon be replaced by a more declarative model created by the latest TOSCA work draft • “This type needs a database connection” instead of “This node is connected to a node that happens to be a database” • Cloudify to follow once spec approved
  46. 46. Requirements and Capabilities • A node type declares a certain capability • Another type in a blueprint declares that it requires this capability • A node in a blueprint can also declare that it needs a capability
  47. 47. Requirements and Capabilities tosca.nodes.Database: derived_from: tosca.nodes.Root properties: db_user: type: string db_password: type: string db_port: type: integer db_name: type: string description: the logical name of the database capabilities: - database_endpoint: tosca.capabilities.DatabaseEndpoint node_templates: wordpress: type: tosca.nodes.WebApplication.WordPress requirements: - host: webserver - database_endpoint: mysql_database
  48. 48. A Word About Blueprint Portability • Cloudify blueprint support the notion of abstract types – Kind of like abstract classes in OOD • Blueprints can be composed of multiple files • To achieve portability: – Topologies should be defined using portable types – Concrete types are then wired at blueprint creation time • Kind of like dependency injection
  49. 49. Workflows • TOSCA 1.0 –Workflows (Plans) are in any WF language. – Strong preference for BPMN 2.0 • TOSCA 2.0 – No change • Cloudify 3 take – Workflows are currently python based – Tinkering with a more declarative approach (OpenStack Mistral?)
  50. 50. Policies • Still not defined by TOSCA, under discussion • Cloudify 3 – YAML mostly, uses Riemann.io under the hood – You can create very sophisticated custom policies, in Clojure…
  51. 51. Putting it all together • TOSCA Template (Blueprint in Cloudify) contains: – Application Topology • Nodes – Interfaces and operations – Properties – Relationships – Workflows (install, uninstall, scale out, CD) – Policies (scale trigger, recovery trigger)
  52. 52. Demo 2 Upload a blueprint and install it
  53. 53. WHAT REALLY HAPPENED HERE?
  54. 54. We Triggered a Few Processes • Blueprint and deployment creation • Workflow execution • Availability reporting • Log and metric collection
  55. 55. BLUEPRINT AND DEPLOYMENT CREATION
  56. 56. Blueprint Upload 59 Gunicorn Node ElasticSearch Cloudify Manager Nginx Blueprint saved in ElasticSearch
  57. 57. Deployment Creation 60 Gunicorn Node ElasticSearch Cloudify Manager Nginx Celery Worker Celery Workers Dedicated deployment workers created RabbitMQ
  58. 58. WORKFLOW EXECUTION
  59. 59. The Bigger Picture Agent CeleryD Worker Worker Worker Plugin RabbitMQ Tasks Logs Workflow Worker Logstash Elastic Search Runtime Properties Created Riemann
  60. 60. Workflow Execution
  61. 61. LOGS & EVENTS
  62. 62. Logs & Events Functionality • Cloudify components logs are gathered, indexed and persisted • Events give the user a simple and clear way to trace the progress of a workflow execution • Available from API, CLI and web GUI
  63. 63. Logs & Events Mechanism Source Component RabbitMQ Log Stash Elastic Search REST API Queued Formatted + Piped Indexed + Persisted Queries
  64. 64. Closing the Feedback Loop Celery Collectd / Diamond RabbitMQ Application Stack Nagios Zabbix … Riemann Cloudify Manager Third party monitoring tool Metrics VM InfluxDB Logstash App VM
  65. 65. Policy Engine Work Model When handler function detects policy violation it emits an event that triggers a workflow 1. User can define a selector to create a stream 2. Apply rules for stream to decide about the output Monitoring tools send events to Riemann
  66. 66. Demo 3 Monitoring
  67. 67. If You Want to Dive Deeper • Plugins – IaaS: CloudStack, OpenStack, AWS (libcloud), vSphere, vCloud, SoftLayer – Chef, Puppet, Salt, Fabric, Docker – Role your own • Workflows • Policies
  68. 68. References • Cloudify home: getcloudify.org • Github: github.com/cloudify-cosmo • CloudStack integration (in the works): github.com/cloudify-cosmo/?query=cloudstack • TOSCA: https://www.oasis-open.org/committees/tosca/
  69. 69. Thank You!

    Seja o primeiro a comentar

    Entre para ver os comentários

  • giganati

    Nov. 20, 2014
  • gershond

    Nov. 20, 2014
  • kkitase

    Nov. 26, 2014
  • ykhroki

    Nov. 27, 2014
  • DanielFry

    Dec. 22, 2014
  • WangTianqing

    Jan. 11, 2015
  • chroum

    Jan. 23, 2015
  • budibudifr

    Apr. 13, 2015
  • jj_tyro

    Aug. 6, 2015
  • ryuleonheart

    Oct. 8, 2015
  • seoksik

    Dec. 10, 2015
  • Bing-WU

    Mar. 14, 2016
  • yuanhang2

    Aug. 26, 2016

Cloudify workshop at CCCEU 2014

Vistos

Vistos totais

2.650

No Slideshare

0

De incorporações

0

Número de incorporações

82

Ações

Baixados

123

Compartilhados

0

Comentários

0

Curtir

13

×