Pivotal CF on Vblock Systems

vce.com

VCE TECHNOLOGY ALLIANCE PARTNER

PIVOTAL CF ON VBLOCK™ SYSTEMS –
SOLUTION AND INFRASTRUCTURE
SCALING OVERVIEW
Solution Validation
January 2014

© 2014 VCE Company, LLC. All rights reserved.

Terminology ......................................................................................................... 3

Business Case ..................................................................................................... 4

Executive Summary ........................................................................................................................ 4

Scope .............................................................................................................................................. 5

Audience ......................................................................................................................................... 5

Why Pivotal CF on Vblock Systems? .............................................................................................. 5

New Approach ................................................................................................................................ 6

Vblock™ System Advantage ............................................................................... 7

Vblock System 300 ......................................................................................................................... 7

Vblock System 700 ......................................................................................................................... 7

Vblock™ Data Protection and Mobility ............................................................................................ 7

VCE Vision™ Intelligent Operations ............................................................................................... 7

Solution Architecture .......................................................................................... 8

Pivotal CF Architecture Overview ................................................................................................... 9

Pivotal CF Elastic Runtime Service ........................................................................................................... 10

Pivotal CF Operations Manager ................................................................................................................ 11

Automated Health Management ................................................................................................................ 13

Pivotal CF Services ................................................................................................................................... 13

Infrastructure Layer Monitoring and Provisioning ......................................... 14

Monitoring ..................................................................................................................................... 14

Provisioning .................................................................................................................................. 16

Infrastructure Validation and Performance..................................................... 16

Testing Scope ............................................................................................................................... 16

Test setup ..................................................................................................................................... 17

Compute........................................................................................................................................ 18

StressTest ................................................................................................................................................. 18

Normalization ............................................................................................................................................. 18

Vertical Scaling .......................................................................................................................................... 20

Horizontal Scaling ...................................................................................................................................... 22

Storage.......................................................................................................................................... 24

Network ......................................................................................................................................... 24

Findings......................................................................................................................................... 25

Conclusion ......................................................................................................... 25

Next Steps..................................................................................................................................... 25


2

Terminology
Table 1 shows the terminology and abbreviations used in this document.

Abbreviation

Description

vCPU

Virtual or Logical CPU within VMware and Pivotal CF

Threading

A code task spawned as a discrete block of code executed by a vCPU

IO latency

Read and write latency in a virtual environment

Transaction

Any relational SQL or other database operation that must succeed or fail or roll back

Virtual memory

Memory presented to an application executing in Pivotal CF supported platform

Droplet

“A droplet is the result of the application staging process, it is an uploaded application
to which a buildpack has been applied. It is the original application (eg., a .WAR file
for Java), that has been compiled with all the required libraries called out in the
buildpack. The DEA has a wrapper around it that accepts one input—the port where
1
it should listen for HTTP requests—and has two methods, a start and a stop.”

DEA

Droplet Execution Agent stages and runs applications in Linux-virtualized containers,
called Warden containers. Warden is an enterprise grade container technology
developed by Pivotal, and is similar to LXC or Docker. The DEA subscribes to
messaging from the Cloud Controller and responds to requests by creating Wardens
as required. The DEA receives droplets and then starts or stops them as required.
Information about the droplet instances is periodically broadcasted using NATS.

Warden

“Warden manages isolated, ephemeral, and resource-controlled environments in a
virtual machine. The primary goal of Warden is to provide a simple API for managing
isolated environments. These isolated environments, or containers, can be limited in
terms of CPU usage, memory usage, disk usage, and network access. As of writing,
2
the only supported OS is Linux.”

Cloud Controller

“The Cloud Controller is written in Ruby and provides REST API endpoints for clients
to access the system. The Cloud Controller maintains a database with tables for
3
orgs, spaces, apps, services, service instances, user roles, and more.”

(GO) Router

“The Router routes traffic coming into Cloud Foundry to the appropriate
component— usually Cloud Controller or a running application on a DEA node. The
router is implemented in GoLang. Implementing a custom router in GoLang gives full
control over every connection to the router, which makes it easier to support
WebSockets and other types of traffic (e.g. via HTTP CONNECT) in the
5
architecture . All routing logic is contained in a single process, removing
unnecessary latency.”

4

1

http://docs.cloudfoundryPivotal CF.com/docs/using/terms.html

2

http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/warden.html

3

http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/cloud-controller.html

4

http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/router.html

5

http://docs.cloudfoundry.com/docs/running/architecture/


3

Abbreviation

Description

Health Manager

“The Health Manager is essential to ensuring that apps running on Cloud Foundry
remain available. It is needed to restart applications whenever the DEA running an
app shuts down for any reason; Warden kills the app because it violated a quota; or
6
the application process exits with a non-zero exit code.”

NATS

“NATS is a lightweight publish, subscribe and distributed queuing messaging system
written in Ruby. The various Cloud Foundry components use NATS to communicate
7
with each other.”

UAA

“The UAA is the identity management service for Cloud Foundry. Its primary role is
as an OAuth2 provider, issuing tokens for client applications to use when they act on
behalf of Cloud Foundry users. It can also authenticate users with their Cloud
Foundry credentials, and can act as an SSO service using those credentials (or
others). It has endpoints for managing user accounts and for registering OAuth2
8
clients, as well as various other management functions.”

BOSH

“BOSH is an open source tool chain for release engineering, deployment, and
lifecycle management of large-scale distributed services.
BOSH was developed to deploy Cloud Foundry, however it has a more general
9
purpose —it can be used to deploy other distributed services”

Instance

A single application droplet running in a Warden container constitutes one instance.

Business Case
Executive Summary
Enterprises are constantly looking for new ways to gain competitive advantage. In today’s rapidly- evolving global
landscape, they are increasingly more aware that technology is at the heart of what is driving business success,
and fine-tuning the IT investment strategy is imperative. One of the key technology-enabled initiatives is to turn a
collection of application infrastructure, development and related services; such as application platform, integration,
business process management, and database services; into a software model with a platform-as-a-service (PaaS)
approach. With PaaS organizations can drive faster time-to-results, gain better business outcomes, and enable
new business opportunities with productivity and scale of “software.”
However, to become a software-driven enterprise, organizations are often impeded by the complexity of
deploying, upgrading, and scaling a PaaS integrated with Infrastructure-as-a-Service (IaaS) on their private cloud.
Pivotal CF, the leading enterprise PaaS solution, powered by Cloud Foundry, is built to solve this complexity and
to deliver a turnkey experience for scaling and updating PaaS on the private cloud. It enables developers to
provision and bind leading application and data services on a unified platform with virtually no down time. It also
empowers businesses to deliver applications and update them with new features at a velocity and scale
previously only available to Internet giants. Another key aspect of gaining faster time-to-results with PaaS is to
deploy it on a converged system acting as a standardized infrastructure with adaptability and investment
protection. It delivers a pre-engineered, pretested, and fully-documented approach, allowing for a tremendous
improvement in the time and expense to bring an IaaS offering to market. Vblock™ Systems from VCE, the
6

http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/health-manager.html

7

http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/messaging-nats.html

8

http://docs.cloudfoundryPivotal CF.com/docs/running/architecture/uaa.html

9

http://docs.cloudfoundryPivotal CF.com/docs/running/bosh/


4

world’s most advanced converged infrastructures, provide predictable performance at scale and are integrated
with industry-leading solutions for virtualization, collaboration, and data protection. Vblock Systems also support a
range of applications and mixed workload environments, helping IT meet aggressive timelines for new initiatives
and respond quickly to changing business needs. This is why a growing number of organizations are pursuing the
deployment of Pivotal CF on Vblock Systems.
This document will look at the combined synergies that can be gained by deploying Pivotal CF on a Vblock
System. In this document, we will:
! Introduce the solution benefits of Pivotal CF and Vblock Systems
! Provide an overview of Pivotal CF architecture and Vblock Systems
! Explain how infrastructure monitoring and provision can be automated
! Validate and measure how infrastructure components of Vblock Systems function with Pivotal CF
The combination of Pivotal CF from Pivotal and Vblock Systems from VCE, serves as an ideal foundation for a
converged stack of IaaS and PaaS, enabling organizations to deploy unique cloud applications to outperform in
the marketplace, control costs and risks, and drive innovations.

Scope
This paper describes high-level solution architecture, key functions for scaling Pivotal CF, infrastructure monitoring
and provisioning, and performance and validation when deploying Pivotal CF on a Vblock System. It assumes that
a reader has a high-level understanding of IaaS and PaaS for cloud deployment. The lower level of functional
discussion is out of scope for this document.

Audience
This document is intended for large enterprises and service providers that are evaluating or planning to deploy
Pivotal CF on Vblock Systems.

Why Pivotal CF on Vblock Systems?
In today’s dynamic environment, agility is vital to turning cloud-based services into strategic advantage. Many
organizations across the industries are looking to PaaS offerings to develop new applications quicker and at great
scale. At the same time, adoption of converged systems has accelerated for an enterprise to take advantage of
the faster time-to-value, higher availability, and lifetime assurance unique to the pre-integrated, tested, and
validated solutions.
Pivotal One™ is a comprehensive, multi-cloud Enterprise PaaS comprised of a set of application and data
services that run on top of Pivotal CF™, the leading enterprise distribution of the Cloud Foundry® platform. This
new solution will allow enterprise companies to bring new innovation to market faster than ever before, enabling
agile development teams to rapidly update and scale applications across public or private clouds, and instantly
expand and upgrade with no downtime.
Vblock Systems from VCE also represents the next evolution of IT—one that unleashes simplicity by delivering
the extraordinary efficiency and business agility of virtualization and cloud computing. Seamlessly integrating bestin-class compute, network, and storage technologies from industry leaders, such as Cisco, EMC, and VMware,
Vblock Systems provide dynamic pools of resources that can be intelligently provisioned and managed to address
changing demands and business opportunities.
By deploying Pivotal CF on Vblock Systems, organizations are better positioned to achieve the speed, agility, and
predictability needed to be successful in deploying business-critical cloud applications. This solution white paper
describes the VCE solution for implementing Pivotal CF on Vblock System 300 and Vblock System 700 that are
suited to enterprises and service-provider environments.


5

New Approach
IT has been maturing its use of the cloud to drive a better business outcome for differentiation and innovation.
Traditionally, cloud-based IT implementations are challenged with gaps in operations, applications, and
infrastructure from people, process, and technology standpoints.
Typical challenges include:
! Inefficiency and delay in managing the application development process in disparate, heterogeneous
environments
! Limited adaptability in managing disparate infrastructures to handle multiple application developments
across the lifecycle
! Lack of predictability and scaling for the future, as transitioning from individual development
environments to QA and production
In order to solve these challenges, we need a solution that can
! Transform individual silos into a single platform
! Address a hybrid cloud approach that incorporates private, public, and existing environments
! Evolve legacy applications into a new platform
By taking a single platform for a hybrid, multi-cloud environment that embraces legacy applications and systems,
an organization can achieve the following benefits:
! Speed – Time-to-Value
o Fast—increase velocity of IT service contribution to business from application development to
infrastructure deployment and monitoring
o Improved—deploy high value-generating applications quickly instead of managing scripting
! Agility – Operational Efficiency
o Adaptable—scale environments elastically based on business demand
o Extensible—embrace and transform legacy environments to a platform approach
! Predictability – Reduced Risk and Cost
o Standardized—use scale as an advantage by replicating a proven, highly available deployment across
data centers
o Manageable—increase portability across private/public infrastructure while reducing footprint and power
needs
Table 2 shows the business benefits of using Pivotal CF on a Vblock™ System.

Benefit

Pivotal CF

Vblock Systems

Speed – Time-to-Value

! Speed application and data service
deployment from months to days

! Get up-and-running with 40+-day
lead time

! Scale hundreds of application
instances in seconds

! Meet 5X faster time for new services

! Ensure faster updates and
upgrades with minimal downtime

! Scale storage/network compute
tuned to demand

! Reduce infrastructure wiring, and
eliminate deployment bottlenecks

! Rapidly deploy mixed workloads on a
single platform

Agility – Operational
Efficiency


6

Benefit

Pivotal CF

Vblock Systems

Predictability – Reduced
Risk and Cost

! Improve application performance
and availability

! Achieve 96% reduction in downtime

! De-risk and separate concerns of
application development and
operations

! Reduce annual data center costs by
up to 50%

Source: VCE Systems metrics based on DC White Paper: Convergence with Vblock Systems:
A Value Measurement, September 2013

Vblock™ System Advantage
The expanded Vblock Systems portfolio is built on a single, highly-secure, standardized, market-leading
infrastructure. Pre-integrated, tested, and validated, the systems can be quickly deployed, easily distributed, and
counted on to increase the predictability and timeliness of IT projects for solution providers and end users alike. In
this validation and performance paper, we simulate a hybrid Vblock System 300 and Vblock System 700
environment that addresses large enterprise and service-provider deployment needs.

Vblock System 300
The Vblock System 300 family is ideally suited to achieve the scale needed for large virtualization and cloud
computing implementations. Engineered and validated to address a broad spectrum of mixed-workloads, the
Vblock System 300 family offers growing enterprise businesses a range of choices to minimize business risk and
Total Cost of Ownership (TCO). Additional information on Vblock System 300 can be found at
http://www.vce.com/products/vblock/system-300

Vblock System 700
The Vblock System 700 family is comprised of enterprise- and service provider-class systems designed to help
organizations benefit from virtualized and cloud computing—faster and easier. The systems are designed for
mission-critical application environments, offering the highest level of data and application availability. Additional
information on Vblock System 700 can be found at http://www.vce.com/products/vblock/system-700

Vblock™ Data Protection and Mobility
Vblock™ Data Protection and Mobility Solution contributes to true peace of mind by providing your choice of
safeguards, optimized for virtualized or cloud environments. Count on highly reliable backup and recovery, data
replication, business continuity, and workload mobility that ensure critical protection for applications and data
running on Vblock Systems. Additional information on Vblock Data Protection can be found at
http://www.vce.com/products/integrated/data-protection.

VCE Vision™ Intelligent Operations
VCE Vision™ Intelligent Operations software enables and simplifies converged operations by dynamically
providing a high level of intelligence to your existing management toolset. This next generation of the software will
enable IT to easily update non-compliant software or firmware and quickly scan for and report on configuration
security issues in need of remediation. VCE offers unprecedented choice in the management platforms used with
Vblock Systems. Plus, using VMware vCenter Operations Manager for Vblock System analytics will be
significantly simplified—for maximum utilization and performance. More information is available at
http://www.vce.com/products/vblock/vce-vision.


7

Solution Architecture
This paper will highlight the combined benefits that can be achieved by the deployment of the leading platform-asa-service (PaaS) offering, Pivotal CF based on Cloud Foundry, with the leading converged infrastructure-as-aservice (IaaS) offering engineered by VCE as shown in Figure 1.
Vblock Systems from VCE is an optimized information technology system that accelerates the adoption of
converged infrastructure and cloud-based computing models that dramatically reduce the cost of IT while
improving time-to-market for customers. Consisting of industry leading compute, network, and storage solutions,
VCE products enable IT organizations to
! Standardize processes and applications
! Improve utilization and lower operational costs
! Accelerate technology adoption and innovation
! Maximize adaptability and investment protection
Pivotal CF is the leading enterprise PaaS, developed by Pivotal on the popular open-source Cloud Foundry
platform. It delivers an always-available, turnkey experience for scaling and updating PaaS on the private cloud.
Just as the IaaS layer provides an abstraction from the infrastructure components, the PaaS layer offers the
abstraction of key components of the data-driven software platform. This includes an Elastic Runtime Service for
deploying, scaling, and managing applications; a flexible multi-language buildpack architecture that allows for
the detection, download, and configuration of the appropriate runtimes, containers, and libraries; an extensibleservices architecture that allows for the dynamic binding of internal and external services to the Pivotal CF
platform; and an Operations Manager (CF BOSH) for deploying, managing, scaling, and updating a PCF
installation.

Figure 1 illustrates how Pivotal and VCE high-level approach that address PaaS and IaaS stack together


8

Pivotal CF Architecture Overview

Figure 2 illustrates the solution components within Pivotal CF that interface with Infrastructure through VMware

Cloud Foundry is not a monolithic software stack. Rather, it is a scalable set of idempotent distributed services
that operate in concert to provide a highly reliable and scalable PaaS environment. As shown in Figure 2, this is
the Cloud Foundry inner shell. Components are dynamically-discoverable and loosely-coupled, exposing health
through HTTP endpoints so agents can collect state information (app state and system state) and act on that
information.
Pivotal CF offers a comprehensive set of the essential elements of an enterprise platform-as-a-service (PaaS)
offering that allows customers to change the way they manage software in order to compete in this new industrial
era. These essential elements allow enterprises to become more agile in their software delivery by allowing
enterprises to concentrate on the components of their software that will provide real business value and not spend
time worrying about the infrastructure and middleware configurations that tie up countless months of criticalresources time.
In the following section, we will examine three key components of Pivotal CF architecture that are crucial for
performance and scalability considerations.


9

Pivotal CF Elastic Runtime Service

Figure 3 depicts the Pivotal CF Elastic Runtime Service

Pivotal CF provides the ideal self-service platform for development teams of any size to start small and rapidly
evolve new ideas into complex applications, increasing business agility. Developers upload their application
source files using the CF CLI or via plugins for popular IDE and build tools like Eclipse, Maven, Gradle, Jenkins,
Bamboo, and more. The simplified semantics of the CF CLI and REST API allow users to manage and scale
applications, as well as create and bind zero-touch data services. Pivotal CF’s role-based control and lifecycle
staging isolation enable different teams to be responsible for early development, testing, staging, and ultimately
production. A web console allows developers and application owners to monitor, manage, and administer users
and services.
Developers simply push applications to Pivotal CF for an “it just works” experience. Pivotal CF uses a flexible
approach called buildpacks to dynamically assemble and configure a complete runtime environment for executing
a particular class of applications. Rather than specifying how to run applications, developers can rely on
buildpacks to detect, download, and configure the appropriate runtimes, containers, and libraries.
Figure 3 above shows how the various Pivotal CF components work together to allow a developer to deploy their
applications. A developer will communicate what application they want to push to the platform and which service
to bind to the application. The developer does not have to be concerned with the underlying plumbing, or how
these components are glued together. The Pivotal CF components handle the service provisioning, application
staging, and service bindings. The CF CLI also provides the ability for the developer to scale out the number of
instances of an application that are deployed by using a simple CF scale command and entering the number of
instances to deploy.
The Pivotal CF platform handles the deployment of the application into the droplet execution agent (DEA). The
DEA stages and runs applications in Warden containers that are responsible for managing isolated and resourcecontrolled environments. Once deployed, the application is available for end-user access. The platform also
ensures that the appropriate routing configuration is handled within the platform through the Router component.
An end-point URL is provided to access the application, and the application is available in the elastic runtime of
the pivotal CF platform. Figure 4 below shows how the Pivotal CF components work together to provide
application services to the end user based on the deployed application.


10

Figure 4 depicts the Pivotal CF workflow to provide applications to the end users

Pivotal CF Operations Manager
The Pivotal CF Operations Manager is an enterprise PaaS management component that manages all aspects of
the platform, from the one-click installation through the intuitive interface for horizontal and vertical scaling of the
platform, applications, and services, with zero downtime. Pivotal CF is the only PaaS that supports direct IaaS API
integration for turnkey deployment and full life-cycle management. Pivotal CF Operations Manager installs from a
single virtual appliance (OVA). From there, Operations Manager fully automates the deployment of Pivotal CF,
using IaaS APIs, onto a resource pool of Linux VMs with Linux containers.
Scaling of Pivotal CF components is done using the same Operations Manager console as shown in Figure 5.
PaaS administrators make changes to the number of instances in the settings tab of the runtime services and
perform an install. Pivotal CF does a comparison of the currently-deployed counts and deploys additional
components for each component size that has increased. This allows the PaaS administrator to dynamically scale
each of the components independently based on the scope of the applications and end-user traffic.


11

Figure 5 shows the Pivotal CF Operations Manager resources

The Pivotal CF Operations Manager also allows for the same installation and management capabilities for
services hosted in the Pivotal CF runtime such as Pivotal HD or Rabbit MQ. The interface provides the ability to
specify the same type of resource-sizing parameters that includes the instance counts, CPU, RAM, and disk
space. These capabilities provide for a consistent operations capability across the entire Pivotal CF platform.

Figure 6 shows the Pivotal CF Operations Manager for Pivotal HD


12

Automated Health Management

Figure 6 shows the Pivotal CF Application Health Management

The Pivotal CF Health Manager component is a third-generation application health manager that continuously
monitors all applications running on a Pivotal CF instance and intervenes to make the actual state match the
intended state as shown in Figure 6 above.
This is an important element of an enterprise PaaS that will be critical for every enterprise. The Health Manager
and the DEA work in concert to ensure that the expected applications, along with the number of instances, are
maintained. In the event of an application failure, the Health Manager will communicate to the Cloud Controller
any anomalies detected in the expected applications, and the Cloud Controller will automatically issue commands
to restart the expected application instances.
Another part of health monitoring is ensuring that each application is given its fair share of the available PaaS
resources, and that no individual application can adversely impact the overall health of the PaaS environment.
The DEA runs applications within Warden containers that are responsible for managing isolated- and resourcecontrolled environments. Quotas for resources such as memory and CPU can be set for each Warden container
in the Pivotal CF environment. The DEA monitors each Warden for adherence to the assigned quotas and will kill
a process that is in violation of these quotas to ensure a healthy platform for all applications hosted in the PaaS.
The normal health management process as described previously will kick in again, and new instances of any
killed applications can be restarted.
Pivotal CF Services
Enterprise applications rely on services to provide capabilities such as messaging and data persistence. Pivotal
CF provides a flexible architecture to incorporate these services with applications hosted in the Pivotal CF runtime.
Pivotal CF provides two ways in which services can be used by applications: managed services and userprovided services. Managed Services have been integrated via APIs and provides for the provisioning of new
service instances and credentials on demand. User-provided Service Instances are a mechanism to deliver
credentials to applications for service instances that have been pre-provisioned outside of the Pivotal CF platform.
The managed services approach is desirable when enterprises want to provide a set of services that can be
dynamically provisioned for new application instances along with being managed within the Pivotal CF runtime.
Pivotal provides a series of managed services including the following;
! Pivotal HD Service – this service provides the ability to build, manage, and scale Hadoop as a nativelyintegrated Pivotal CF Service. Via the Service Broker, applications can bind to this service, automatically

13

assigning capacity in Hadoop Distributed File System (HDFS), a database in HAWQ, and a resource
queue in YARN. This reduces development cycle time by eliminating common Hadoop development
complexities around deployment, security, networking, and resource management.
! Pivotal AX Service – this service offers a self-service analytics environment to the teams responsible for
the creation, collection, storage, querying, and visualization of data and lowers the barriers of entry to
analyze Pivotal CF-deployed applications and relevant supporting data to discover and communicate
meaningful patterns that affect profitability and future product direction. Pivotal AX service is on-premises
analytics software that is purpose-built on Pivotal HD and deploys and scales as a Cloud Foundry
Service.
! Pivotal RabbitMQ Service – this service provides increased application speed, scalability, and reliability by
delivering asynchronous messaging to applications. A message broker for applications running on Pivotal
CF, RabbitMQ Service applications can integrate with other Pivotal CF applications and with applications
outside Pivotal CF using the service broker.
In addition, Pivotal provides a fully-documented Service Broker API to allow developers or service providers to
incorporate their own set of services into the Pivotal CF runtime.
In scenarios where an enterprise already has existing, pre-provisioned services that live outside of the Pivotal CF
platform, a user-provided service instance can be utilized to bind an application hosted in the Pivotal CF runtime to
this external service. This flexibility accommodates all types of application and service configurations.

Infrastructure Layer Monitoring and Provisioning
Monitoring
One important consideration in a PaaS strategy is that, while the PaaS layer does automate the component
placement and installation on top of the IaaS layer, it does not take over the initial and ongoing operations and
management of the underlying IaaS layer. Existing IaaS administrators will continue with their operations
management using their tools of choice to manage, monitor, and maintain the operational efficiencies of the IaaS
Platform.
A tool such as vCenter Operations Manager from VMware can greatly simplify and automate operations
management of the IaaS layer. vCenter Operations Manager provides real-time analytics along with capacity
management capabilities that are critical functions for the management and monitoring of the IaaS layer. The realtime analytics of vCenter Operations Manager works by learning the operational patterns in the infrastructure
layer, and once the normal patterns are recognized, the system detects anomalies and alerts the IaaS
administrator of potential issues with the infrastructure shown in Figure 7 below.


14

Figure 7 shows the vCenter Operations Manager

The vCenter Operations Intelligent Operations provides the follow capabilities:
! Proactively avoid performance issues and gain deep insights into the health, risk, and efficiency of your
virtual and physical infrastructure, operating systems, and applications.
! Performance data is abstracted to health, risk, and efficiency measures that allow IT to efficiently identify
building performance problems with less effort.
! Customizable operations dashboards visualize key performance indicators, provide role-based access,
and enable better collaboration across infrastructure, operations, and applications teams.
! Integrated capacity management and cost metering capabilities track the usage, cost, and value of IT
resources being consumed.
! Intelligent operations groups provide operational visibility into logical groups of resources and
applications.
! Flexible group policies prioritize alert types and notifications for business-critical applications, production
workloads, or business units.
Planning for capacity involves analyzing, optimizing, and forecasting data to determine how much capacity is
available and whether you are making efficient use of the infrastructure. vCenter Operations Manager provides a
series of badges and reports that allows IaaS administrators to understand remaining capacity in the infrastructure
based on current sizing and utilization information from the analytics shown in Figure 8 below.


15

Figure 8 shows the vCenter Operations Manager for capacity scenarios

Forecasting data for capacity risk in vCenter Operations Manager involves creating capacity scenarios to examine
the demand and supply of resources in the virtual infrastructure. A what-if scenario is a supposition about how
capacity and load change if certain conditions are changed, without making actual changes to your virtual
infrastructure. If you implement the scenario, you know in advance what your capacity requirements are and what
additional capacity might be needed.

Provisioning
Organizations can also automate infrastructure deployment and provisioning, performing non-intrusive elastic
provisioning to add or release compute resources within a given infrastructure service.
Enabled by VCE Vision Intelligent Operations, EMC Ionix Unified Infrastructure Manager (UIM), and VMware
vCenter Operations Manager; infrastructure, application, and development teams can collaborate in configuring
and provisioning network access, storage, and compute profiles. Traditional methods to stand up a new
virtualization cluster that aggregates key data center functions such as compute, networking, storage, and
10
memory has been shown to take upwards of 160 days. Starting with a Vblock System reduces this time due to
pre-integrated components that are factory tested and pre-integrated. Additional tools from VCE, EMC, and
VMware help to further automate the additional tasks of virtualizing the infrastructure and the associated
management tasks. This includes capabilities such as automating the provisioning, compliance checking,
firmware upgrades, and scanning and reporting on security configuration issues. All these capabilities enable
enterprises to automate the provisioning and management of their IaaS layer and provide for greater agility.

Infrastructure Validation and Performance
Testing Scope
Stress testing on a hybrid Vblock System (see Test Setup below for details on the configuration) was performed to
identify how Pivotal CF would perform under stress on the underlying infrastructure components. This testing was

10

IDC Whitepaper – Convergence with Vblock Systems: A Value Measurement http://www.vce.com/asset/documents/noindex/idc-vblock-value-whitepaper.pdf


16

performed to help the infrastructure administrators identify how Pivotal CF utilizes the various infrastructure
components, including CPU, memory, networking, and storage.
A vSphere cluster was configured based on the requirements specified in the Pivotal CF Getting Started guide.
This guide recommends a minimal capacity based on the starting configuration of the Pivotal CF virtual machines
that are deployed during the installation process. Table 3 below summarizes the virtual machines with their
associated resource sizes after a Pivotal CF installation.
Table 3 summarizes the virtual machines with resource sizes

INSTANCES

VM

RAM (GB)

DISK (GB)

CPU

1

NATS

1

10

1

1

Health Manager

3

36

1

1

NFS Server

1

100

1

1

Cloud Controller Database

1

4

1

1

Cloud Controller

1

2

1

1

Router

1

2

1

1

Syslog

1

10

1

1

Collector

1

2

1

1

UAA Database

1

10

1

1

UAA

1

3

1

1

SAML Login

1

3

1

1

Console Database

1

3

1

1

DEA

16

11

2

1

Loggregator Server

1

2

1

1

Loggregator Router

1

2

1

6

Compilation

1

4

2

21

TOTALS

33

204

18

Test setup
For this validation and performance testing, we have chosen a hybrid structure where Vblock System 700 and
Vblock System 300, based on EMC Symmetrix VMAX and VNX, respectively.
Vblock System Environment
! EMC VMAX10K and VNX 5300
! 2 x UCS B200-M3 Blades
! 2 x Intel E5 2620 Processors
! 6 x 2.00 GHz cores per socket
! 2 Hyper-threads per core
! 192 GB RAM


17

! 96GB Licensed
! VMWare vCenter Server 5.1.0
! VMWare vSphere Hypervisor ESXi 5.1.0

Compute
StressTest
The stress application performs a repeated series of CPU intensive tasks within its container with varying memory
as part of the Compute testing. The application uses standard Cloud Foundry libraries and tools, and can be
deployed into a standalone instance, or multiple, distributed instances. The StressTest application is not
responsible for, nor does not task, storage or memory I/O.
Initial testing involved a single 8 vCPU DEA. A single 128MB instance of StressTest was uploaded into the DEA
and invoked via HTTP request.
Normalization
The size of the deployed instances was scaled to 256MB, 512MB, 1GB, and 2GB to confirm the StressTest CPU
stress isolation as shown in Figure 9 below.

Stress
2/2000
Single
Instance
Computational
Time
(ms)

at
Instances
128MB
-‐
2GB

18000

13000

8000

128MB

256MB

512MB

1G

2G

C:node_stress_testnode_stress_test>cf instances stress_master
DL is deprecated, please use Fiddle
Using manifest file manifest.yml
Getting instances for stress_master... OK
instance #0: running
started: 2013-11-08 02:53:36 PM
Output of cf instances command illustrating number of running instances
Figure 9 shows averaged results of StressTest at incremental instance sizes

StressTest was also demonstrated to consume 100% of the deployed Warden. It was observed that a Warden is
isolated to a single logical vCPU, instantiated within a DEA. The single StressTest instance deployed within the
normalization phase of testing was found to completely consume 1 of the 4 vCPUs deployed to the host DEA. It
was also found that the DEA successfully contained the StressTest Warden instance, by restricting access to
additional resources, proving that application isolation worked as expected in Figures 9, 10, and 11.


18

Figure 10 shows Single StressTest (node.js) consuming 1/8 DEA resources

Figure 11 shows vCenter view – DEA containment of warden resources


19

Vertical Scaling
The next phase of testing involved spawning multiple instances of the StressTest within a single DEA with the
intention of consuming up to 100% of DEA resources. For this test 8 StressTest instances were deployed into a
single 8 vCPU DEA. Multiple calls to the StressTest application instance gateway were made: first 2, then 4, 6,
and then 8; to demonstrate the effect of vertical scaling. As memory sizing was proved to have no impact to
StressTest testing earlier, 512MB instances were deployed as shown in Figure 12.
C:node_stress_testnode_stress_test>cf scale stress_master
Instances> 8
1: 128M
2: 256M
3: 512M
4: 1G
Memory Limit> 512M
Scaling stress_master... OK
Stopping stress_master... OK
Preparing to start stress_master... OK
Checking status of app 'stress_master'...
4 of 8 instances running (4 running, 4 starting)
Push successful! App 'stress_master' available at http://stresstest.cfapp.vce.superna.net
TIP: The system will continue to start all requested app instances. Try 'cf app' to monitor
app status.
C:node_stress_testnode_stress_test>cf instances stress_master
Getting instances for stress_master... OK
started: 2013-11-08 07:07:15 PM
started: 2013-11-08 07:07:15 PM


20

started: 2013-11-08 07:07:15 PM
started: 2013-11-08 07:07:15 PM
started: 2013-11-08 07:07:15 PM
started: 2013-11-08 07:07:15 PM
started: 2013-11-08 07:17:15 PM
started: 2013-11-08 07:07:15 PM
Figure 12 shows scaling up to 8 instances (0 to 7)

Table 4 illustrates compute resource consumption on a single DEA
Table 4 shows DEA OS CPU% utilization, reported by 'top'

Instances

DEA CPU% Util. (TOP)

1

12.5

2

25

4

56.7

6

74.5

8

99.4

DEA
OS
CPU
Utilization
(TOP)

120

100

99.4

80

74.5

60

56.7

40

20

25

12.5

0

1

2


4

6

8

21

Figure 13 shows DEA VM resource consumption OS CPU utilization plot (TOP)

Horizontal Scaling
While the previous section addressed vertical scaling of a single DEA on a single blade, this section addresses
horizontal scaling across two blades. First an additional DEA must be deployed. This operation is performed by
the PaaS administrator from the Pivotal CF Operations Manager interface. Pivotal CF Operations Manager is the
graphical interface to BOSH as shown in Figure 14 below.

Figure 14 shows Pivotal CF Operations Manager resources allocation

The PasS administrator logs into Pivotal CF Operations Manager and increases the number of DEA instances.
For this test, one additional DEA will be deployed to the second available blade. The configuration is then
updated, and BOSH seamlessly and automatically deploys a new DEA for use as shown in Figures 15 and 16
below.


22

Figure 15 shows DEA VM resource consumption


23

Figure 16 shows Blade Resource Consumption

Storage
As discussed earlier, Vblock System, as the IaaS component of this solution, can support elastic, non-disruptive,
on-demand scaling of PaaS environment. EMC VNX and VMAX platforms, as part of the storage-tier of the
infrastructure components, can be managed with EMC Unisphere with an easy-to-use wizard that provides an
intuitive, context-based approach to configuring storage, creating replicas, monitoring the environment, and
managing host connections.
Storage utilization within a Pivotal CF deployment will come from the initial platform storage requirements along
with the storage requirements for the associated services utilized by the hosted applications. Typical storagebased services would be those involved with the persistence or sources of data the applications will utilize, such
as a database service or a Hadoop platform, such as the Pivotal HD services described previously. The amount of
storage these services will require will depend on the hosted application and the business scenario supported by
the application. Application owners should be consulted in order to determine the specific storage requirements in
these scenarios.
It is noted that Pivotal CF application design best practices recommends the avoidance of writing to the local file
system. This means that application layer is state-less and that it does not share data.
This recommendation is due to two reasons: the local file system storage is short-lived, and instances of the same
application do not share a local file system (see http://docs.cloudfoundry.com/docs/using/app-arch/ for more
information on Application design consideration). Instead, it is recommended to use a database service, a cloud
storage provider, or a cache service as alternative approaches.

Network
This particular Vblock System configuration consists of the host blade (Cisco B200-M3), the UCS 6248, and the
Nexus 5548. This allows for up to 80Gbits of Ethernet data to be forwarded to Cisco UCS B200 Blade on a Vblock
System, when deployed with the Cisco 2208XP fabric extenders and VIC 1240/1280 combination.
For use with Pivotal CF, when the BOSH director creates a virtual machine through the BOSH IaaS interface, it
passes along configurations for networking and storage, as well as the location and credentials for the message
bus and the blobstore. A BOSH agent is also embedded in the template so that BOSH can take control of virtual
machines.
VMware virtual interfaces provided by VMM have been tested and demonstrated as being effective in limiting the
flow of Ethernet traffic to a configure rate, without impact to the VMM/Hypervisor, allowing normal operation to all


24

infrastructure systems, even when the virtual Ethernet ‘wire’ is saturated with traffic. Excess traffic is discarded,
and the TCP layer (windowing) transparently throttles packet flow.

Findings
We summarize the findings from this infrastructure scaling and performance testing on Pivotal CF on Vblock
Systems below.
! Application Isolation: The Warden component of Pivotal CF manages the isolation of environments in
terms of resource utilization. This capability will ensure that the applications are contained within the limits
prescribed and will only utilize the CPU, memory, disk, and network quotas defined. This ensures that no
single application will adversely affect the overall infrastructure platform or neighboring applications. When
reviewing infrastructure-monitoring tools, infrastructure administrators seeing adverse stress to an
individual application should work with their PaaS administrators to add additional instances of the
application.
! Vertical scaling: the compute linearly scaled and increased utilization with added instances on one DEA
on the first blade. The memory sizing was proved to have no impact to StressTest testing. DEA CPU
utilization increased linearly with increasing StressTest Warden instances.
! Horizontal scaling: the compute also demonstrated that a new DEA on the second blade also shows the
expected horizontal scaling. When one additional DEA was deployed to the second available blade, the
configuration was updated, and BOSH seamlessly and automatically deployed a new DEA for use.

Conclusion
Pivotal CF is a hypervisor agnostic platform that offers next generation lightweight application portability for rapid
workload execution scale-up, regardless of provider or underlying operating system. Vblock Systems provide the
world’s most advanced converged infrastructure to drive faster time-to-results, ensure lifetime assurance, and
reduce cost and risk of operations. The combined capabilities of a Vblock System for converged infrastructure and
the Pivotal CF platform for an enterprise PaaS empowers enterprises to drive agility into their application
development and operation lifecycle.
In this paper, we
! Introduced the solution benefits of Pivotal CF and Vblock Systems
! Provided an overview of Pivotal CF architecture and Vblock Systems
! Explained how infrastructure monitoring and provision can be automated
! Validated and measured how infrastructure components of Vblock Systems function with Pivotal CF
Organizations seeking to gain competitive advantage by becoming a more agile enterprise can look toward
deploying Pivotal CF on a Vblock System and take advantage of speed, agility, and predictability unique to the
joint offerings with the approaches described in this document.

Next Steps
To learn more about this and other solutions, contact a VCE representative or visit www.vce.com.

ABOUT VCE
VCE, formed by Cisco and EMC with investments from VMware and Intel, accelerates the adoption of
converged infrastructure and cloud-based computing models that dramatically reduce the cost of IT while
improving time to market for our customers. VCE, through the Vblock Systems, delivers the industry's
only fully integrated and fully virtualized cloud infrastructure system. VCE solutions are available
through an extensive partner network, and cover horizontal applications, vertical industry offerings, and


25

application development environments, allowing customers to focus on business innovation instead of
integrating, validating, and managing IT infrastructure.
For more information, go to vce.com.

!
Copyright © 2014 VCE Company, LLC. All rights reserved. VCE, VCE Vision, Vblock, and the VCE logo are registered trademarks or trademarks
of VCE Company LLC or its affiliates in the United States and/or other countries. All other trademarks used herein are the property of their
respective owners.


26

Pivotal CF on Vblock Systems

Recommended

Recommended

More Related Content

What's hot

What's hot (16)

Viewers also liked

Viewers also liked (17)

Similar to Pivotal CF on Vblock Systems

Similar to Pivotal CF on Vblock Systems (20)

More from EMC

More from EMC (20)

Recently uploaded

Recently uploaded (20)

Pivotal CF on Vblock Systems