Exploring a simpler, more portable, less overhead solution to deploy Elasticsearch Pink Metrics and Customizer for IBM Connections

Philadelphia, April 26-27 2018
13
Exploring a simpler, more portable, less
overhead solution to deploy Elasticsearch
Metrics and Customizer
Conall Ó Cofaigh, IBM
ocofaigh@ie.ibm.com

Social Connections 13 Philadelphia, April 26-27 2018
PLATINUM SPONSOR
GOLD SPONSORS
BRONZE SPONSORS
GOLD PLUS SPONSOR
SILVER SPONSORS
SPEEDSPONSORING BEER SPONSOR

Agenda
• About Me
• What is Customizer and Elasticsearch?
• Reflection & Feedback
• IBM Cloud Private
• Proof of Concept
• Demo
• Next Steps
• Summary
• Q&A

About Me
• Working in IBM Collaboration Solutions (ICS) since 2008.
• Based out of the IBM Technology Campus, Mulhuddart,
Dublin, Ireland
• Background in System Test on the IBM Sametime product.
• Working on Connections Pink for the last year.
• Current role:
• DevOps for IBM Connections private cloud offering
• Responsible for the “Component Pack” installer (the big zip!)
• Installer/bootstrap scripts
• Packaging up the zip so all of our microservices will work together
• Installation & configuration documentation

Questions For You
• Who has experience with Docker?
• Who is using a Docker registry?
• Who has experience with Kubernetes?
• Who has worked with Helm?
• Who is using Helm repo?
• Who has services hosted in a managed cloud (e.g.
Bluemix, AWS, Azure etc)
• Who has deployed Component Pack?

What is Customizer?
• IBM Connections Customizer is a proxy service that lets you
modify the IBM Connections user experience.
• Customizer intercepts and modifies requests from clients and
responses from servers, so it can customize anything that
flows through it, such as the behaviour of APIs or the look-
and-feel of the user interface.
• More info:

What is Elasticsearch?
• IBM's Elasticsearch Metrics lets you visualize your data more
easily and get meaningful metrics.
NB: To continue to receive IBM support for the IBM
Connections Metrics feature after April 30, customers are
required to install Elasticsearch to replace IBM Cognos
• Both Elasticsearch and Customizer are available in the
Component Pack 6.0.0.5 release.

Reflection
• Component Pack 6.0.0.5 shipped in Feb 2018.
• Since then, IBM have spent some time discussing the current
solution for supplying IBM Connections services to customers.
• We have gathered customer and business partner feedback,
as well as explored the current industry standard for supplying
services.
• General feedback was:
• Even though deployment is a lot easier than the WebSphere
Connections solution, some people still find it too complicated.
• Love the “Pink” functionality, but requires a lot of resources for what it
does.
• Not really flexible for hosted clouds such as Bluemix /AWS / Azure etc.

Feedback
Even though deployment is a lot easier than the WebSphere Connections
solution, some people still find it too complicated.
• Trying to make the installer work for every kind of possible customer architecture led
to a lot of optional flags…

Feedback
Love the “Pink” functionality, but requires a lot of resources for what it does.
• For example, to deploy a full Component Pack stack, production ready
environment, with full high-availability support, one would require:
• 3 Master machines (4 CPU, 16GB RAM, 100GB disk) + a floating IP to load balance
• 3 Generic Worker machines – co-located proxy on each worker node (6 CPU, 16GB RAM,
100GB disk)
• 3 Infrastructure Worker machines (4 CPU, 16GB RAM, 100GB disk)
• 1 Storage server for persistent volumes and ICp image repositories (150GB disk)
• 9 very high specked machines (+ Storage) on top of the current
Connections WebSphere / DB2 stack was just ringing alarm bells!
• The bulk of these specifications come from IBM Cloud Private.

Feedback
Not really flexible for hosted clouds such as AWS / Azure etc.
• Many of Connections customers have deployed (or plan to deploy) third
party services on hosted clouds such as AWS, Azure or Bluemix.
• Such cloud platforms provide very simple startup scripts to deploy
Kubernetes clusters.
For example:
• AWS – Kops, Kube-AWS
• Azure - Azure Container Service (Using CLI or online web browser portal)
• Bluemix - IBM Cloud Container Service CLI or API
• The Component Pack installer installs and configures the underlying
Kubernetes cluster using the IBM Cloud Private product.

IBM Cloud Private
• Formerly IBM Spectrum Conductor for Containers (CfC).
• Server platform for developing and managing on-premises,
containerized applications using Docker and Kubernetes.
• Provides services such as authentication, storage,
networking, logging, and monitoring.
• A cluster management console is also provided, which allows
for centralized management of these services.

IBM Cloud Private + Component Pack
• When “Pink” development started more than 2 years ago, Docker and
Kubernetes were technologies new to many IBM Connections admins.
• We choose IBM Cloud Private (or CfC as it was called then) as a base
platform to set the Kubernetes environment up as part of our installer.
• CfC could be pulled from the web with a docker pull command which fit
nicely into our bash installer scripts.
• Our installer also installed Docker and configured Kubernetes for our
microservices to run on.
• This meant admins did not have to have an in-depth knowledge about
Docker or Kubernetes, and by following the online documentation, could
deploy a working Orient Me solution out of the box.

IBM Cloud Private Evolution
• CfC became ICp.
• ICp was split into CE and EE installers:
• Community Edition (CE) – no HA support
• Enterprise Edition (EE) – HA support but only available
from Passport Advantage (1.7GB in size)
• Component Pack 6.0.0.5 is shipped with IBM Cloud Private
1.2.1.
• ICp 1.2.1 deploys Kubernetes v1.6 – no longer supported by
Kubernetes (current latest is v1.10).

IBM Cloud Private 2.1.0.2
• The latest ICp version is 2.1.0.2.
• This comes with Kubernetes v1.9.
• Installer package is now 8GB in size.
• Contains more than is needed by IBM Connections. EG:
• IBM Cloud Private Cloud Foundry
• Metering dashboard
• VMware integration
• ICp 2.1.0.2 EE is also still only available from Passport
Advantage (no docker pull).
• Conclusion: ICp is a fantastic product, but it is over featured
for Connections services.

Action Items
• Does our solution work without ICp?
• Does it work without Kubernetes?
• Can we simplify our deployment process?
• Can we make it more flexible?
• Can we make it less resource heavy?

Proof of Concept – Removing ICp
• Think back to why we choose to use ICp…
• It installed and configured the Kubernetes environment – no in-
depth Kubernetes knowledge required
• Lightweight and easily automated – docker pull
• It did everything we needed to at the time

• Reality check:
• ICp EE only available on Passport Advantage – download can’t be
automated
• ICp 2.1.0.2 is 8GB in size, too big to fit inside our installer zip
• ICp is over featured for Connections
• Kubernetes has become a well known technology and can now be
deployed very easily using a wide range of different tools
• Many IBM Connections customers are using their own Kubernetes
platforms for other services
• It does not make sense to deploy ICp on hosted clouds such as AWS,
Azure or Bluemix when similar solutions are already provided
• Most service providers pre-req Docker and an orchestration tool (such as
Kubernetes, Swarm, Apache Marathon etc) – more of an industry
standard these days

• As a proof of concept, we decided to “hack” the Component
Pack 6.0.0.5 installer scripts to see if everything would work in
a world without ICp.
• We decided to focus on our 2 smaller stacks, Elasticsearch
and Customizer.
• To do this, we had to manually install Docker and Kubernetes
as a pre-req.
• We used the Kubeadm tool to install a Kubernetes cluster,
but there are tons of tools out there to do this for you
depending on what your hosting platform is.
• Result:
• We successfully deployed all pods required for both stacks – great!

Proof of Concept – Removing K8s?
• With ICp now gone in our POC, we decided to investigate if we
could remove the Kubernetes dependency.
• We looked at using Docker Compose (tool for defining and
running multi-container Docker applications).
• With a single command and a YAML file, you can create and start
the services.
• Disadvantages:
• No orchestration
• No container management
• Not recommended for production
• No HA
• Hard to manage

Proof of Concept – Removing K8s?
• Conclusion:
• Docker Compose is great for standing up a service with a
single command, but without orchestration to manage your
containers, launch them on the underlying machines, make
sure they are distributed, and keeping them healthy, its not
going to be feasible for production
• For this reason, we decided to continue using Kubernetes for
this proof of concept …

Helm
• Helm is a package manager for Kubernetes.
• Think of it like apt/yum/homebrew for Kubernetes.
• A Helm Chart is a package that contains information that is
required to stand up a service.
• Charts can be stored on disk, or fetched from remote chart
repos.
• Component Pack 6.0.0.5 is using Helm v2.7.2 to install /
upgrade services.

Installing Helm
• Installing Helm is extremely simple..
Prereq: A Kubernetes cluster
1. Download the Helm binary from
https://github.com/kubernetes/helm
2. Unpack the helm binary and add it to you PATH

Helm commands
• Once you have Helm ready, you can initialize the local CLI
and also install Tiller into your Kubernetes cluster in one step:
helm init
• This will install Tiller into the Kubernetes cluster.
• To install a chart, you can run the helm install command:
$ helm install alpine
---> No chart named "alpine" in your workspace. Fetching now.
---> Running `kubectl create -f` ...
pods/alpine
---> Done
========================================
# Alpine
This Chart provides a Pod running Alpine Linux.
========================================

Helm commands
• Helm handles the packages up through initial installation, but
kubectl is still the way to go for working with your cluster:
$ kubectl get pods
NAME READY REASON RESTARTS AGE
alpine 1/1 Running 0 22s
• The helm list function will show you a list of all deployed
releases.
$ helm list
NAME VERSION UPDATED STATUS CHART
Alpine 1 Wed Sep 28 12:59:46 2016 DEPLOYED alpine-0.1.0

Helm commands
• To uninstall a release, use the helm delete command:
$ helm delete alpine --purge
release “alpine” deleted
Use --purge to remove the release from the release list, freeing up its name for later use
(removes release history)

Proof of Concept – The Overview
• Based on the investigation done, a POC without ICp using
pure Helm commands seemed to be the way forward to a
simpler, more portable, less overhead solution.
• This would open the door for:
• Deploying in hosted clouds
• Using more lightweight Linux Operating Systems such as Alpine
or CoreOS
• Configuring a Docker registry to pull Docker images
• Configuring a Helm repository to pull Helm charts

Proof of Concept – Pre-reqs
• Install Docker
• For this POC, we choose to stick with Docker 17.03 as this is the
recommended version to use with Kubeadm
• Install Kubernetes
• For this POC, we choose Kubernetes v1.8, as this currently the
latest version that our service YAML files will work with
• Install Helm
• For this POC, we choose Helm v2.7.2, as it is the version
applicable for Kubernetes v1.8

Proof of Concept – Pre-reqs
• Create Kubernetes namespace
• A namespace must be created for our services to live in
• Command: kubectl create namespace <namespace>
• Create Storage
• Both Customizer and Elasticsearch require Kubernetes persistent
storage
• Docker Registry
• For this POC we have removed the images from the zip, and in the
values.yaml of all the Helm packages we have added our internal
pipeline JFrog server
• This can be updated to use a Docker Registry (using global variable
replacement) when running the helm install command

Proof of Concept – The Breakdown
• setup.sh
• Sets up environment for Helm install
• Supports choosing a pre-exisitng namespace (-n connections)
• Supports multiple Starter Stack set up (-st elasticsearch,customizer)
• Calls validate.sh and configure.sh
• validate.sh
• Validate Docker
• Validate Kubernetes
• Validate Helm
• Validate Persistent Volumes
• configure.sh
• Creates required secrets / certs by prompting for passwords
• All prompts can be avoided (for automation) by using flags with setup.sh

Proof of Concept – Install with Helm
• In Component Pack, every service has its own Helm Chart.
• For this POC, in order to use Helm to install Elasticsearch
and Customizer, we require just 4 Helm charts (using sub-
charts):
1. connections-env – creates a configmap of values used by
the services and some generic secrets
2. infrastructure – mongo, redis, redis-sentinel & haproxy
3. elasticsearch - elasticsearch
4. customizer – mw-proxy, appreg-client, & appreg-service

Proof of Concept – Install with Helm
• Before installation, run the setup.sh
• To install Customizer, the following Helm charts would need to
be installed:
1. connections-env
2. infrastructure
3. customizer
• To install Elasticsearch, the following Helm charts would
need to be installed:
1. connections-env
2. elasticsearch

Proof of Concept – Install Customizer
• Install connections-env:
helm install
--name=connections-env helmbuilds/connections-env/connections-env-
0.1.40-20180413-180651.tgz
--set
createSecret=false,
ic.host=ic-entry.ibm.com,
ic.internal=http-server.ibm.com
This can be Helm repo instead
IBM Connections front door
address (e.g. load balancer)
IBM Connections HTTP Server

• Install infrastructure:
helm install
--name=infrastructure helmbuilds/infrastructure/infrastructure-
0.1.0.tgz
--set
mongodb.createSecret=false,
mongodb.onPrem=true
This is where you would pass in your Docker registry that would contain the images
,
image_repo=https://mydockerrepo.com

• Install customizer:
helm install
--name=customizer helmbuilds/customizer/customizer-0.1.0.tgz

Proof of Concept – Install Elasticsearch
• Install connections-env:
• NOTE: If you installed customizer, this will already be installed
helm install
--name=connections-env helmbuilds/connections-env/connections-env-
0.1.40-20180413-180651.tgz
--set
createSecret=false,
ic.host=ic-entry.ibm.com,
ic.internal=http-server.ibm.com
IBM Connections front door
address (e.g. load balancer)
IBM Connections HTTP Server

Proof of Concept – Install Elasticsearch
• Install elasticsearch:
helm install
--name=elasticsearch helmbuilds/elasticsearch/elasticsearch-0.1.0-
20180307-024239.tgz

Proof of Concept – Upgrading
• Releases can be upgraded using the helm upgrade command
Example:
helm upgrade customizer helmbuilds/customizer/customizer-0.2.0.tgz
• You can check the history of a release using the helm history
command:
$ helm history customizer
REVISION UPDATED STATUS CHART DESCRIPTION
1 Mon Apr 23 11:01:49 2018 SUPERSEDED customizer-0.1.0 Install complete
2 Mon Apr 23 11:02:17 2018 DEPLOYED customizer-0.2.0 Upgrade complete
Example:

13
Demo

Proof of Concept – Next Steps
• All of this is still a work in progress!
• The following items are currently being investigated:
• Can we remove the bootstrapping bash scripts that set up the Kubernetes
secrets and certificates? (Vault, Docker container ??)
• Can we reduce the sub-charts required to run our services?
• Can we implement some kind of monitoring dashboard (Prometheus??)

Summary
• POC takes minutes to deploy
• Reduced complexity and cost
• Flexible deployment location
• Leverages Helm to its full potential
• Opens the door for Docker Registry and Helm repo release
model

13

Exploring a simpler, more portable, less overhead solution to deploy Elasticsearch Pink Metrics and Customizer for IBM Connections

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Exploring a simpler, more portable, less overhead solution to deploy Elasticsearch Pink Metrics and Customizer for IBM Connections

Semelhante a Exploring a simpler, more portable, less overhead solution to deploy Elasticsearch Pink Metrics and Customizer for IBM Connections (20)

Mais de LetsConnect

Mais de LetsConnect (20)

Último

Último (20)

Exploring a simpler, more portable, less overhead solution to deploy Elasticsearch Pink Metrics and Customizer for IBM Connections

Notas do Editor