Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans

•

0 gostou•440 visualizações

1) The author parallelized unit tests by running them in Docker containers on multiple Jenkins slaves to reduce test runtime from 25 minutes to 3 minutes. 2) Configuration management integration with Docker and challenges around dependencies, layering, and the "PID 1" problem were discussed. 3) Additional topics discussed included trimming container fat, container layering questions, linking containers, maintenance profiles, and open-sourcing projects using Docker.

Tecnologia

Contain(erize)
The Tests
@markeijsermans

Software Engineer
Love DevOps
Love Docker

Our unit test problems
monolithic code base
complex setup
few mocks
hits databases
slow

Jenkins Woes
Jenkins was bottlenecking our server configurations
Php 5.5.x roll-out took months

Parallelize the tests
solved a problem
low risk

Jenkins x Jenkins
run tests on 10 (or more) slaves
goal: Docker and git are the only dependencies

unittest.sh kickoff script
private docker registry
container for running test
linked container for dbs

Results
25min down to 3min (could still be lower)
...but, not all is rosy

Configuration Management
How to integrate Ansbile?
sshd adds +40MB (seems wrong)
installed Ansible in base image
maybe provision with chroot? (but lose layering)
Packer.io? isn't quite there yet
CM integration with Docker rumored
[soon to be fixed]

Why not just use Ansible
directly on slaves?
continuous integration is continually integrating itself!
automatic refresh (pull) before running tests
easy to scale more slaves
slaves re-usable for other tasks - not polluted with dependencies
part of the experiment!

The "PID 1" problem
needs to watch sub-processes if they die and re-launch
can't be SIGKILL'd
some hacks needed to quiet upstart
we used supervisord - works well
init integration coming (systemd, SysV, etc.)
[soon to be fixed]

libcontainer
new in version 0.9
nsinit allows you to attach
might be implemented as "docker exec"
lxc namespacing still has lxc-attach

Registry is slow from the cloud
push/pulling from local network to EC2 is painful
+1GB containers!
how about a local mirror?
EDIT: added to the docker-registry a few weeks ago
[fixed]

Trimming the fat
installing packages dumps a TON of stuff on the system
great way to find out what a package does
dpkg --no-install-recommends
filter out /usr/share/{man | doc | info}
trash apt cache (+100MB)
flatten: docker export -> import
[solvable]

Container layering questions
base -> Ansible -> runtime -> app -> unit test extras

Linking containers
cumbersome.
lots of verbose ENV VARS
ambassador container (hacky)?
service discovery - serf, etcd, etc?
DB_PORT_3306_TCP=tcp://127.0.0.1
DB_PORT_3306_TCP_ADDR=127.0.0.1
DB_PORT_3306_TCP_PORT=3306
DB_PORT_3306_TCP_PROTO=tcp
...
[unknown]

API first design
*except a some commands like attach

Great for open-sourcing
a project
docker run ... done!

Thank You!
@markeijsermans
mark.eijsermans@hootsuite.com

Mais conteúdo relacionado

Mais procurados

Dockercon 16 Wrap-up (Docker for Mac and Win, Docker 1.12, Swarm Mode, etc.)

Nils De Moor

Ansible presentation

Suresh Kumar

Docker is awesome and there's been a lot of excitement over .NET Core running in Linux containers. But why do older apps have to miss out on the fun? With Windows Server 2016 and Windows Server containers, there's finally a way to dockerize .NET 4.6 apps using the same Docker tools and commands as we're used to on Linux. In this intermediate level talk, I'll give an overview of Docker and Windows Server containers. Then I'll demonstrate different ways to run existing ASP.NET Web API, MVC, and even WebForms applications inside Docker containers.

Docker All The Things - ASP.NET 4.x and Windows Server Containers

Anthony Chu

The topic of orchestration comes up a lot when discussing applications. We will cover: How the word "orchestration" has been used to mean so many different things that it is a meaningless term now for shared understanding What I think people actually mean when they say they want orchestration (ordered execution of operations across a fleet) Why this model of thinking is incompatible with the autonomous actor model How we could build systems that get us the outcomes of orchestration without needing to build it explicitly. How this is particularly pertinent in an era of containerization and microservices.

Orchestration? You Don't Need Orchestration. What You Want Is Choreography by...

Docker, Inc.

Hands-off logging for OpenShift in AWS

Amir Moghimi

DockerCon US 2016 - Extending Docker With APIs, Drivers, and Plugins

Arnaud Porterie

Democratizing Development - Scott Gress

Docker, Inc.

Continuous Delivery helps to keep your software and Docker images updated and deploy new versions in production easily. Microservices are great at reducing the attack vector and limiting the privileges or credentials access of each piece of your application. Containers provide an opportunity to implement better security, small, immutable, single process and purpose. In this session, we will discover real use case examples on how to make your CI/CD pipeline interact with Docker security tools. But security doesn’t stop where your deployment pipeline ends. How can we prepare for 0-days and policy violations that happen at run-time? Can we make it part of the CI/CD process?

CI / CD / CS - Continuous Security in Kubernetes

Sysdig

Dockerize Me: Distributed PHP applications with Symfony, Docker, Consul and A...

Alexey Petrov

Responsible disclosure is a key ingredient of any solid security strategy. In this session, Docker maintainer Michael Crosby will explain the ins and outs of CVE-2016-9962: how it was discovered, how it could even happen in the first place, and how it was addressed. A vertiginous abseil at the boundaries of the kernel, in the fascinating land of system calls and randomized address space. You will think twice before leaking a file descriptor again.

Securing Containers, One Patch at a Time - Michael Crosby, Docker

Docker, Inc.

SQL Sever on Docker: Database Containers 3 Ways

Elton Stoneman

Ansible container

Scott van Kalken

Dockerfiles are great. They provide a zero-barrier-to-entry format for describing a single Docker image which is immediately clear to anyone reading them. But with that simplicity comes problems that become apparent as your adoption of Docker gathers pace. * Dockerfiles can inherit from other docker images, but images are not Dockerfiles * Dockerfile provides no built-in mechanism for creating abstractions, so as usage grows identical or similar instructions can be duplicated across many files * The Docker APi exposes a build endpoint, but the API is very course, taking Dockerfile as the transport rather than exposing the individual instructions * Dockerfiles are just that, files. So they can come from anywhere The one layer per line in a Dockerfile limitation can lead to an explosion of layers, which fail to take advantage of the promised space and performance benefits.

The Dockerfile Explosion and the Need for Higher Level Tools by Gareth Rushgrove

Docker, Inc.

In this presentation Carlos Sanchez will share his experience running Jenkins at scale, using Docker and Apache Mesos to create one of the biggest (if not the biggest) Jenkins clusters to date. By taking advantage of Apache Mesos, the Jenkins platform is dynamically scaled to run jobs across hundreds of Jenkins masters, on Docker containers distributed across the Mesos cluster. Jenkins slaves are dynamically created based on load, using the Jenkins Mesos and Docker plugins, running in containers distributed across multiple hosts, and isolating job execution. This presentation will allow a better understanding of Apache Mesos and the challenges of running Docker containerized and distributed applications, particularly JVM ones, by sharing a real world use case, including good and bad decisions and how they affected the development.

CI and CD at Scale: Scaling Jenkins with Docker and Apache Mesos

Carlos Sanchez

Dockerize your Symfony application - Symfony Live NYC 2014

André Rømcke

Erlend Oftedal, Blank Immutable infrastructure and serverless architectures have very interesting security properties. This talk will give an introduction to immutable infrastructure and serverless architecture and try to highlight some of the properties of such architectures. Next we will look at the positive effects this can have on the security of our systems, but also highlight some of the negative aspects and potential problems. At the conclusion of this sessions, we hope to have shed some light on the positive and negative security effects of such architectures.

There is No Server: Immutable Infrastructure and Serverless Architecture

Sonatype

Docker leverages capabilities in Linux like namespaces and cgroups to enable containers and then builds tooling on top to enable users to build distributed apps. A common question is "What about Docker support for Windows?" In this session the Windows engineering leads will dive deep into the primitives within Windows to enable an awesome Docker experience on Windows. This session will also include a live demo of Docker and Windows Server.

Windows Server and Docker - The Internals Behind Bringing Docker and Containe...

Docker, Inc.

Continuous Delivery With Selenium Grid And Docker

Barbara Gonzalez

CI and CD at Scale: Scaling Jenkins with Docker and Apache Mesos

Carlos Sanchez

Hooking Docker With Selenium

Sujith Vakathanam

Mais procurados (20)

Dockercon 16 Wrap-up (Docker for Mac and Win, Docker 1.12, Swarm Mode, etc.)

Ansible presentation

Docker All The Things - ASP.NET 4.x and Windows Server Containers

Orchestration? You Don't Need Orchestration. What You Want Is Choreography by...

Hands-off logging for OpenShift in AWS

DockerCon US 2016 - Extending Docker With APIs, Drivers, and Plugins

Democratizing Development - Scott Gress

CI / CD / CS - Continuous Security in Kubernetes

Dockerize Me: Distributed PHP applications with Symfony, Docker, Consul and A...

Securing Containers, One Patch at a Time - Michael Crosby, Docker

SQL Sever on Docker: Database Containers 3 Ways

Ansible container

The Dockerfile Explosion and the Need for Higher Level Tools by Gareth Rushgrove

CI and CD at Scale: Scaling Jenkins with Docker and Apache Mesos

Dockerize your Symfony application - Symfony Live NYC 2014

There is No Server: Immutable Infrastructure and Serverless Architecture

Windows Server and Docker - The Internals Behind Bringing Docker and Containe...

Continuous Delivery With Selenium Grid And Docker

CI and CD at Scale: Scaling Jenkins with Docker and Apache Mesos

Hooking Docker With Selenium

Destaque

Docker-Vancouver Meetup - March 18, 2014 - An Introduction to Docker

bacongobbler

Introduction to Containers and Docker

Rob Loach

Abstract— The Media and Entertainment industry is in the midst of rapid evolution as movie studios and broadcast television productions move away from traditional analog formats to digital. Rapid advancements in display resolution and file formats such as 4k, Ultra High Definition, and High Frame Rate filming are placing additional stress on I.T. systems and increasing the demand for storage and network bandwidth. While many media organizations are already leveraging Internet technologies in their day -today operations, this paper will present concepts to improve the media creation process, creative collaboration, efficiency, and security via traditional network technologies, as well as newer advancements in cloud computing and software defined networking.

Media and Entertainment Network Exchange Concept

Jason Banks

Docker with BGP - OpenDNS

bacongobbler

Network Function Virtualization - Telkomsel Perspective (SDN NFV Day ITB 2016)

SDNRG ITB

Cisco SDN/NVF Innovations (SDN NVF Day ITB 2016)

SDNRG ITB

Destaque (6)

Docker-Vancouver Meetup - March 18, 2014 - An Introduction to Docker

Introduction to Containers and Docker

Media and Entertainment Network Exchange Concept

Docker with BGP - OpenDNS

Network Function Virtualization - Telkomsel Perspective (SDN NFV Day ITB 2016)

Cisco SDN/NVF Innovations (SDN NVF Day ITB 2016)

Semelhante a Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans

In less than two years Docker went from first line of code to major Open Source project with contributions from all the big names in IT. Everyone is excited, but what's in for me - as a Dev or Ops? In short, Docker makes creating Development, Test and even Production environments an order of magnitude simpler, faster and completely portable across both local and cloud infrastructure. We will start from Docker main concepts: how to create a Linux Container from base images, run your application in it, and version your runtimes as you would with source code, and finish with a concrete example.

Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...

Codemotion

Docker Online Meetup #3: Docker in Production

Docker, Inc.

Docker is an Open Source engine to build, run, and manage Linux Containers. Containers use less resources than virtual machines, they boot faster, but they have similar guarantees of portability and repeatability for Linux applications. Those features made Docker and Linux Containers extremely popular for development and testing environments. But what does it take to use Docker and Containers for production workloads?

Shipping Applications to Production in Containers with Docker

Jérôme Petazzoni

codemotion-docker-2014

Carlo Bonamico

Versioning an API can be a somewhat daunting task for the uninitiated. Even worse, some of the most common approaches are less than ideal. In this session I discuss the struggles and outcomes of my first foray into versioning and deploying. I will show how using a combination immutable docker containers, nginx, and a few other friendly tools made for the creation of a fully automated versioning and deployment system at the push of a button.

Api versioning w_docker_and_nginx

Lee Wilkins

The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration

Erica Windisch

Docker intro

Frei Zhang

IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...

Eric Smalling

Docker - Lightweight Virtualization

Mehdi Hasan

BDM32: AdamCloud Project - Part II

David Lauzon

Austin Web Architecture

joaquincasares

One of the upsides of Microservices is the ability to deploy often,at arbitrary schedules, and independently of other services, instead of requiring synchronized deployments happening on a fixed time. But to really leverage this advantage, we need fast, efficient, and reliable deployment processes. That's one of the value propositions of Containers in general, and Docker in particular. Docker offers a new, lightweight approach to application portability.It can build applications using easy-to-write, repeatable, efficient recipes; then it can ship them across environments using a common container format; and it can run them within isolated namespaces which abstract the operating environment, independently of the distribution,versions, network setup, and other details of this environment. But Docker can do way more than deploy your apps. Docker also enables you to generalize Microservices principles and apply them on operational tasks like logging, remote access, backups, and troubleshooting.This decoupling results in independent, smaller, simpler moving parts.

Containers, Docker, and Microservices: the Terrific Trio

Jérôme Petazzoni

Linux containers & Devops

Maciej Lasyk

Unikernels - Bristech June 2016

Daniel Drozdzewski

Docker: do's and don'ts

Paolo Tonin

Api Versioning with Docker and Nginx

tech.kartenmacherei

Continuous integration and deployment with docker

pebble {code}

Docker Ecosystem on Azure

Patrick Chanezon

Ansible+docker (highload++2015)

Pavel Alexeev

Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more. In this demo, I will show how to build a Apache image from a Dockerfile and deploy a PHP application which is present in an external folder using custom configuration files.

Docker - Demo on PHP Application deployment

Arun prasath

Semelhante a Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans (20)

Why everyone is excited about Docker (and you should too...) - Carlo Bonamic...

Docker Online Meetup #3: Docker in Production

Shipping Applications to Production in Containers with Docker

codemotion-docker-2014

Api versioning w_docker_and_nginx

The Docker "Gauntlet" - Introduction, Ecosystem, Deployment, Orchestration

Docker intro

IBM Index 2018 Conference Workshop: Modernizing Traditional Java App's with D...

Docker - Lightweight Virtualization

BDM32: AdamCloud Project - Part II

Austin Web Architecture

Containers, Docker, and Microservices: the Terrific Trio

Linux containers & Devops

Unikernels - Bristech June 2016

Docker: do's and don'ts

Api Versioning with Docker and Nginx

Continuous integration and deployment with docker

Docker Ecosystem on Azure

Ansible+docker (highload++2015)

Docker - Demo on PHP Application deployment

Último

CNv6 Instructor Chapter 6 Quality of Service

giselly40

MySQL Webinar, presented on the 25th of April, 2024. Summary: MySQL solutions enable the deployment of diverse Database Architectures tailored to specific needs, including High Availability, Disaster Recovery, and Read Scale-Out. With MySQL Shell's AdminAPI, administrators can seamlessly set up, manage, and monitor these solutions, ensuring efficiency and ease of use in their administration. MySQL Router, on the other hand, provides transparent routing from the application traffic to the backend servers in the architectures, requiring minimal configuration. Completely built in-house and supported by Oracle, these solutions have been adopted by enterprises of all sizes for their business-critical applications. In this presentation, we'll delve into various database architecture solutions to help you choose the right one based on your business requirements. Focusing on technical details and the latest features to maximize the potential of these solutions.

Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...

Miguel Araújo

GenCyber Cyber Security Day Presentation

Michael W. Hawkins

Axa Assurance Maroc - Insurer Innovation Award 2024

The Digital Insurer

Building Digital Trust in a Digital Economy Veronica Tan, Director - Cyber Security Agency of Singapore Apidays Singapore 2024: Connecting Customers, Business and Technology (April 17 & 18, 2024) ------ Check out our conferences at https://www.apidays.global/ Do you want to sponsor or talk at one of our conferences? https://apidays.typeform.com/to/ILJeAaV8 Learn more on APIscene, the global media made by the community for the community: https://www.apiscene.io Explore the API ecosystem with the API Landscape: https://apilandscape.apiscene.io/

Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...

apidays

🐬 The future of MySQL is Postgres 🐘

RTylerCroy

Explore 'The Codex of Business: Writing Software for Real-World Solutions,' a compelling SlideShare presentation that delves into digital transformation in healthcare. Discover through a detailed case study how Agile methodologies empower healthcare providers to develop, iterate, and refine digital solutions that address real-world challenges. Learn how strategic planning, user feedback, and continuous improvement drive success in deploying technologies that enhance patient care and operational efficiency. Ideal for healthcare professionals, IT specialists, and digital transformation advocates seeking actionable insights and practical examples of technology making a real difference.

The Codex of Business Writing Software for Real-World Solutions 2.pptx

Malak Abu Hammad

Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024

The Digital Insurer

Imagine a world where information flows as swiftly as thought itself, making decision-making as fluid as the data driving it. Every moment is critical, and the right tools can significantly boost your organization’s performance. The power of real-time data automation through FME can turn this vision into reality. Aimed at professionals eager to leverage real-time data for enhanced decision-making and efficiency, this webinar will cover the essentials of real-time data and its significance. We’ll explore: FME’s role in real-time event processing, from data intake and analysis to transformation and reporting An overview of leveraging streams vs. automations FME’s impact across various industries highlighted by real-life case studies Live demonstrations on setting up FME workflows for real-time data Practical advice on getting started, best practices, and tips for effective implementation Join us to enhance your skills in real-time data automation with FME, and take your operational capabilities to the next level.

From Event to Action: Accelerate Your Decision Making with Real-Time Automation

Safe Software

Discord is a free app offering voice, video, and text chat functionalities, primarily catering to the gaming community. It serves as a hub for users to create and join servers tailored to their interests. Discord’s ecosystem comprises servers, each functioning as a distinct online community with its own channels dedicated to specific topics or activities. Users can engage in text-based discussions, voice calls, or video chats within these channels. Understanding Discord Servers Discord servers are virtual spaces where users congregate to interact, share content, and build communities. Servers may revolve around gaming, hobbies, interests, or fandoms, providing a platform for like-minded individuals to connect. Communication Features Discord offers a range of communication tools, including text channels for messaging, voice channels for real-time audio conversations, and video channels for face-to-face interactions. These features facilitate seamless communication and collaboration. What Does NSFW Mean? The acronym NSFW stands for “Not Safe For Work,” indicating content that may be inappropriate for professional or public settings. NSFW Content NSFW content encompasses material that is sexually explicit, violent, or otherwise graphic in nature. It often includes nudity, profanity, or depictions of sensitive topics.

Understanding Discord NSFW Servers A Guide for Responsible Users.pdf

UK Journal

Abhishek Deb(1), Mr Abdul Kalam(2) M. Des (UX) , School of Design, DIT University , Dehradun. This paper explores the future potential of AI-enabled smartphone processors, aiming to investigate the advancements, capabilities, and implications of integrating artificial intelligence (AI) into smartphone technology. The research study goals consist of evaluating the development of AI in mobile phone processors, analyzing the existing state as well as abilities of AI-enabled cpus determining future patterns as well as chances together with reviewing obstacles as well as factors to consider for more growth.

Exploring the Future Potential of AI-Enabled Smartphone Processors

debabhi2

Automating Google Workspace (GWS) & more with Apps Script

wesley chun

Heather Hedden, Senior Consultant at Enterprise Knowledge, presented “The Role of Taxonomy and Ontology in Semantic Layers” at a webinar hosted by Progress Semaphore on April 16, 2024. Taxonomies at their core enable effective tagging and retrieval of content, and combined with ontologies they extend to the management and understanding of related data. There are even greater benefits of taxonomies and ontologies to enhance your enterprise information architecture when applying them to a semantic layer. A survey by DBP-Institute found that enterprises using a semantic layer see their business outcomes improve by four times, while reducing their data and analytics costs. Extending taxonomies to a semantic layer can be a game-changing solution, allowing you to connect information silos, alleviate knowledge gaps, and derive new insights. Hedden, who specializes in taxonomy design and implementation, presented how the value of taxonomies shouldn’t reside in silos but be integrated with ontologies into a semantic layer. Learn about: - The essence and purpose of taxonomies and ontologies in information and knowledge management; - Advantages of semantic layers leveraging organizational taxonomies; and - Components and approaches to creating a semantic layer, including the integration of taxonomies and ontologies

The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf

Enterprise Knowledge

Choosing the right accounts payable services provider is a strategic decision that can significantly impact your business's financial performance and operational efficiency. By considering factors such as expertise, range of services, technology infrastructure, scalability, cost, and reputation, businesses can make informed decisions and select a provider that aligns with their unique needs and objectives. Partnering with the right provider can streamline accounts payable processes, drive cost savings, and position your business for long-term success. https://katprotech.com/accounts-payable-and-purchase-order-automation/

Factors to Consider When Choosing Accounts Payable Services Providers.pptx

Katpro Technologies

In an era where artificial intelligence (AI) stands at the forefront of business innovation, Information Architecture (IA) is at the core of functionality. See “There’s No AI Without IA” – (from 2016 but even more relevant today) Understanding and leveraging how Information Architecture (IA) supports AI synergies between knowledge engineering and prompt engineering is critical for senior leaders looking to successfully deploy AI for internal and externally facing knowledge processes. This webinar be a high-level overview of the methodologies that can elevate AI-driven knowledge processes supporting both employees and customers. Core Insights Include: Strategic Knowledge Engineering: Delve into how structuring AI's knowledge base is required to prevent hallucinations, enable contextual retrieval of accurate information. This will include discussion of gold standard libraries of use cases support testing various LLMs and structures and configurations of knowledge base. Precision in Prompt Engineering: Learn the art of crafting prompts that direct AI to deliver targeted, relevant responses, thereby optimizing customer experiences and business outcomes. Unified Approach for Enhanced AI Performance: Explore the intersection of knowledge and prompt engineering to develop AI systems that are not only more responsive but also aligned with overarching business strategies. Guiding Principles for Implementation: Equip yourself with best practices, ethical guidelines, and strategic considerations for embedding these technologies into your business ecosystem effectively. This webinar is designed to empower business and technology leaders with the knowledge to harness the full potential of AI, ensuring their organizations not only keep pace with digital transformation but lead the charge. Join us to map a roadmap to fully leverage Information Architecture (IA) and AI chart a course towards a future where AI is a key pillar of strategic innovation and business success.

EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx

Earley Information Science

A Domino Admins Adventures (Engage 2024)

Gabriella Davis

The Raspberry Pi 5 was announced on October 2023. This new version of the popular embedded device comes with a new iteration of Broadcom’s VideoCore GPU platform, and was released with a fully open source driver stack, developed by Igalia. The presentation will discuss some of the major changes required to support this new Video Core iteration, the challenges we faced in the process and the solutions we provided in order to deliver conformant OpenGL ES and Vulkan drivers. The talk will also cover the next steps for the open source Raspberry Pi 5 graphics stack. (c) Embedded Open Source Summit 2024 April 16-18, 2024 Seattle, Washington (US) https://events.linuxfoundation.org/embedded-open-source-summit/ https://eoss24.sched.com/event/1aBEx

Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...

Igalia

Data Cloud, More than a CDP by Matt Robison

Anna Loughnan Colquhoun

How to Troubleshoot Apps for the Modern Connected Worker

ThousandEyes

2024: Domino Containers - The Next Step. News from the Domino Container commu...

Martijn de Jong

Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans

1. Contain(erize) The Tests @markeijsermans

2. Software Engineer Love DevOps Love Docker

3. When you're a hammer...

4. Our unit test problems monolithic code base complex setup few mocks hits databases slow

5. Jenkins Woes Jenkins was bottlenecking our server configurations Php 5.5.x roll-out took months

6. ...the nail

7. Parallelize the tests solved a problem low risk

8. Jenkins x Jenkins run tests on 10 (or more) slaves goal: Docker and git are the only dependencies

9. unittest.sh kickoff script private docker registry container for running test linked container for dbs

10. Results 25min down to 3min (could still be lower) ...but, not all is rosy

11. Findings

12. Configuration Management How to integrate Ansbile? sshd adds +40MB (seems wrong) installed Ansible in base image maybe provision with chroot? (but lose layering) Packer.io? isn't quite there yet CM integration with Docker rumored [soon to be fixed]

13. Why not just use Ansible directly on slaves? continuous integration is continually integrating itself! automatic refresh (pull) before running tests easy to scale more slaves slaves re-usable for other tasks - not polluted with dependencies part of the experiment!

14. The "PID 1" problem needs to watch sub-processes if they die and re-launch can't be SIGKILL'd some hacks needed to quiet upstart we used supervisord - works well init integration coming (systemd, SysV, etc.) [soon to be fixed]

15. libcontainer new in version 0.9 nsinit allows you to attach might be implemented as "docker exec" lxc namespacing still has lxc-attach

16. Registry is slow from the cloud push/pulling from local network to EC2 is painful +1GB containers! how about a local mirror? EDIT: added to the docker-registry a few weeks ago [fixed]

17. Trimming the fat installing packages dumps a TON of stuff on the system great way to find out what a package does dpkg --no-install-recommends filter out /usr/share/{man | doc | info} trash apt cache (+100MB) flatten: docker export -> import [solvable]

18. Container layering questions base -> Ansible -> runtime -> app -> unit test extras

19. Linking containers cumbersome. lots of verbose ENV VARS ambassador container (hacky)? service discovery - serf, etcd, etc? DB_PORT_3306_TCP=tcp://127.0.0.1 DB_PORT_3306_TCP_ADDR=127.0.0.1 DB_PORT_3306_TCP_PORT=3306 DB_PORT_3306_TCP_PROTO=tcp ... [unknown]

20. Unknown maintenance profile not all *nix tools ready for namespacing free & top are broken (shows full system memory allocation) logs (on aufs, inside the container - good/bad?) problem? copy container locally, inspect?

21. API first design *except a some commands like attach

22. Great for open-sourcing a project docker run ... done!

23. Thank You! @markeijsermans mark.eijsermans@hootsuite.com

Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (6)

Semelhante a Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans

Semelhante a Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans (20)

Último

Último (20)

Docker-Vancouver Meetup - March 18, 2014 - Contain(erize) the tests - Mark Eijsermans