Using Multi-stage Docker, Go, Java,& Bazel to DESTROY Long Build Times
•Transferir como PPTX, PDF•
1 gostou•910 visualizações
Those long build times are EMBARRASSING! In this EPIC click-batey talk, we'll open up our toolbox to optimize build times down to nothing. Multi-stage Docker will be critical but so will Bazel, Go, and yes, even Java. No matter what kind of environment you're running, you'll find some best practices to speed up your times, scratch that, you'll DESTROY those AWFUL build times with DEVOPS and CI TOOLS.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Using Multi-stage Docker, Go, Java,& Bazel to DESTROY Long Build Times
Semelhante a Using Multi-stage Docker, Go, Java,& Bazel to DESTROY Long Build Times (20)
Mais de DevOps.com
Mais de DevOps.com (20)
Último
Último (20)
Using Multi-stage Docker, Go, Java,& Bazel to DESTROY Long Build Times
- 1. DESTROY Long Build Times using Docker, Go, Java, Bazel & Codefresh DAN GARFIELD GUY SALTON Thanks! KOSTIS KAPELONIS
- 3. Why speed up builds?
- 4. 100 Engineers 5 Builds/Day x Saving 5 Min/Build x Making Average $150k/yr Annual Savings $781k
- 5. Intangible Benefits Happy Devs Faster IterationCreativity Flourishes
- 6. What you’ll learn to day How to strategically use distributed caching Tips for optimizing Docker builds Why you need multi- stage builds
- 7. Codefresh The 1st container-native CI/CD Platform for Microservices Cloud-native Intuitive & Robust Enterprise Ready Flexible Delivery
- 8. CODEFRESH ARCHITECTURE DIAGRAM MULTI CLOUD
- 9. Why do CI systems usually struggle with speed?
- 10. Local Build Isn’t the Answer Blocks Dev from working Not secure or traceable Not reliable
- 11. Why do CI systems usually struggle with speed? Build Node Build Node Build Node Build Node Build Node Build Node Build Node Build Node Build Node
- 12. Why do CI systems usually struggle with speed? Build Node Build Node Build Node Build Node Build Node Build Node Build Node Build Node Build Node Local Volume
- 13. Why do CI systems usually struggle with speed? Build Node Build Node Build Node Build Node Build Node Build Node Build Node Build Node Local Volume Build Node Local Volume Secrets Artifacts Code
- 14. Every step a container Codefresh attaches build volumes CODEFRESH BUILD OPTIMIZATION Step 1 Build Volume Step 2 Step 3 Step 4
- 15. Codefresh Distributed Caching Build Node Build Node Build Node Build Node Build Node Build Node Build Node Build Node CF Volume Build Node CF Volume CF Volume CF Volume CF Volume CF VolumeCF VolumeCF Volume CF Volume CF Volume CF Volume CF Volume
- 16. Demo 1: Using Distributed Cache https://g.codefresh.io/build/5d831fed577e2f81e4569104
- 17. Demo 2: Running Locally $ codefresh run 'Go Speed/Hugo Build' --local
- 18. My code change Rebuilt dependencies 80-90% Build Time is Wasted!
- 19. My code change Only Rebuild the Part that changes ● Cached Docker Layers ● Gradle Cache ● Go Cache ● Bazel
- 20. Go has GOPATH Using ${{CF_VOLUME_PATH}} for your go path with cache the modules. Java has Gradle Cache Using ${{CF_VOLUME_PATH}} for your gradle cache. Bazel has Cache Using ${{CF_VOLUME_PATH}} for your gradle cache. Caching Build Volume
- 21. Demo 3: Using App Cache https://g.codefresh.io/build/5d82fb46c23d4552518b95e3
- 23. Docker multi-stage build ● “oh man! thank you! I've been fighting with build times on an image stack for weeks” ● “Wow, this is great for deploying tonnes of microservices!” ● “I've been meaning to use multistage builds, thanks for the walkthrough!” https://www.reddit.com/r/golang/comments/crkibq/docker_golang_reducing_container_size_using/ Multi-stage build is available starting from Docker 17.05 (released in 2017!) - so why now?
- 24. Dockerfile and Docker build ● Dockerfile - imperative DSL that defines build commands ● Each Docker build command generates ONE image layer ● Complete Docker build execution generates ONE Docker image
- 25. Dockerfile and Docker build
- 26. Demo 5: Docker build on GO app https://github.com/codefresh-contrib/helm-sample-app
- 27. The Problem with Docker build Image we want Image we build runtime configuration application Compilers, debuggers, ... Linters, tests, profilers, ... code, build and test logs, ... runtime configuration application X (4..10)
- 28. The Problem with Docker build ● 2 Dockerfiles ○ 1st for build tools ○ 2nd for runtime ● Drawbacks ○ 2+ Dockerfiles ○ Orchestration needed: Bash, make, YAML, ...
- 29. Solution: Docker multi-stage build ● Benefits ○ One Dockerfile ○ One syntax to learn ○ Same build ■ Local and CI ○ Create multiple stages
- 30. guy.salton@codefresh.i o FROM: alpine:3.5 curl FROM: ubuntu /hello FROM: debian /world FROM: base /hello /world
- 31. Demo 6: Docker multi-stage build
- 32. Docker multi-stage build You can enjoy multi-stage build with every programming language (not only GO): ● GO example - https://codefresh.io/docs/docs/learn-by-example/golang/golang-hello-world/#create-a-multi-stage-docker-image-for-go ● JAVA example - https://codefresh.io/docs/docs/learn-by-example/java/spring-boot-2/#spring-boot-2-and-docker-multi-stage-builds ● Node example - https://codefresh.io/docs/docs/learn-by-example/nodejs/react/#react-and-docker-multi-stage-builds ● PHP example - https://codefresh.io/docs/docs/learn-by-example/php/#the-example-php-project https://codefresh.io/containers/docker-anti-patterns/Docker anti-patterns
- 33. Summary ● Saving just 5 min is worth big $$ ● Distributed Caching is basically free optimization ● Pair with Application Cache
- 34. Summary ● Using 1 Docker image for both build and production results in slow deployment and lots of CVE violations ● Multi-stage build to produce lean, secure and production ready Docker image ● On Codefresh, speedier builds thanks to caching across all images and layers
- 35. Try it free at codefresh.io Questions?
Notas do Editor
- Container-Native: Every step is a docker image. Easy to assemble and maintain pipelines. OOTB steps for common steps - Build images, Ehlm charts etc… Run on kubernetes. Complete Code-to-Cloud traceability Intuitive & Robust: Intuitive UI. Very user friendly Enterprise Ready: RBAC, Roles and Permissions, Can scale to run thousands of pipelines Flexible Delivery: Cloud, Hybrid, On-Premise
- Dynamic build nodes often destroy everything in the workspace. It’s hard to configure!
- Bazel is more aggressive about rebuilding components, Gradle has some support but it’s not perfect. Codefresh supports any build system and works just like your workstation.
- Bazel is more aggressive about rebuilding components,
- Docker can build images by reading the instructions from a Dockerfile. A Dockerfile is a text file that contains a list of all the commands needed to build a new Docker image. The core principle is very simple: 1 Dockerfile -> 1 Docker Image. This principle works just fine for basic use cases, where you just need to demonstrate Docker capabilities or put some “static” content into a Docker image.
- Dockerfiles always begin with the FROM instruction, which sets the base image on which to run all subsequent commands. The base image can be any valid Docker image that has been custom created or pulled from public repositories. Subsequent commands like COPY and RUN define not only what the image is to contain, but more frequently command line steps to derive your final application run state. For example, by using the Golang “SDK” image, a user may copy in their source code and perform a build all inside the Dockerfile, resulting in a convenient single image. Unfortunately, the size of the resulting image frequently ends up ballooning to over 700MB. If your application is deployed using several or more Docker images, this can be a serious problem.
- When we create a new image we want to include the basic runtime, some configuration, and the application itself. However, we often build our image with additional tools/ packages, such as compilers, debuggers, test frameworks, profilers, build artifacts, etc. We end up with an image that’s 4-10x bigger than we need to run our application - find themselves with slow Docker builds, huge Docker images (several GB size images), slow deployment time and lots of CVE violations embedded into these images. The deployment images should contain: The application code in minified/compiled form plus its runtime dependencies. Nothing else. Really nothing else.
- So how can we solve the problem? We could create 2 Dockerfiles - 1 for the build tools and 1 for the runtime but this requires us to write and maintain 2 dockerfiles for the same app and also write some script to manage the orchestration between them
- Docker 17.05 extends Dockerfile syntax to support new multi-stage build, by extending two commands: FROM and COPY. The multi-stage build allows using multiple FROM commands in the same Dockerfile. The last FROM command produces the final Docker image, all other images are intermediate images (no final Docker image is produced, but all layers are cached). The FROM syntax also supports AS keyword. Use AS keyword to give the current image a logical name and reference to it later by this name. To copy files from intermediate images use COPY --from=<image_AS_name|image_number>, where number starts from 0 (but better to use logical name through AS keyword).