O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Stop Being Lazy and Test Your Software!
Laura Frank
Software Engineer, Codeship
0.7.0
ImageLayers
Panamax
Berlin
Agenda
Continuous Delivery
Why Do We Test?
Testing with Docker Compose
Faster Testing with Docker
Why Do We Test?
Motivations and frustrations
Testing software has been
a practice since the very
first machines were used
for computing.
The women who
programmed the ENIAC to
calculate trajectories
periodically checked the
results with a correct,
hand-compute...
working software
in production
testing
code reviews
version control
pair programming
high-availability architecture
Motivators
Update application without breaking things!
Validate functionality of updates
Be able to trust deployment check...
Why do you skip tests?
45%
15%
5%
35% Too slow to run suite 35%
Annoying extra step 15%
Don’t see the point 5%
Slows down ...
Frustrators
It takes me an hour to write a single test
My new tests just duplicate existing coverages so there’s no
point ...
Frustrators
It takes me an hour to write a single test
My new tests just duplicate existing coverages so there’s no
point ...
Using Docker can alleviate
some frustrations
associated with testing.
Testing with Docker
Compose
It’s easy, I promise
Docker and testing are a great pair.
❌ ✔
With Docker Compose, it is incredibly
easy to create consistent,
reproducible environments.
❌ ✔
Many of us already use
Docker Compose to set
up dev environments.
…But we stop there.
Use your Docker Compose
environment for testing
as well.
app:
build: .
command: rails server -p 3000 -b '0.0.0.0'
volumes:
- .:/app
ports:
- '3000:3000'
links:
- postgres
postgres...
?!
git clone && docker-compose up
hackity hack
How do I make tests happen?
Development workflow
app:
build: .
command: rails server -p 3000 -b '0.0.0.0'
volumes:
- .:/app
ports:
- '3000:3000'
links:
- postgres
postgres...
app:
build: .
command: rspec spec
volumes:
- .:/app
ports:
- '3000:3000'
links:
- postgres
postgres:
image: postgres:9.4
A...
In most cases, we need to
do a bit of setup
before running tests.
You might even need a
different Dockerfile.
Docker Compose makes this easy
app:
build: .
dockerfile: Dockerfile.test
command: rake test
volumes:
- .:/app
🚨 A word of warning! 🚨
🚨 Race conditions! 🚨
You can also run one-off
commands against a service
with Docker Compose.
docker-compose run -e "RAILS_ENV=test" 
app rake db:setup
docker-compose run -e "RAILS_ENV=test" 
app test-command path/to...
Additional Docker Compose Testing
Config Options
environment:
RACK_ENV: test
Additional Docker Compose Testing
Config Options
$ docker-compose up -d
$ ./run_tests
$ docker-compose stop
$ docker-compo...
Docker delivers a
predictable, reproducible
testing environment. Period.
Continuous Integration,
Testing, and Docker
Fail fast and not in production!
Continuous integration and
testing go hand in hand.
👯
Relying on CI/CD without
adequate test coverage
is not a great idea.
Would you rather…
✔
💩
Find out your code is broken by
seeing a failed run of your CI/CD
system?
Find out your code is brok...
What about running tests
like this…
…inside a container?
We run our unit tests in a
conatiner during development,
so we should do that during
deployment too, right?
And if we’re running our
services in containers
during development,
they should be running
in containers in
production, ri...
🚨 A word of warning! 🚨
—JérômePetazzoni
“Docker-in-Docker is not
100% made of sparkles,
ponies, and unicorns.
46
Docker in Docker
CAUTION!
YMMV
See https://jpetazzo.github.io
- Mixing file systems == bad time
- Shared build cache == bad time
- Lots of ...
Shared daemon – what we really want
We get this by binding the
Docker socket.
Parallel Testing with
Docker
Break it up.
Why do you skip tests?
45%
15%
5%
35% Too slow to run suite 35%
Annoying extra step 15%
Don’t see the point 5%
Slows down ...
When developing, it’s easy
to think of a container as
a small VM that runs a
specific workload.
✔
But if we change our thinking
to treat containers as just
processes, then we can do
some pretty cool stuff.
✔
A syntax similar to Docker Compose for
service definition…
57
web:
build:
image: my_app
dockerfile_path: Dockerfile
links:...
And use it to also define testing steps…
58
- type: parallel
steps:
- service: demo
command: ruby check.rb
- service: demo...
- type: parallel
steps:
- service: demo
command: some-test-command
- service: demo
command: another-test-command
- service...
- service: demo
command: some-test-command
- service: demo
command: another-test-command
- service: demo
command: yet-anot...
Test compose context
Test runner
jet
test-1
test-0
test-3
Docker makes this possible by
providing the means to create
a predictable, reproducible
testing environment.
Super Cool Tip™
Because quality is everyone’s problem.
Add an additional
pipeline to fail builds if
quality decreases.
Two good examples are
code coverate and
linting errors.
#!/bin/bash
set –e
ALLOWED_WARNINGS=100 #some arbitrary threshold
warnings=`grep "offenses detected" rubocop.txt | cut -d ...
✔
✔
❌
❌
—Solomon(moreorless)
“Improving quality is a
lot of unglamourous
work that really adds up.”
68
Resources
#keepshipping
Highly recommended talks about development, testing,
and lots of interesting stuff: https://speakerdeck.com/searls
Ruby ge...
—EdsgerDijkstra
“Testing can be a very
effective way to show the
presence of bugs, but is
hopelessly inadequate for
showin...
Testing does not
guarantee that
our software works.
We test to know
when our software is
definitely broken.
Work harder to know
when you’re wrong.
Thank you!
Laura Frank
@rhein_wein
laura@codeship.com
Stop Being Lazy and Test Your Software
Stop Being Lazy and Test Your Software
Stop Being Lazy and Test Your Software
Stop Being Lazy and Test Your Software
Stop Being Lazy and Test Your Software
Próximos SlideShares
Carregando em…5
×

Stop Being Lazy and Test Your Software

14.534 visualizações

Publicada em

DockerCon EU 2015 in Barcelona

Practical tips for using Docker to run tests during development, CI/CD, and different strategies for speeding up the run of your test suite by using parallel pipelines in containers.

The jet tool used to demonstrate parallel testing is available here: https://codeship.com/documentation/docker/installation/

Publicada em: Tecnologia
  • Seja o primeiro a comentar

Stop Being Lazy and Test Your Software

  1. 1. Stop Being Lazy and Test Your Software! Laura Frank Software Engineer, Codeship
  2. 2. 0.7.0 ImageLayers Panamax Berlin
  3. 3. Agenda Continuous Delivery Why Do We Test? Testing with Docker Compose Faster Testing with Docker
  4. 4. Why Do We Test? Motivations and frustrations
  5. 5. Testing software has been a practice since the very first machines were used for computing.
  6. 6. The women who programmed the ENIAC to calculate trajectories periodically checked the results with a correct, hand-computed table.
  7. 7. working software in production testing code reviews version control pair programming high-availability architecture
  8. 8. Motivators Update application without breaking things! Validate functionality of updates Be able to trust deployment checks (in CI/CD workflow) Confirm that refactoring didn’t break existing functionality
  9. 9. Why do you skip tests? 45% 15% 5% 35% Too slow to run suite 35% Annoying extra step 15% Don’t see the point 5% Slows down deployment 45% 10
  10. 10. Frustrators It takes me an hour to write a single test My new tests just duplicate existing coverages so there’s no point (integration overlapping with unit) My suite fails all the time for no apparent reason…? It takes my test suite over 20 minutes to run, so I don’t run it locally Testing distracts me from my normal development workflow Setting up a testing environment is a huge pain It takes too long to deploy when I have a huge test suite
  11. 11. Frustrators It takes me an hour to write a single test My new tests just duplicate existing coverages so there’s no point (integration overlapping with unit) My suite fails all the time for no apparent reason…? It takes my test suite over 20 minutes to run, so I don’t run it locally Testing distracts me from my normal development workflow Setting up a testing environment is a huge pain It takes too long to deploy when I have a huge test suite
  12. 12. Using Docker can alleviate some frustrations associated with testing.
  13. 13. Testing with Docker Compose It’s easy, I promise
  14. 14. Docker and testing are a great pair. ❌ ✔
  15. 15. With Docker Compose, it is incredibly easy to create consistent, reproducible environments. ❌ ✔
  16. 16. Many of us already use Docker Compose to set up dev environments.
  17. 17. …But we stop there.
  18. 18. Use your Docker Compose environment for testing as well.
  19. 19. app: build: . command: rails server -p 3000 -b '0.0.0.0' volumes: - .:/app ports: - '3000:3000' links: - postgres postgres: image: postgres:9.4 A simple Rails app
  20. 20. ?! git clone && docker-compose up hackity hack How do I make tests happen? Development workflow
  21. 21. app: build: . command: rails server -p 3000 -b '0.0.0.0' volumes: - .:/app ports: - '3000:3000' links: - postgres postgres: image: postgres:9.4 A simple Rails app
  22. 22. app: build: . command: rspec spec volumes: - .:/app ports: - '3000:3000' links: - postgres postgres: image: postgres:9.4 A simple Rails app
  23. 23. In most cases, we need to do a bit of setup before running tests.
  24. 24. You might even need a different Dockerfile.
  25. 25. Docker Compose makes this easy app: build: . dockerfile: Dockerfile.test command: rake test volumes: - .:/app
  26. 26. 🚨 A word of warning! 🚨
  27. 27. 🚨 Race conditions! 🚨
  28. 28. You can also run one-off commands against a service with Docker Compose.
  29. 29. docker-compose run -e "RAILS_ENV=test" app rake db:setup docker-compose run -e "RAILS_ENV=test" app test-command path/to/spec.rb docker-compose up #-d if you wish Usual Docker Compose development environment Run rake task to set up db Then run tests against Docker services
  30. 30. Additional Docker Compose Testing Config Options environment: RACK_ENV: test
  31. 31. Additional Docker Compose Testing Config Options $ docker-compose up -d $ ./run_tests $ docker-compose stop $ docker-compose rm –f
  32. 32. Docker delivers a predictable, reproducible testing environment. Period.
  33. 33. Continuous Integration, Testing, and Docker Fail fast and not in production!
  34. 34. Continuous integration and testing go hand in hand. 👯
  35. 35. Relying on CI/CD without adequate test coverage is not a great idea.
  36. 36. Would you rather… ✔ 💩 Find out your code is broken by seeing a failed run of your CI/CD system? Find out your code is broken by seeing 500s on 500s on 500s in production?
  37. 37. What about running tests like this… …inside a container?
  38. 38. We run our unit tests in a conatiner during development, so we should do that during deployment too, right?
  39. 39. And if we’re running our services in containers during development, they should be running in containers in production, right?
  40. 40. 🚨 A word of warning! 🚨
  41. 41. —JérômePetazzoni “Docker-in-Docker is not 100% made of sparkles, ponies, and unicorns. 46
  42. 42. Docker in Docker
  43. 43. CAUTION! YMMV See https://jpetazzo.github.io - Mixing file systems == bad time - Shared build cache == bad time - Lots of other stuff == bad time
  44. 44. Shared daemon – what we really want
  45. 45. We get this by binding the Docker socket.
  46. 46. Parallel Testing with Docker Break it up.
  47. 47. Why do you skip tests? 45% 15% 5% 35% Too slow to run suite 35% Annoying extra step 15% Don’t see the point 5% Slows down deployment 45% 52
  48. 48. When developing, it’s easy to think of a container as a small VM that runs a specific workload.
  49. 49.
  50. 50. But if we change our thinking to treat containers as just processes, then we can do some pretty cool stuff.
  51. 51.
  52. 52. A syntax similar to Docker Compose for service definition… 57 web: build: image: my_app dockerfile_path: Dockerfile links: - redis - postgres redis: image: redis postgres: image: postgres
  53. 53. And use it to also define testing steps… 58 - type: parallel steps: - service: demo command: ruby check.rb - service: demo command: rake teaspoon suite=jasmine - service: demo command: rake test
  54. 54. - type: parallel steps: - service: demo command: some-test-command - service: demo command: another-test-command - service: demo command: yet-another-test-command Each step is run independently in a container 59
  55. 55. - service: demo command: some-test-command - service: demo command: another-test-command - service: demo command: yet-another-test-command And each container may be located in a range of availability zones 60
  56. 56. Test compose context Test runner jet test-1 test-0 test-3
  57. 57. Docker makes this possible by providing the means to create a predictable, reproducible testing environment.
  58. 58. Super Cool Tip™ Because quality is everyone’s problem.
  59. 59. Add an additional pipeline to fail builds if quality decreases.
  60. 60. Two good examples are code coverate and linting errors.
  61. 61. #!/bin/bash set –e ALLOWED_WARNINGS=100 #some arbitrary threshold warnings=`grep "offenses detected" rubocop.txt | cut -d " " –f4` if [ $warnings -gt $ALLOWED_WARNINGS ] then echo -e "033[31mallowed warnings $ALLOWED_WARNINGS033[0m" echo -e "033[31mactual warnings $warnings033[0m" echo -e "033[31mToo many rubocop warnings033[0m" echo -e "033[31mTry running 'bin/check_rubocop_against_master’033[0m" exit 1 else echo $warnings/$ALLOWED_WARNINGS is close enough ಠ_ಠ exit 0 fi
  62. 62. ✔ ✔ ❌ ❌
  63. 63. —Solomon(moreorless) “Improving quality is a lot of unglamourous work that really adds up.” 68
  64. 64. Resources #keepshipping
  65. 65. Highly recommended talks about development, testing, and lots of interesting stuff: https://speakerdeck.com/searls Ruby gem for parallel tests: grosser/parallel_tests Parallel Docker testing: Jet (from Codeship) CI/CD with Docker: http://pages.codeship.com/docker Running commands with Docker Compose: http://docs.docker.com/compose The perils of Docker-In-Docker: https://jpetazzo.github.io This talk: slideshare.net/rheinwein
  66. 66. —EdsgerDijkstra “Testing can be a very effective way to show the presence of bugs, but is hopelessly inadequate for showing their absence.” 71
  67. 67. Testing does not guarantee that our software works.
  68. 68. We test to know when our software is definitely broken.
  69. 69. Work harder to know when you’re wrong.
  70. 70. Thank you! Laura Frank @rhein_wein laura@codeship.com

×