O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
@burythehammer
MATT LONG
TESTING PROGRAMMABLE
INFRASTRUCTURE (WITH RUBY)
InfoQ.com: News & Community Site
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentati...
Purpose of QCon
- to empower software development by facilitating the spread of
knowledge and innovation
Strategy
- practi...
@burythehammer
PROGRAMMABLE
INFRASTRUCTURE IS GREAT, BUT
WE'RE MISSING SOMETHING.
TESTING.
@burythehammer
I'M A TESTER
HELLO, I'M MATT
@burythehammer
@burythehammer
I WORK HERE ↑
@burythehammer
I AM NOT A
SYSADMIN
@burythehammer
WHAT IS
PROGRAMMABLE
INFRASTRUCTURE?
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
THE APPLICATION OF METHODS AND TOOLING
FROM SOFTWARE DEVELOPMENT TO THE...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
WHAT DO WE MEAN BY THIS?
▸ Automated provisioning & configuration
▸ Confi...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
TOOLING EXAMPLES
@burythehammer
PROGRAMMABLE
INFRASTRUCTURE
IS AWESOME!
Credit: Vault Boy, Bethesda Softworks
@burythehammer
BUT IT CAN GET
COMPLEX
@burythehammer
TESTING IS USED TO
MITIGATE COMPLEXITY
& RISK
@burythehammer
BUT INFRA TESTING IS RARE
Credit: Gunshow, KC Green
@burythehammer
TESTING A
CLOUD BROKER
AN INFRASTRUCTURE HEAVY PRODUCT
@burythehammer
THE PROBLEM
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
WE WANT TO MOVE TO THE CLOUD...
BUT WE'RE WARY OF LOCK IN
Large organis...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
USERS
CLOUD BROKER
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
BENEFITS
▸ Quick, easy provisioning
▸ one team previously took 3 months...
@burythehammer
THIS IS A REALLY
COMPLICATED
APPLICATION
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
WORKFLOW
▸ Log into Web UI
▸ Fill in information about environment
▸ Br...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
WEB TESTING
▸ Log into Web UI
▸ Fill in information about environment
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
???
▸ Broker creates and bootstraps resources
▸ SSH into resources
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
INFRASTRUCTURE TESTING
▸ Broker creates and bootstraps resources
▸ SSH ...
@burythehammer
HOW DO YOU TEST
INFRASTRUCTURE?
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
WHAT TO TEST?
Deployment
scripts unit tests?
Does the VPN server work?
...
@burythehammer
THIS SEEMS
FAMILIAR..
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
Does the VPN box work?

Can I SSH into a server?
Do our deployment scri...
@burythehammer
TOOLING
@burythehammer
UNIT TESTING
▸ Bash scripts
▸ Ansible scripts
▸ Terraform scripts
@burythehammer
LINTING
▸ Quick sanity check
▸ Available in all tools
▸ Run in CI before committing
@burythehammer
UNIT TESTING IS HARD
▸ Cultural issues
▸ Technical issues
▸ Return on investment
@burythehammer
INTEGRATION
TESTING
▸ Packages installed
▸ Services running
▸ Ports listening
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
▸ Ruby / RSpec based
▸ Great community
▸ Very readable
▸ Very quick!
▸ ...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
SERVERSPEC EXAMPLE
describe package('jenkins') do
it { should be_instal...
@burythehammer
ACCEPTANCE
TESTING
▸ SSHing into machines
▸ Using applications
▸ e.g. samba, openvpn
@burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE
▸ Executable specifications
▸ Both test & business can read it
▸ Availabl...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer


Scenario: Searching for Wikipedia
Given I am on the website "www.goog...
@burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE
RUBY
▸ Fantastic testing community
▸ More suitable for SSHing into boxes...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
TOOLS WE DIDN'T USE
▸ Bats
▸ ShUnit2
▸ Goss
▸ Inspec / TestInfra
▸ AWS ...
@burythehammer
OUR
SOLUTION
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
USERS
CLOUD BROKER
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
USERS
WEB TEST FRAMEWORK
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
USERS
WEB TESTS
https://github.com/opencredo/test-automation-quickstart
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
USERS
INFRASTRUCTURE TEST FRAMEWORK
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
INFRASTRUCTURE TESTS
Serverspec
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
INFRASTRUCTURE TESTING STACK
/ Serverspec
Linting tools
@burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE
SERVERSPEC SMOKE TESTS
▸ Run before everything else
▸ Really quick
▸ Cat...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
Background:

Given environment has been created

And the following user...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
Background:

Given environment has been created

And the following user...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
Background:

Given environment has been created

And the following user...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
Background:

Given environment has been created

And the following user...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
UNDER THE CUCUMBER, PLAIN RUBY
Then(/^I should be able to echo "([^"]*)...
@burythehammer
LESSONS
LEARNED
THE GOOD, THE BAD, AND
THE UGLY
@burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE
THE GOOD
▸ Specialised tests for each layer
▸ Quick, expressive ServerSp...
@burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE
THE BAD
▸ Over reliance on
acceptance tests
▸ Context switching with
two...
@burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE
THE UGLY
▸ Starting infrastructure is SLOW.
▸ It's expensive...
@burythehammer
IT WAS WORTH IT
DESPITE ALL THAT
@burythehammer
CONCLUSIONS
@burythehammer
TESTING IS
IMPORTANT
BUT OFTEN IGNORED
@burythehammer
TESTERS AND OPS
SHOULD WORK TOGETHER
WE NEED TO GET OUT OF
OUR COMFORT ZONES
@burythehammer
TOOLS EXIST
BUT BE PREPARED
TO HACK
@burythehammer
FINALLY...
TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
THE APPLICATION OF METHODS AND TOOLING
FROM SOFTWARE DEVELOPMENT TO
MAN...
@burythehammer
TESTING IS A SOFTWARE
DEVELOPMENT METHOD
WE SHOULD APPLY IT TO
INFRASTRUCTURE
@burythehammer
THANKS!
QUESTIONS?
matt.long@opencredo.com
<-- sponsors booth on 5th floor!
@burythehammer
Watch the video with slide
synchronization on InfoQ.com!
https://www.infoq.com/presentations/
opencredo-ruby
Próximos SlideShares
Carregando em…5
×

Testing Programmable Infrastructure with Ruby

182 visualizações

Publicada em

Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2ompPdX.

Matt Long talks about some approaches to environment infrastructure testing that his team at OpenCredo has created using Ruby, a language they feel is uniquely positioned to work with both infrastructure and testing. Filmed at qconlondon.com.

Matt Long works as a QA Consultant for OpenCredo, a London-based consultancy specializing in helping clients build and deploy emerging technologies. He is responsible for the testing requirements in a number of OpenCredo engagements, with specialist knowledge in the creation and deployment of automated testing frameworks. He works with tools such as Java, Selenium, Cucumber, Ruby, and Gatling.

Publicada em: Tecnologia
  • Settling for less on valentine's? Then you need VigRX pLUS! ➤➤ http://t.cn/Ai88iYkP
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • I went from getting $3 surveys to $500 surveys every day!! learn more... ➤➤ https://tinyurl.com/realmoneystreams2019
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • Like to know how to take easy surveys and get huge checks - then you need to visit us now! Having so many paid surveys available to you all the time let you live the kind of life you want. learn more...◆◆◆ https://tinyurl.com/realmoneystreams2019
       Responder 
    Tem certeza que deseja  Sim  Não
    Insira sua mensagem aqui
  • Seja a primeira pessoa a gostar disto

Testing Programmable Infrastructure with Ruby

  1. 1. @burythehammer MATT LONG TESTING PROGRAMMABLE INFRASTRUCTURE (WITH RUBY)
  2. 2. InfoQ.com: News & Community Site Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations /opencredo-ruby • Over 1,000,000 software developers, architects and CTOs read the site world- wide every month • 250,000 senior developers subscribe to our weekly newsletter • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • 2 dedicated podcast channels: The InfoQ Podcast, with a focus on Architecture and The Engineering Culture Podcast, with a focus on building • 96 deep dives on innovative topics packed as downloadable emags and minibooks • Over 40 new content items per week
  3. 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon London www.qconlondon.com
  4. 4. @burythehammer PROGRAMMABLE INFRASTRUCTURE IS GREAT, BUT WE'RE MISSING SOMETHING. TESTING.
  5. 5. @burythehammer I'M A TESTER HELLO, I'M MATT @burythehammer
  6. 6. @burythehammer I WORK HERE ↑
  7. 7. @burythehammer I AM NOT A SYSADMIN
  8. 8. @burythehammer WHAT IS PROGRAMMABLE INFRASTRUCTURE?
  9. 9. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer THE APPLICATION OF METHODS AND TOOLING FROM SOFTWARE DEVELOPMENT TO THE MANAGEMENT OF IT INFRASTRUCTURE PROGRAMMABLE INFRASTRUCTURE IS.. THE INTERNET
  10. 10. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer WHAT DO WE MEAN BY THIS? ▸ Automated provisioning & configuration ▸ Configuration as code ▸ Version / source controlled
  11. 11. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer TOOLING EXAMPLES
  12. 12. @burythehammer PROGRAMMABLE INFRASTRUCTURE IS AWESOME! Credit: Vault Boy, Bethesda Softworks
  13. 13. @burythehammer BUT IT CAN GET COMPLEX
  14. 14. @burythehammer TESTING IS USED TO MITIGATE COMPLEXITY & RISK
  15. 15. @burythehammer BUT INFRA TESTING IS RARE Credit: Gunshow, KC Green
  16. 16. @burythehammer TESTING A CLOUD BROKER AN INFRASTRUCTURE HEAVY PRODUCT
  17. 17. @burythehammer THE PROBLEM
  18. 18. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer WE WANT TO MOVE TO THE CLOUD... BUT WE'RE WARY OF LOCK IN Large organisation
  19. 19. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer USERS CLOUD BROKER
  20. 20. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer BENEFITS ▸ Quick, easy provisioning ▸ one team previously took 3 months ▸ Don't need to work at a low level ▸ Templates for common dev environments ▸ Built in best practice: monitoring, security ▸ Track spending
  21. 21. @burythehammer THIS IS A REALLY COMPLICATED APPLICATION
  22. 22. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer
  23. 23. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer WORKFLOW ▸ Log into Web UI ▸ Fill in information about environment ▸ Broker creates and bootstraps resources ▸ SSH into resources
  24. 24. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer WEB TESTING ▸ Log into Web UI ▸ Fill in information about environment
  25. 25. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer ??? ▸ Broker creates and bootstraps resources ▸ SSH into resources
  26. 26. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer INFRASTRUCTURE TESTING ▸ Broker creates and bootstraps resources ▸ SSH into resources
  27. 27. @burythehammer HOW DO YOU TEST INFRASTRUCTURE?
  28. 28. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer WHAT TO TEST? Deployment scripts unit tests? Does the VPN server work? Can instances 
 access one another? Are services running? Linting?
  29. 29. @burythehammer THIS SEEMS FAMILIAR..
  30. 30. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer Does the VPN box work?
 Can I SSH into a server? Do our deployment scripts work?
 Linting? Are services running? ANOTHER TESTING PYRAMID? credit: Ubuntu dev quality guide
 https://developer.ubuntu.com/en/phone/platform/quality/ Can instances access one another? expensive, slow cheap, fast
  31. 31. @burythehammer TOOLING
  32. 32. @burythehammer UNIT TESTING ▸ Bash scripts ▸ Ansible scripts ▸ Terraform scripts
  33. 33. @burythehammer LINTING ▸ Quick sanity check ▸ Available in all tools ▸ Run in CI before committing
  34. 34. @burythehammer UNIT TESTING IS HARD ▸ Cultural issues ▸ Technical issues ▸ Return on investment
  35. 35. @burythehammer INTEGRATION TESTING ▸ Packages installed ▸ Services running ▸ Ports listening
  36. 36. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer ▸ Ruby / RSpec based ▸ Great community ▸ Very readable ▸ Very quick! ▸ Can SSH into instances SERVERSPEC
  37. 37. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer SERVERSPEC EXAMPLE describe package('jenkins') do it { should be_installed } end describe service('jenkins') do it { should be_enabled } it { should be_running } end describe port(8080) do it { should be_listening } end
  38. 38. @burythehammer ACCEPTANCE TESTING ▸ SSHing into machines ▸ Using applications ▸ e.g. samba, openvpn
  39. 39. @burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE ▸ Executable specifications ▸ Both test & business can read it ▸ Available in Ruby, Java, C#... etc ▸ Great reporting ▸ Good CI integration CUCUMBER
  40. 40. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer 
 Scenario: Searching for Wikipedia Given I am on the website "www.google.com" When I search for "Wikipedia" Then the first link should be "www.wikipedia.org" CUCUMBER EXAMPLE
  41. 41. @burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE RUBY ▸ Fantastic testing community ▸ More suitable for SSHing into boxes ▸ "Win RM" gem ▸ Ops already familiar with it ▸ Reduces tech stack
  42. 42. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer TOOLS WE DIDN'T USE ▸ Bats ▸ ShUnit2 ▸ Goss ▸ Inspec / TestInfra ▸ AWS Spec ▸ Test Kitchen
  43. 43. @burythehammer OUR SOLUTION
  44. 44. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer USERS CLOUD BROKER
  45. 45. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer USERS WEB TEST FRAMEWORK
  46. 46. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer USERS WEB TESTS https://github.com/opencredo/test-automation-quickstart
  47. 47. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer USERS INFRASTRUCTURE TEST FRAMEWORK
  48. 48. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer INFRASTRUCTURE TESTS Serverspec
  49. 49. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer INFRASTRUCTURE TESTING STACK / Serverspec Linting tools
  50. 50. @burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE SERVERSPEC SMOKE TESTS ▸ Run before everything else ▸ Really quick ▸ Catches obvious errors ▸ Not complex tasks
  51. 51. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer Background:
 Given environment has been created
 And the following user details:
 | user_alias | username | public_key |
 | userA | testuser | test | 
 
 Scenario: IPA - Login via SSH Key authentication succeeds
 Given user "userA" is authorised to access environment vms
 When user "userA" starts ssh session in host "env" 
 Then I should be able to echo "hello world"
 CUCUMBER FOR ACCEPTANCE TESTING
  52. 52. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer Background:
 Given environment has been created
 And the following user details:
 | user_alias | username | public_key |
 | userA | testuser | test | 
 
 
 Scenario: IPA - Login via SSH Key authentication succeeds
 Given user "userA" is authorised to access environment vms
 When user "userA" starts ssh session in host "env" 
 Then I should be able to echo "hello world"
 CUCUMBER FOR ACCEPTANCE TESTING Cloud broker APIs
  53. 53. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer Background:
 Given environment has been created
 And the following user details:
 | user_alias | username | public_key |
 | userA | testuser | test | 
 
 
 Scenario: IPA - Login via SSH Key authentication succeeds
 Given user "userA" is authorised to access environment vms
 When user "userA" starts ssh session in host "env" 
 Then I should be able to echo "hello world"
 CUCUMBER FOR ACCEPTANCE TESTING Standard Ruby
  54. 54. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer Background:
 Given environment has been created
 And the following user details:
 | user_alias | username | public_key |
 | userA | testuser | test | 
 
 
 Scenario: IPA - Login via SSH Key authentication succeeds
 Given user "userA" is authorised to access environment vms
 When user "userA" starts ssh session in host "env" 
 Then I should be able to echo "hello world"
 CUCUMBER FOR ACCEPTANCE TESTING RSpec assertions
  55. 55. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer UNDER THE CUCUMBER, PLAIN RUBY Then(/^I should be able to echo "([^"]*)"$/) do |text| cmd = "echo #{text}" output = @session.exec!(cmd) close_ssh(@session) expect(output.to_s.strip).to eql(text) end
  56. 56. @burythehammer LESSONS LEARNED THE GOOD, THE BAD, AND THE UGLY
  57. 57. @burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE THE GOOD ▸ Specialised tests for each layer ▸ Quick, expressive ServerSpec tests ▸ Power of a full programming language for user tests ▸ ... it's also totally doable! Credit: Overwatch, Blizzard Entertainment
  58. 58. @burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE THE BAD ▸ Over reliance on acceptance tests ▸ Context switching with two suites ▸ Out of my comfort zone Credit: Futurama, Fox
  59. 59. @burythehammerTESTING PROGRAMMABLE INFRASTRUCTURE THE UGLY ▸ Starting infrastructure is SLOW. ▸ It's expensive...
  60. 60. @burythehammer IT WAS WORTH IT DESPITE ALL THAT
  61. 61. @burythehammer CONCLUSIONS
  62. 62. @burythehammer TESTING IS IMPORTANT BUT OFTEN IGNORED
  63. 63. @burythehammer TESTERS AND OPS SHOULD WORK TOGETHER WE NEED TO GET OUT OF OUR COMFORT ZONES
  64. 64. @burythehammer TOOLS EXIST BUT BE PREPARED TO HACK
  65. 65. @burythehammer FINALLY...
  66. 66. TESTING PROGRAMMABLE INFRASTRUCTURE @burythehammer THE APPLICATION OF METHODS AND TOOLING FROM SOFTWARE DEVELOPMENT TO MANAGEMENT OF IT INFRASTRUCTURE PROGRAMMABLE INFRASTRUCTURE IS..
  67. 67. @burythehammer TESTING IS A SOFTWARE DEVELOPMENT METHOD WE SHOULD APPLY IT TO INFRASTRUCTURE
  68. 68. @burythehammer THANKS! QUESTIONS? matt.long@opencredo.com <-- sponsors booth on 5th floor! @burythehammer
  69. 69. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ opencredo-ruby

×