In this advanced session, we will investigate all the ways that you can automate your testing processes with TestBox and many CI and automation tools. From Jenkins integration, Travis CI, Node runners, Grunt watchers and much more. This session will show you the value of continuous integration and how to apply it with modern tools and technologies.

1. Testing Automaton
Luis Majano @lmajano

2. Who am I
• Luis Majano
• Computer Engineer
• Imported from El Salvador
• CEO of Ortus Solutions
• Creator of all things Box

3. Why people don’t test
Why we need to test
How to test
How to automate

4. Where to start?
Write Tests

5. WHY PEOPLE DON’T TEST
COMFORT

6. WHY PEOPLE DON’T TEST
New Methodology
New Learned Behavior
It is a leap….

7. BIGGEST LIE IN SOFTWARE DEV
Don’t worry, we will create the
tests and refactor it later!

9. Bugs cost money
Cost To Fix Time detected
Requirements Design Building Testing Post-Release
Time
Introduced
Requirements 1x 3x 5-10x 10x 10-100x
Design --- 1x 10x 15x 25-100x
Building -- -- 1x 10x 10-25x
^ Kaner, Cem; James Bach, Bret Pettichord (2001). Lessons Learned in Software Testing: A Context-Driven Approach. Wiley. p. 4. ISBN 0-471-08112-4.

10. • Just do it!
• You will get dirty
• It can hurt (a little)
• Learned behavior
NO MORE EXCUSES
IT WILL ACCELERATE YOUR
DEVELOPMENT

11. TDD/BDD = Process
Continuous Integration
Continuous Delivery
Continuous Improvement
AUTOMATION

13. TestBox : What you get
API Docs
Core
Test Browser
Test Harness
Test Runner
Samples
install testbox —saveDev
install testbox@be —saveDev

14. Test Harness
Automated test
results!
xUnit/BDDTest
Bundles
Harness bootstrap
HTML Runner
ANT Runner

15. Test Bundle CFCs
• Inherits from testbox.system.BaseSpec
• TesBox will then execute all tests/specs/suites within 1 or more bundles

16. RUN TESTS
• Execute bundles via the URL
• http://localhost/tests/bundle.cfc?method=runRemote
• SOAP Runner
• HTTP/REST Runner
• ANT Runner
• NodeJS Runner
• CommandBox Runner
• Custom Runner

17. TESTBOX REPORTERS
•ANTJunit :Variant of JUnit for Jenkins
•Codexwiki : Mediawiki syntax
•Console : Server console
•Doc : Semantic HTML
•Dot :Awesome Dots
•JSON
•JUnit
•Raw : CFML Struct
•Simple :A basic HTML reporter
•Text : Back to the 80’s,Text
•MinText : Great for CommandBox Runners
•XML
•Tap :A test anything protocol reporter
•Min : Minimalistic Report

18. MANUAL TESTING
Bundle CFC Runner

19. CommandBox Runner
package set testbox.runner = "http://127.0.0.1:57538/tests/runner.cfm"
testbox run
1
2

20. {
"runner": "http://127.0.0.1:57538/tests/runner.cfm",
"directory": "/tests/specs",
"recurse": true
}
NodeJS Test Runner
npm install -g testbox-runner
create .testbox-runnerrc
1
2
Run it : testbox-runner2

21. ANT TESTING

22. DEVELOPER AUTOMATION
1.Grunt Watchers + LiveReload
2.Grunt Watchers + testbox-runner
3.Grunt Watchers + commanbox
4.ColdBox Elixir

23. CONTINOUS INTEGRATION

24. CI SERVERS

25. Jenkins + CommandBox
wget http://mirrors.jenkins-ci.org/war-stable/latest/jenkins.war
box server start warpath=jenkins.war
1
2

26. Travis CI - Sign Up
https://travis-ci.org/
Sign Up
1
2

27. Accounts
Activate For Repositories
1
2
Create .travis.yml3
Commit to repository4
Travis CI - Getting Started

28. TRAVIS + ANTlanguage: java
branches:
only:
- development
sudo: required
dist: trusty
before_install:
- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622
- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a
/etc/apt/sources.list.d/commandbox.list
install:
- sudo apt-get update && sudo apt-get --assume-yes install commandbox
- box install
- box server start
before_script:
# Startup the app
- curl http://localhost:49616/
script:
- ant -f workbench/build.xml

29. TRAVIS + COMMANDBOXlanguage: java
sudo: required
dist: trusty
before_install:
- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622
- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a
/etc/apt/sources.list.d/commandbox.list
install:
- sudo apt-get update && sudo apt-get --assume-yes install commandbox
- cd tests
- box install
- box server start
before_script:
# Startup the app
- curl http://localhost:49615/
script: >
testResults="echo $(box testbox run)";
echo "$testResults";
if grep -i "[Failures: [1-9][0-9]?[0-9]?]|[Errors: [1-9][0-9]?[0-9]?]|
<t[^>]*>|<b[^>]*>" <<< $testResults; then exit 1; fi

30. MULTIPLE VERSIONS
MULTIPLE ENGINES

31. TRAVIS + MULTIPLE ENGINESlanguage: java
sudo: required
jdk:
- oraclejdk8
env:
matrix:
- ENGINE=lucee@4.5
- ENGINE=lucee@5
- ENGINE=adobe@2016
- ENGINE=adobe@11
- ENGINE=adobe@10
before_install:
- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622
- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a
/etc/apt/sources.list.d/commandbox.list
install:
- sudo apt-get update && sudo apt-get --assume-yes install commandbox
- box install
before_script:
- box server start cfengine=$ENGINE port=8500
script: >
testResults="$(box testbox run runner='http://127.0.0.1:8500/tests/runner.cfm' )";
echo "$testResults";
if grep -i "[(Failures|Errors): [1-9][0-9]?[0-9]?]|[Errors: [1-9][0-9]?[0-9]?]|<t[^>]*>|
<b[^>]*>" <<< $testResults; then exit 1; fi
notifications:
email: false

32. Resources
• TestBox : ortussolutions.com/products/testbox
• CommandBox: ortussolutions.com/products/commandbox
• Slack: boxteam.herokuapp.com
• CFML Slack: #box-products
• travis-ci.org
• jenkins.io

33. Thank
you!