Testbox is a tool we all should be using to test our ColdFusion Applications which was created and is maintained by Ortus Solutions, the people that brought you ColdBox. We will have Gavin from Ortus in house on this day to go over some testbox examples, talk about its importance, and answer any questions you have. SO --- if you have and high level questions for Gavin, reply to this post (or hit me up) so I can get the questions to Gavin a head of time just in case he needs to consult others at Ortus. Gavin Pickin is a proud ColdFusion developer, starting with ColdFusion in the late 90s. Now working with Ortus Solutions, a leading force in CFML Development frameworks and tools, Gavin gets to work on a lot of great projects, for a big variety of clients. At Ortus Solutions, a big focus is on free and open source tools, on open source Fridays, Gavin spend most of his open source time working on ContentBox Content Management System.

1. TestBox
CFUG Meetup
July 19th

2. Who am I
Gavin Pickin – developing Web Apps since late 90s
● New Addition to Ortus Solutions
● ContentBox Evangelist
What else do you need to know?
● Blog - http://www.gpickin.com
● Twitter – http://twitter.com/gpickin
● Github - https://github.com/gpickin
Let’s get on with the show.

3. So who is already using testing?

4. Trick Question - You are all testing your apps.
● Most people look like this guy
Clicking around in the browser yourself
● Setup Selenium / Web Driver to click
around for you
● Structured Programmatic Tests

5. Bugs Hurt You

6. Bugs Hurt
•Bugs hurt – the later in the process, the harder to
fix.
•Test Early and Often
–Find them before they rot your foundation
•Testable Code is Maintainable Code

7. Types of Testing
● Black/White Box
● Unit Testing
● Integration Testing
● Functional Tests
● System Tests
● End to End Tests
● Sanity Testing
● Regression Test
● Acceptance Tests
● Load Testing
● Stress Test
● Performance Tests
● Usability Tests
● + More

8. Important Testing Types
•Unit Testing
–Test behavior of individual objects
•Integration Testing
–Test Entire Application from Top Down
•UI verification testing
Verification via HTML/Visual elements

9. Important Testing Tools
•TestBox (Run BDD and MXUnit style)
•IDE - CF Builder / Eclipse
•Mocking Framework
•ANT
•Jenkins, Bamboo, Teamcity, other Cis
•Selenium
•Jmeter or Webstress Tool, Apache AB

10. What is TestBox?
TestBox is a next generation testing framework for ColdFusion (CFML)
that is based on BDD (Behavior Driven Development) for providing a
clean obvious syntax for writing tests. It contains not only a testing
framework, runner, assertions and expectations library but also ships
with MockBox, A Mocking & Stubbing Framework. It also supports
xUnit style of testing and MXUnit compatibilities.
More info: https://www.ortussolutions.com/products/testbox

11. Integration Tests
● Integration Tests several of the pieces together
● Most of the types of tests are variations of an Integration Test
● Can include mocks but can full end to end tests including DB /
APIs

12. What is Unit Testing
“unit testing is a software verification and validation method
in which a programmer tests if individual units of source code
are fit for use. A unit is the smallest testable part of an
application”
- wikipedia

13. Unit Testing
•Can improve code quality -> quick error discovery
•Code confidence via immediate verification
•Can expose high coupling
•Will encourage refactoring to produce > testable code
•Remember: Testing is all about behavior and expectations

14. Style of Testing - TDD vs BDD
● TDD = Test Driven Development
● Write Tests
● Run them and they Fail
● Write Functions to Fulfill the Tests
● Tests should pass
● Refactor in confidence
● Test focus on Functionality

15. Style of Testing - TDD vs BDD
● BDD = Behavior Driven Development
● Actually similar to TDD except:
● Focuses on Behavior and Specifications
● Specs (tests) are fluent and readable
● Readability makes them great for all levels of testing in the
organization
● Hard to find TDD examples in JS that are not using BDD
describe and it blocks

16. TDD Example
Test( ‘Email address must not be blank’, function(){
notEqual(email, “”, "failed");
});

17. BDD Example
Describe( ‘Email Address’, function(){
It(‘should not be blank’, function(){
expect(email).not.toBe(“”);
});
});

18. BDD Matchers
expect(true).toBe(true);
expect(true).toBe(true);
expect(true).toBe(true);
expect(true).toBe(true);

19. BDD Matchers
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);
expect(true).not.toBe(true);

20. Matcher Examples
expect(true).toBe(true);
expect(a).not.toBe(null);
expect(a).toEqual(12);
expect(message).toMatch(/bar/);
expect(message).toMatch("bar");
expect(message).not.toMatch(/quux/);
expect(a.foo).toBeDefined();
expect(a.bar).not.toBeDefined();

21. BDD Example
describe("Hello world function", function() {
it(”contains the word world", function() {
expect(helloWorld()).toContain("world");
});
});

22. New BDD Example
feature( "Box Size", function(){
describe( "In order to know what size box I need
As a distribution manager
I want to know the volume of the box", function(){
scenario( "Get box volume", function(){
given( "I have entered a width of 20
And a height of 30
And a depth of 40", function(){
when( "I run the calculation", function(){
then( "the result should be 24000", function(){
// call the method with the arguments and test the outcome
expect( myObject.myFunction(20,30,40) ).toBe( 24000 );
});
});

23. Installing Testbox
Install Testbox – Thanks to Commandbox - this is easy.
# box install testbox
Next, we need to decide how you want to run Testbox

24. Create a Runner.cfm File to Run your tests
<cfsetting showDebugOutput="false">
<!--- Executes all tests in the 'specs' folder with simple reporter by default --->
<cfparam name="url.reporter" default="simple">
<cfparam name="url.directory" default="tests.specs">
<cfparam name="url.recurse" default="true" type="boolean">
<cfparam name="url.bundles" default="">
<cfparam name="url.labels" default="">
<!--- Include the TestBox HTML Runner --->
<cfinclude template="/testbox/system/runners/HTMLRunner.cfm" >

25. Create a test suite
// tests/specs/CFCTest.cfc
component extends="testbox.system.BaseSpec" {
function run() {
it( "will error with incorrect login", function(){
var oTest = new cfcs.userServiceRemote();
expect( oTest.login( 'gavin@gavin.com', 'topsecret').result ).toBe('400');
});
}
}

26. Create 2nd Test Suite
// tests/specs/APITest.cfc
component extends="testbox.system.BaseSpec" {
function run() {
describe("userService API Login", function(){
it( "will error with incorrect login", function(){
var email = "gavin@gavin.com";
var password = "topsecret”;
var result = "";
http
url="http://www.testableapi.local.com:8504/cfcs/userServiceRemote.cfc?method=login&e
mail=#email#&password=#password#" result="result”;
expect( DeserializeJSON(result.filecontent).result ).toBe('400');
});
});
}

27. Run tests in Browser with Runner

28. Run tests with Grunt
● Install Testbox Runner – Thanks Sean Coyne
# npm install testbox-runner
● Install Grunt Shell
# npm install grunt-shell
Then, we need to add Grunt Configuration

29. Add Grunt Config - 1
module.exports = function (grunt) {
grunt.loadNpmTasks('grunt-shell');
grunt.initConfig({ … })
}

30. Add Grunt Config - 2
Watch: {
…
cfml: {
files: [ "cfcs/*.cfc"],
tasks: [ "testbox" ]
}
}

31. Add Grunt Config - 3
shell: {
testbox: {
command: "./node_modules/testbox-runner/index.js
--colors --runner
http://www.testableapi.local.com:8504/tests/runner.cfm
--directory /tests/specs --recurse true”
}
}

32. Add Grunt Config - 4
grunt.registerTask("testbox", [ "shell:testbox" ]);
grunt.loadNpmTasks('grunt-contrib-jasmine');
grunt.loadNpmTasks('grunt-contrib-watch');

33. Grunt Config Gists
Jasmine + Testbox
https://gist.github.com/gpickin/9fc82df3667eeb63c7e7

34. TestBox output with Grunt

35. TestBox JSON output
Testbox has several runners, you have seen the HTML one, this
Runner uses the JSON runner and then formats it.
http://www.testableapi.local.com:8504/tests/runner.cfm?report
er=JSON&directory=%2Ftests%2Fspecs&recurse=true

36. Running TestBox in Sublime 2
Install PackageControl into Sublime Text
Install Grunt from PackageControl
https://packagecontrol.io/packages/Grunt
Update Grunt Sublime Settings for paths
{
"exec_args": { "path": "/bin:/usr/bin:/usr/local/bin” }
}
Then Command Shift P – grunt

37. TestBox Output in Sublime 2

38. Run TestBox with CommandBox
Run the tests in the box.json file.
# testbox run
Watch the folders and run tests on change
# testbox watch
More info on TestBox usage from CommandBox:
https://www.ortussolutions.com/blog/using-testbox-watch-to-a
utomate-your-testing-suite

39. More Testing - Mocking with MockBox
"A mock object is an object that takes the place of
a ‘real’ object in such a way that makes testing
easier and more meaningful, or in some cases,
possible at all"
by Scott Bain - Emergent Design

40. More Testing - Integrated
Start using an easy, fluent API for your integration tests!
More Info: https://github.com/elpete/integrated
Api - https://elpete.github.io/integrated/

41. Some real life examples
https://github.com/framework-one/fw1/tree/develop/tests maybe?
https://github.com/Ortus-Solutions/TestBox/tree/development/tests
https://github.com/foundeo/cfdocs/tree/master/tests
https://github.com/foundeo/cfmlparser/tree/master/tests/tests
https://github.com/foundeo/toscript/tree/master/tests
https://github.com/foundeo/bolthttp/tree/master/test
https://github.com/foundeo/cfml-security/tree/master/tests/tests/secureupload

42. Some real life examples cont.
https://github.com/ryanguill/emit/tree/master/tests
https://github.com/ryanguill/cfmlBase62/tree/master/tests
https://gitlab.com/ryanguill/FPcfc/tree/master/tests
https://gitlab.com/ryanguill/template-cfc/tree/master/tests
https://github.com/MotorsportReg/sidecar/tree/master/tests/basic
https://github.com/MotorsportReg/sidecar/tree/master/tests/basic