Catching regressions faster with automated acceptance tests

1. Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Catching regressions faster with
automated acceptance tests
Jonathan Bardo

2. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
2Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
About me
• Senior Product Engineer – WordPress for GoDaddy
• Currently working on
• New onboarding experience for GoDaddy’s clients
• Love
• PHP
• JavaScript
• Big Data platforms
• Making WordPress plugins
• Tests
• Based in Montreal
jonathanbardo

3. Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
What is acceptance
testing?
3

4. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
4Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Definition
Acceptance testing is used to make sure
the requirements of a specification are met.

5. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
5Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Example
Using Codeception PHP framework
<?php
$I->wantTo( 'Log into WordPress admin' );
// Let's start on the login page
$I->amOnPage( wp_login_url() );
// Populate the login form's user id field
$I->fillField( [ 'id' => 'user_login' ], 'admin' );
// Populate the login form's password field
$I->fillField( [ 'id' => 'user_pass' ], 'password' );
// Submit the login form
$I->click( [ 'name' => 'wp-submit' ] );

6. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
6Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
In short
If there’s a new WordPress release coming out soon.
😱 😎
Go from this To this

7. Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
The tools
7

8. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
8Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
The tools

9. Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Installation
9
Running tests locally
jonathanbardo.com/?p=559

10. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
10Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Steps
1. Setup your local environment (VVV, Laravel Valet, MAMP, others)
2. Install Selenium Server
3. Install a browser driver (e.g. Chromedriver)

11. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
11Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Demo: My local setup

12. Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Writing acceptance
tests

13. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
13Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
codeception.yml
actor: Tester
paths:
tests: tests/codeception
log: tests/codeception/_output
data: tests/codeception/_data
helpers: tests/codeception/_support
settings:
bootstrap: _bootstrap.php
colors: true
memory_limit: 256M

14. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
14Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
acceptance.suite.yml
tests/codeception/acceptance.suite.yml
class_name: AcceptanceTester
modules:
enabled:
- WebDriver
- WordPress
- AcceptanceHelper
config:
WebDriver:
window_size: 1366x764

15. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
15Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Demo: Sample plugin

16. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
16Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
A look back
<?php
$I->wantTo( 'Log into WordPress admin' );
// Let's start on the login page
$I->amOnPage( wp_login_url() );
// Populate the login form's user id field
$I->fillField( [ 'id' => 'user_login' ], 'admin' );
// Populate the login form's password field
$I->fillField( [ 'id' => 'user_pass' ], 'password' );
// Submit the login form
$I->click( [ 'name' => 'wp-submit' ] );

17. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
17Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Demo: Let’s write a test

18. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
18Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Let’s write an actual test
/**
* Validate front end site description is present
*
* @param AcceptanceTester $I
*/
public function validateFrontEndSiteDescription( AcceptanceTester $I ) {
$I->wantTo( 'Make sure the site description is "Codeception demo"' );
$I->amOnPage( home_url() );
$I->canSee( __( 'Codeception demo', 'codeception-demo' ) );
}
tests/codeception/acceptance/UserTestCest.php

19. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
19Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Let’s write an actual test
/**
* Validate that our admin menu has the right text
*
* @before login
* @param AcceptanceTester $I
*/
public function validateAdminMenu( AcceptanceTester $I ) {
$I->wantTo( 'Make sure the Dashboard admin menu text is "Codeception demo"' );
$I->amOnPage( admin_url() );
$I->canSee(
__( 'Codeception demo', 'codeception-demo' ),
[ 'css' => '#menu-dashboard .wp-menu-name' ]
);
}
tests/codeception/acceptance/UserTestCest.php

20. Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Running acceptance
tests

21. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
21Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Running codeception in the terminal
$ wp codeception run --debug

22. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
22Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Demo

24. Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Using Travis CI

25. CONFIDENTIAL. COPYRIGHT © 2016 GODADDYINC. ALL RIGHTSRESERVED.
25Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Demo

26. Copyright© 2016 GoDaddy Inc. · 14455 N. Hayden Road Scottsdale, Arizona 85260 (480) 505-8800 · All Rights Reserved.
Thank you
jonathanbardo