The document provides an overview of behavior driven development (BDD) and the Behat testing framework. Some key points:
- BDD is an "outside-in" methodology that starts by identifying business outcomes and defining features and acceptance criteria through scenarios.
- Gherkin is the business-readable domain-specific language used to write features, scenarios, and acceptance criteria.
- Features describe the intended behavior using scenarios written in Given/When/Then format.
- Behat supports drivers, hooks, fixtures, and steps to implement scenario behavior in code.
- Writing good features focuses on exact context, independent scenarios, intention over implementation, and exploring all paths.
3. What is
Behavior
Driven
Development?
Behaviour-driven development is an “outside-in”
methodology. It starts at the outside by identifying
business outcomes, and then drills down into the
feature set that will achieve those outcomes. Each
feature is captured as a “story”, which defines the
scope of the feature along with its acceptance criteria.
Dan North, “What’s in a Story”
http://dannorth.net/whats-in-a-story
It’s the idea that you start by writing human-readable
sentences that describe a feature of your application
and how it should work, and only then implement this
behavior in software.
Behat Documentation
http://docs.behat.org/en/v2.5
4. BDD is not about...
Well Designed Code
Automated Testing
Implementation
UI Testing
5. Why Practice
Behavior Driven
Development?
“You can turn an idea for a
requirement into implemented,
tested, production-ready code
simply and effectively, as long as
the requirement is specific
enough that everyone knows
what’s going on.”
Dan North, “What’s in a Story”
http://dannorth.net/whats-in-a-story
6. Gherkin
Business Readable (Writable?), Domain Specific Language
Living Documentation / User Manual
Story: Feature File
one Feature with a narrative
one or more Scenarios
Acceptance Criteria: Scenarios
7. Narrative
As a [role]
I want [feature]
So that [benefit / business reason]
Use “5 Whys” to determine narrative
Feature: Account Holder Withdraws Cash
Feature: Short, Descriptive, Action
Acceptance Criteria: Scenarios
Given: Exact Context
When: Action/Event
Then: Outcomes
And/But: More of the same...
Scenario: Account has sufficient funds
Given the account balance is $100
And the card is valid
And the machine contains enough money
When I request $20
Then the ATM should dispense $20
And the account balance should be $80
And the card should be returned
Scenario: Account has insufficient funds
Given the account balance is $10
And the card is valid
And the machine contains at least the amount of my balance
When I request $20
Then the ATM should not dispense any money
And the ATM should say there are insufficient funds
And my account balance should be $10
And the card should be returned
Scenario: Card has been disabled
Given the card is disabled
When I request $20
Then the ATM should retain the card
And the ATM should say the card has been retained
As an Account Holder
I want to withdraw cash from an ATM
So that I can get money when the bank is closed
8. Gherkin Keywords: Auto-Generated Steps
You can implement step definitions for undefined steps with these snippets:
/**
* @When /^I select Texas from the states list$/
*/
public function iSelectTexasFromTheStatesList()
{
throw new PendingException();
}
/**
* @Then /^I should see the list of services offered$/
*/
public function iShouldSeeTheListOfServicesOffered()
{
throw new PendingException();
}
11. Feature: View Countdown before
Broadcast
As a user viewing a broadcast page before the
broadcast starts
I want to see a countdown timer
So that I can know how long until the broadcast
actually starts
Scenario: View Countdown before Broadcast
Given I view the “Future Broadcast” broadcast
Then I should see "Future Broadcast" on the page
And I should see "Future Broadcast Author" on the
page
And I should see "This broadcast begins at 6:00
pm EST" on the page
And I should see a countdown timer
Process
Author describes Feature with
implementation specific
example
Developer adds fixture data
Issues
Test only works before 6pm
What is the intent?
Confusion for business users
12. Feature: View Countdown before
Broadcast
As a user viewing a broadcast page before the
broadcast starts
I want to see a countdown timer
So that I can know how long until the broadcast
actually starts
Scenario: View Countdown before Broadcast
Given there is a broadcast scheduled for the future
When I view that broadcast’s page
Then I should see the broadcast title
And I should see the broadcast author’s name
And I should see "This broadcast begins at"
followed by the start time in EST
And I should see a countdown timer
Changes
Given now explains exact
context
Test is no longer time-
dependent
Intent - not implementation
Why?
Overall understanding
Communication value
Help identify poorly written
code
13. Feature: Customer Views Product and Services Catalog
As a customer
I want to view the product catalog
So that I can browse products and services
Scenario: Customer views Product Catalog
Given I view the catalog
When I select "Texas" from the list of states
Then I should see the list of products for sale
Scenario: Customer views Local Services in Product Catalog
Given I view the catalog
When I select "Texas" from the list of states
Then I should see the list of services offered
Issues
What is the intent?
Is Texas relevant?
14. Feature: Customer Views Product and Services Catalog
As a customer
I want to view the product catalog
So that I can browse products and services
Scenario: Customer views Product Catalog
Given I view the catalog
When I select "Texas" from the list of states
Then I should see the list of products for sale
Scenario: Customer views Local Services in Product Catalog
Given I view the catalog
When I select "Texas" from the list of states
Then I should see the list of services offered
Issues
What is the intent?
Is Texas relevant?
if not: implementation detail
Scenario: Customer views Product Catalog
Given I view the catalog
When I select a state from the list of states
Then I should see the list of products for sale
Scenario: Display Local Services in Product Catalog
Given I view the catalog
When I select a state from the list of states
Then I should see the list of services offered
15. Feature: Customer Views Product and Services Catalog
As a customer
I want to view the product catalog
So that I can browse products and services
Scenario: Customer views Product Catalog
Given I view the catalog
When I select "Texas" from the list of states
Then I should see the list of products for sale
Scenario: Customer views Local Services in Product Catalog
Given I view the catalog
When I select "Texas" from the list of states
Then I should see the list of services offered
Issues
What is the intent?
Is Texas relevant?
if not: implementation detail
if yes: uncover business value and
context
Scenario: Customer views Product Catalog
Given I view the catalog
When I select a state from the list of states
Then I should see the list of products for sale
Scenario: Display Local Services in Product Catalog
Given the company has a regional office in a state
When I view the catalog
And I select that state from the list of states
Then I should see the list of services offered
16. Feature: Customer Views Product and Services Catalog
As a customer
I want to view the product catalog
So that I can browse products and services
Scenario: Customer views Product Catalog
Given I view the catalog
When I select "Texas" from the list of states
Then I should see the list of products for sale
Scenario: Customer views Local Services in Product Catalog
Given I view the catalog
When I select "Texas" from the list of states
Then I should see the list of services offered
Narrative
Use narrative to explain
business value for feature
Not processed in Behat 2.5
Behat 3.0+ role can be used to
specify which context files are
used
Feature: Customer Views Texas-Specific Catalog
As a customer in Texas
I want to view the products and services for sale in Texas
So that I can buy them
Scenario: Display Local Services when viewing Texas
Given I view the catalog
And I select Texas from the list of states
Then I should see the products for sale in Texas
And I should see the list of services offered in Texas
17. Implementation
When I select "Texas" from the states list
/**
* @When /^I select "([^"]*)" from the states
list$/
*/
public function iSelectFromTheStatesList($arg1){}
Intent
When I select Texas from the states list
/**
* @When /^I select Texas from the states list$/
*/
public function iSelectTexasFromTheStatesList(){}
Demonstrate Intent by Removing
Variables
Shows specific intent rather
than an example of behavior
Indicates there is an important
business value to the state
Impossible for anyone to plug
in another state
18. Feature: Customer Views Product and Services Catalog
As a customer
I want to view the product catalog
So that I can browse products and services
Scenario: Customer views Product Catalog
Given I view the catalog
When I select a state from the list of states
Then I should see the list of products for sale
Scenario: Display Local Services in Product Catalog
Given the company has a regional office in a state
When I view the catalog
And I select that state from the list of states
Then I should see the list of services offered
echo '<h1>Products</h1>';
foreach ($products AS $product) {
echo '<p>' . $product->getName() . '</p>';
}
echo '<h1>Services</h1>';
foreach ($services AS $service) {
echo '<p>' . $service->getName() . '</p>';
}
Scenario: Don’t Display Local Services in Product Catalog For States
With No Regional Office
Given the company does not have a regional office in a
state
When I view the catalog
And I select that state from the list of states
Then I should not see a list of services offered
19. Feature: Customer Views Product and Services Catalog
As a customer
I want to view the product catalog
So that I can browse products and services
Scenario: Customer views Product Catalog
Given I view the catalog
When I select a state from the list of states
Then I should see the list of products for sale
Scenario: Display Local Services in Product Catalog
Given the company has a regional office in a state
When I view the catalog
And I select that state from the list of states
Then I should see the list of services offered
Scenario: Don’t Display Local Services in Product Catalog For States
With No Regional Office
Given the company does not have a regional office in a
state
When I view the catalog
And I select that state from the list of states
Then I should not see a list of services offered
echo '<h1>Products</h1>';
foreach ($products AS $product) {
echo '<p>' . $product->getName() . '</p>';
}
if(count($services) > 0) {
echo '<h1>Services</h1>';
foreach ($services AS $service) {
echo '<p>' . $service->getName() . '</p>';
}
}
20. Writing Great Features
Exact Context
Independent Scenarios &
Features
Intention, not Implementation
Defined Narrative
All Paths Explored
Feature “Smells”
Time Dependency
Interdependency
Multi-Scenario Scenarios
Missing Scenarios
Overuse of Variables
Examples of Behavior
(Implementation, not
Intention)
28. Fixture Data
namespace AcmeAppBundleDataFixtures;
use DoctrineCommonPersistenceObjectManager;
use DoctrineCommonDataFixturesFixtureInterface;
class UserFixtureLoader implements FixtureInterface
{
public function load(ObjectManager $manager)
{
$user = new User();
$user->setUsername('admin');
$user->setPassword('password');
$manager->persist($user);
$manager->flush();
}
}
29. Load Fixture Data
/** @BeforeFeature */
public function beforeFeatureReloadDatabase($event)
{
$loader = new Loader();
$directory = __DIR__ . DIRECTORY_SEPARATOR .
'..' . DIRECTORY_SEPARATOR .
'DataFixtures';
$loader->loadFromDirectory($directory);
$entityManager = $this->getEntityManager();
$purger = new ORMPurger();
$executor = new ORMExecutor($entityManager, $purger);
$executor->execute($loader->getFixtures());
}
31. Multiple Regular Expressions
/**
* @Given /^I view the catalog$/
* @Given /^I am viewing the catalog$/
*/
public function iViewTheCatalog(){
$this->getPage('Catalog')->open();
}
32. Case Insensitive - Flag
Given I view the catalog
Given I view the Catalog
/**
* @Given /^I am viewing the catalog$/i
*/
public function iViewTheCatalog(){
$this->getPage('Catalog')->open();
}
33. Given I view the catalog
Given I view the Catalog
/**
* @Given /^I am viewing the (?i)catalog$/
*/
public function iViewTheCatalog(){
$this->getPage('Catalog')->open();
}
Case Insensitive - Inline
34. Quoted Variables
Then I should see an "error" message
/**
* @Given /^I should see an "([^"])" message$/
*/
public function iShouldSeeAnMessage($arg1){
}
35. Then I should see an error message
/**
* @Given /^I should see an (.*) message$/
*/
public function iShouldSeeAnMessage($arg1){
}
Unquoted Variables
36. Unquoted Variables with List of Options
Then I should see an error message
Then I should see a success message
Then I should see a warning message
/**
* @Given /^I should see an? (error|success|warning) message$/
*/
public function iShouldSeeAnMessage($messageType){
$class = '.alert-'.$messageType;
$this->assertElementExists($class, 'css');
}
37. Optional Variables
Then I should see an error message
Then I should see an error message that says "Stop!"
/**
* @Given /^I should see an? (error|success|warning) message$/
* @Given /^I should see an? (error|success|warning) message that says "([^"]*)"$/
*/
public function iShouldSeeAnMessageThatSays($messageType, $message = null)
{
$class = '.alert -' . $messageType;
$this->assertElementExists($class, 'css');
if ($message !== null) {
$this->assertElementContainsText($class, 'css', $message);
}
}
38. Non-Capturing Groups
Then I view the catalog for "Texas"
Then I am viewing the catalog for "Texas"
/**
* @Given /^I view the catalog for "([^"]*)"$/
* @Given /^I am viewing the catalog "([^"]*)"$/
*/
public function iViewTheCatalogForState($stateName)
{
$args = ['stateName' => $stateName];
$this->getPage('Catalog')->open($args);
}
39. Non-Capturing Groups
Then I view the catalog for "Texas"
Then I am viewing the catalog for "Texas"
/**
* @Given /^I (?:am viewing|view) the catalog for "([^"]*)"$/
*/
public function iViewTheCatalogForState($stateName)
{
$args = ['stateName' => $stateName];
$this->getPage('Catalog')->open($args);
}
40. Step Definition Changes in Behat 3.x
Then I view the catalog for "Texas"
Then I view the catalog for Texas
/**
* @Given I view the catalog for :stateName
*/
public function iViewTheCatalogForState($stateName)
{
$args = ['stateName' => $stateName];
$this->getPage('Catalog')->open($args);
}
42. SubContext (Behat 2.x)
namespace AcmeAppBundleContext;
use BehatMinkExtensionContextMinkContext;
class FeatureContext extends MinkContext
{
public function __construct(array $parameters)
{
$this->parameters = $parameters;
$this->useContext('MessageContext', new MessageContext());
}
}
43. Several SubContexts (Behat 2.x)
[...]
public function __construct(array $parameters)
{
$this->parameters = $parameters;
$this->useContext('AdminContext', new AdminContext());
$this->useContext('FormContext', new FormContext());
$this->useContext('EditUserContext', new EditUserContext());
$this->useContext('ApiContext', new ApiContext());
}
44. Alias All SubContexts Automatically (Behat 2.x)
private function loadSubContexts()
{
$finder = new Finder();
$finder->name('*Context.php')
->notName('FeatureContext.php')
->notName('CoreContext.php');
$finder->files()->in(__DIR__);
}
45. Alias All SubContexts Automatically (Behat 2.x)
private function loadSubContexts()
{
$finder = new Finder();
$finder->name('*Context.php')
->notName('FeatureContext.php')
->notName('CoreContext.php');
$finder->files()->in(__DIR__);
foreach ($finder as $file) {
$className = $file->getBaseName('.php');
$namespace = __NAMESPACE__ . '' . $file->getRelativePath();
if (substr($namespace, -1) !== '') {
$namespace .= '';
}
$reflectionClass = new ReflectionClass($namespace . $className);
$this->useContext($className, $reflectionClass->newInstance());
}
}
46. <?php
namespace AcmeAppBundleContext;
class FeatureContext extends CoreContext
{
/** @Given /^I should see an? (error|success|warning) message that says
"([^"])"$/ */
public function iShouldSeeAnMessageThatSays($messageType, $message = null)
{
$class = '.alert -' . $messageType;
$element = $this->getPage()->find('css', $class);
$actualMessage = $element->getText();
$this->assertEqual($actualMessage, $message);
}
}
Message Context
47. Find Required Element Shortcut
public function findRequiredElement($locator, $selector = 'xpath', $parent = null)
{
if (null === $parent) {
$parent = $this->getPage();
}
$element = $parent->find($selector, $locator);
if (null === $element) {
throw new ElementNotFoundException($this->getSession(), null, $selector, $locator);
}
return $element;
}
48. Message Context
namespace AcmeAppBundleContext;
class FeatureContext extends CoreContext
{
/** @Given /^I should see an? (w*) message that says "([^"])"$/ */
public function iShouldSeeAnMessageThatSays($messageType, $message = null)
{
$class = '.alert -' . $messageType;
$element = $this->findRequiredElement($class, 'css');
$actualMessage = $element->getText();
$this->assertEqual($actualMessage, $message);
}
}
49. CoreContext with Step Annotation causes Error
[BehatBehatExceptionRedundantException]
Step "/^I should be redirected to "([^"]*)"$/" is already defined in
AcmeAppBundleContextFeatureContext::iShouldBeRedirectedTo()
AcmeAppBundleContextFeatureContext::iShouldBeRedirectedTo()
AcmeAppBundleContextMessageContext::iShouldBeRedirectedTo()
51. Reusing Multiple Steps
Scenario: Upload a csv file
Given I am viewing the csv import form
When I attach a csv to "Import File"
And I submit the form
Then I should see a success message
And I should see the file review screen
Scenario: Review and Confirm the csv file
Given I have uploaded a csv
And I am viewing the file review screen
When I select a property for each column
And I submit the form
Then I should see a success message
52. Meta-Steps
use BehatBehatContextStep;
class FeatureContext
{
/** @Given /^I have uploaded a csv$/ */
public function iHaveUploadedACsv()
{
return [
new StepGiven('I am viewing the csv import form'),
new StepWhen('I attach a csv to "Import File"'),
new StepWhen('I submit the form')
];
}
}
53. Meta-Steps With Multi-line Arguments
use BehatBehatContextStep;
use BehatGherkinNodePyStringNode;
class FeatureContext
{
/** @Given /^I should see the file review screen$/ */
public function iShouldSeeTheFileReviewScreen()
{
$content = 'Please review your file .' . PHP_EOL .
'Press Submit to continue';
$pyString = new PyStringNode($content);
return new StepGiven('I should see', $pyString);
}
}
54. Direct Method Call - Same Context
/** @Given /^I should see an error about the file type$/ */
public function iShouldSeeAnErrorAboutTheFileType()
{
$message = 'This file type is invalid';
$this->iShouldSeeAnMessageThatSays('error', $message);
}
/** @Given /^I should see an? (.*) message that says "([^"])"$/ */
public function iShouldSeeAnMessageThatSays($messageType, $message = null)
{
$class = '.alert -' . $messageType;
$this->assertElementExists($class, 'css');
if ($message !== null) {
$this->assertElementContainsText($class, 'css', $message);
}
}
55. Direct Method Call to Another Context (Behat 2.x)
/**
* @Given /^I should see an error about the file type$/
*/
public function iShouldSeeAnErrorAboutTheFileType()
{
$message = "This file type is invalid";
$this->getMainContext()
->getSubContext('MessageContext')
->iShouldSeeAnMessageThatSays('error', $message);
}
56. Direct Method Call to Another Context (Behat 2.x)
/**
* @return MessageContext
*/
public function getMessageContext()
{
return $this->getMainContext()->getSubContext('messageContext');
}
/**
* @Given /^I should see an error about the file type$/
*/
public function iShouldSeeAnErrorAboutTheFileType()
{
$message = "This file type is invalid";
$this->getMessageContext()->iShouldSeeAnMessageThatSays('error', $message);
}
58. Store Other Contexts (Behat 3.x)
use BehatBehatContextContext;
use BehatBehatHookScopeBeforeScenarioScope;
class FeatureContext implements Context
{
/** @var MessageContext */
private $messageContext;
/** @BeforeScenario */
public function gatherContexts(BeforeScenarioScope $scope)
{
$environment = $scope->getEnvironment();
$this->messageContext = $environment->getContext('MessageContext');
}
}
// http://docs.behat.org/en/v3.0/cookbooks/context_communication.html
59. Direct Method Call to Another Context (Behat 3.x)
use BehatBehatContextContext;
use BehatBehatHookScopeBeforeScenarioScope;
class FeatureContext implements Context
{
/** @var BehatMinkExtensionContextMinkContext */
private $minkContext;
/** @BeforeScenario */
public function gatherContexts(BeforeScenarioScope $scope)
{
$environment = $scope->getEnvironment();
$this->minkContext = $environment->getContext('BehatMinkExtensionContextMinkContext');
}
}
// http://docs.behat.org/en/v3.0/cookbooks/context_communication.html
/**
* @Given /^I should see an error about the file type$/
*/
public function iShouldSeeAnErrorAboutTheFileType()
{
$message = "This file type is invalid";
$this->messageContext->iShouldSeeAnMessageThatSays('error', $message);
}
60. Direct Call
● Like any other method
call
● Hooks do not fire
(typically faster)
● Moving Step definitions
might require refactor
Meta-Steps
Return a Step or array of
Steps
Hooks will fire
(could be slow)
Moving Step definitions
does not break
Removed in 3.0
63. Seminar Page Object
Scenario: Visit Seminar Page before Broadcast Time
Given there is a seminar scheduled for the future
When I visit that seminar's page
Then I should see the seminar's name
And I should see the seminar's author’s name
And I should see "This seminar begins at"
And I should see the seminar’s start time in EST
And I should see a countdown timer
Scenario: Visit Seminar Page before Broadcast Time
Given there is a seminar scheduled
When I visit that seminar's page during the broadcast time
Then I should see the seminar's name
And I should see the seminar video
And I should not see a countdown timer
64. Seminar Page Object
namespace AcmeAppBundlePageObjects;
use SensioLabsBehatPageObjectExtensionPageObjectPage;
class Seminar extends Page
{
protected $path = '/seminar/{id}';
}
68. CSV Report
@javascript
Scenario: View Summary Report
Given a user in a group has registered for a seminar with a company
And I am logged in as an admin
And I am viewing the reports area
When I download the "Summary" report
Then I should see the following columns:
| column |
| Group |
| Company |
| Total |
And I should see that user in the report
69. File Download Test
/**
* @When /^I download the "([^"]*)" report$/
*/
public function iDownloadTheReport($reportName)
{
$xpath = "//a[normalize-space()='{$reportName}']";
$link = $this->findRequiredElement($xpath);
$this->getSession()->visit('view-source:' . $link->getAttribute('href'));
$content = $this->getSession()->getPage()->getContent();
$lines = explode(PHP_EOL, $content);
$this->csvRows = [];
foreach ($lines as $line) {
if (strlen(trim($line))) {
$this->csvRows[] = str_getcsv($line);
}
}
}
70. Zip File Download
@javascript
Scenario: View Large Report
Given I am viewing the reports area
When I click "Export" for the "Extremely Large Report" report
Then a zip file should be downloaded
When I unzip the file and I open the extracted csv file
Then I should see the following columns:
| column |
| Group |
| Company |
| Total |
71. File Download Test
/**
* @When /^I click "Export" for the "([^"]*)" report$/
*/
public function iExportTheReport($reportName)
{
$this->file = $this->getArtifactsDir() . 'download-' . time() . '.zip';
file_put_contents($this->file, $this->getSession()->getDriver() > getContent());
}
/**
* @Then /^a zip file should be downloaded$/
*/
public function aZipFileShouldBeDownloaded()
{
$header = $this->getSession()->getDriver()->getResponseHeaders();
$this->assertContains($header['Content-Type'][0], 'application/forced-download');
$this->assertContains($header['Content-Disposition'][0], "zip");
}
72. File Download Test
/**
* @When /^I unzip the file and I open the extracted csv file$/
*/
public function iUnzipTheFileAndOpenCsvFile()
{
$zip = new ZipArchive;
$unzipped = $zip->open($this->file);
$csv = $zip->getNameIndex(1);
$zip->extractTo($this->getArtifactsDir());
$zip->close();
$fileRef = fopen($this->getArtifactsDir() . $csv, 'r');
$this->csvContents = [];
while (($data = fgetcsv($fileRef)) !== false) {
$this->csvContents[] = $data;
}
fclose($fileRef);
}
73. Confirm Text In PDF
/**
* @Given /^I should see a PDF with the order total$/
*/
public function iShouldSeeAPdfWithTheOrderTotal()
{
$total = 'Order Total: ' . $this->orderTotal;
$this->getMainContext()->assertPageContainsText($total);
}
@javascript
Scenario: View PDF Receipt
Given I am viewing my order history
When I click "View Receipt" for an order
Then I should see a PDF with the order total
75. Testing a Command Line Process with Behat
Scenario: Test User Import With a Large Data Set
Given the system already has 100000 users
And there is a company with 10000 of the users assigned to it
And an admin has uploaded a spreadsheet for the company with 10000 rows
When the system has begun to process the spreadsheet
And I have waited 1 minute
Then the batch process status should be set to "Running" or "Completed"
And I should see at least 100 new users in the company
76. The System Already Has 100000 Users
/** @Given /^the system already has (d+) users$/ */
public function theSystemAlreadyHasUsers($numUsers)
{
$faker = $this->getFaker();
$userValues = [];
for ($i = 0; $i < $numUsers; $i++) {
$firstname = addslashes($faker->firstName);
$lastname = addslashes($faker->lastName);
$username = $faker->username . $i; //unique
$userValues[] = "('{$firstname}', '{$lastname}', '{$username}')";
}
$userQuery = "INSERT INTO `user`(firstname, lastname, username) VALUES" .
implode(', ', $userValues);
$this->getEntityManager()->getConnection()->exec($userQuery);
}
77. There is a company with 10000 users assigned to
it/** @Given /^there is a company with (d+) of the users assigned to it$/ */
public function thereIsACompanyWithOfTheUsersAssignedToIt($num)
{
$company = $this->generateCompany();
$conn = $this->getEntityManager()->getConnection();
$userCompanySQL = "INSERT INTO `user_company`(user_id, company_id)
SELECT `user`.id, {$company->getId()} FROM `user` LIMIT {$num}";
$conn->exec($userCompanySQL);
$this->getEntityManager()->refresh($company);
$companyUsersCount = $company->getUserCompanies()->count();
$this->assertGreaterThanOrEqual($num, $companyUsersCount);
$this->company = $company;
$this->companyNumUsers = $companyUsersCount;
}
78. An Admin Has Uploaded a Spreadsheet
/** @Given /^an admin has uploaded a spreadsheet for the company with (d*) rows$/
*/
public function adminHasUploadedSpreadsheetForTheCompanyWithRows($numRows)
{
$faker = $this->getFaker();
$this->filePath = $this->getUploadsDirectory() . 'import -' . $numRows . '.csv';
$fh = fopen($this->filePath, "w");
$rows = 'firstname, lastname, username' . PHP_EOL;
for ($i = 0; $i < $numRows; $i++) {
$firstname = addslashes($faker->firstName);
$lastname = addslashes($faker->lastName);
$username = $faker->username . $i; //add $i to force unique
$rows .= "{$firstname}, {$lastname}, {$username}" . PHP_EOL;
}
fwrite($fh, $rows);
fclose($fh);
$repository = $this->getRepository('BatchProcess');
$this->batchProcess = $repository->create()->setFilename($this->filePath);
$repository->save($this->batchProcess);
79. The System Has Begun To Process The
Spreadsheet/**
* @When /^the system has begun to process the spreadsheet$/i
*/
public function theSystemHasBegunToProcessTheSpreadsheet()
{
$command = 'php app' . DIRECTORY_SEPARATOR;
$command .= 'console batch:process --batch_id=';
$command .= $this->batchProcess->getId();
if (substr(php_uname(), 0, 7) == "Windows") {
return pclose(popen("start /B " . $command, "r"));
}
return exec($command . " > /dev/null &");
}
80. I Have Waited 1 Minute
/**
* @When /^I have waited (d+) minutes?$/
*/
public function iHaveWaitedSomeMinutes($num)
{
$seconds = 60;
$outputEvery = 30;
$cycles = ($num * $seconds) / $outputEvery;
for ($i = 0; $i < $cycles; $i++) {
sleep($outputEvery);
echo '.';
}
echo PHP_EOL;
}
81. The Batch Process Status Should Be
/**
* @Given /^the batch process status should be set to "(.*)" or "(.*)"$/
*/
public function theBatchProcessStatusShouldBeSetTo($statusA, $statusB)
{
$this->getEntityManager()->refresh($this->batchProcess);
$statusName = $this->batchProcess->getStatus()->getName();
if ($statusName !== $statusA && $statusName !== $statusB) {
throw new Exception("Status is currently: {$statusName}");
}
}
82. I should see at least 100 new users
/**
* @Then /^I should see at least (d+) new users in the company$/
*/
public function iShouldSeeAtLeastNewUsersInTheCompany($num)
{
$company = $this->company;
$this->getEntityManager()->refresh($company);
$companyNumUsersNow = $company->getUserCompanies()->count();
$originalNumUsers = $this->companyNumUsers;
$difference = ($companyNumUsersNow - $originalNumUsers);
$this->assertGreaterThanOrEqual($num, $difference);
}
Behaviour-driven development is an “outside-in” methodology. It starts at the outside by identifying business outcomes, and then drills down into the feature set that will achieve those outcomes. Each feature is captured as a “story”, which defines the scope of the feature along with its acceptance criteria.
Dan North, “What’s in a Story”http://dannorth.net/whats-in-a-story
It’s the idea that you start by writing human-readable sentences that describe a feature of your application and how it should work, and only then implement this behavior in software.
Behat Documentationhttp://docs.behat.org/en/v2.5
While unit testing tells us that the code is correct, acceptance testing tells us the feature is correct.
Because these “tests” deal with such larger components and typically how they interact together, they are forced to be slower to write, slower to run, and more work to maintain compared to unit tests
BDD is not for describing implementation details - it is for communicating and documenting intentions
BDD acceptance tests can be used to test the UI - but that is not the only thing it’s used for, and not a primary goal.
So, if we’re not going to get well design code out of BDD, or an automated test suite to prevent bugs, What is the purpose?
Software delivery is about writing software to achieve business outcomes. It sounds obvious, but often political or environmental factors distract us from remembering this. Sometimes software delivery can appear to be about producing optimistic reports to keep senior management happy, or just creating “busy work” to keep people in paid employment, but that’s a topic for another day.
Usually, the business outcomes are too coarse-grained to be used to directly write software (where do you start coding when the outcome is “save 5% of my operating costs”?) so we need to define requirements at some intermediate level in order to get work done.
Behaviour-driven development (BDD) takes the position that you can turn an idea for a requirement into implemented, tested, production-ready code simply and effectively, as long as the requirement is specific enough that everyone knows what’s going on. To do this, we need a way to describe the requirement such that everyone – the business folks, the analyst, the developer and the tester – have a common understanding of the scope of the work. From this they can agree a common definition of “done”, and we escape the dual gumption traps of “that’s not what I asked for” or “I forgot to tell you about this other thing”.
This, then, is the role of a Story. It has to be a description of a requirement and its business benefit, and a set of criteria by which we all agree that it is “done”.
Last line from “In a story”:
“This, then, is the role of a Story. It has to be a description of a requirement and its business benefit, and a set of criteria by which we all agree that it is “done”.”
role of a Story
description of a requirement
its business benefit,
and a set of criteria by which we all agree that it is “done”.
To determine the benefit, use 5 whys.
Too much context is distracting and confusing - if it doesn’t matter, leave it out
Not enough context causes assumptions - if you can get a different outcome with the same givens and whens - some context is missing
In the example, the first scenario says something about the account balance, the card and the ATM itself. All of these are required to fully define the scenario. In the third scenario, we don’t say anything about the account balance or whether the ATM has any money. This implies that the machine will retain the card whatever the account balance, and whatever the state of the ATM.
If you’ve used Behat before, you’ve probably done the steps of writing out the .feature file, running it, then using the regular expression and function that Behat generates for you.
You may or may not have noticed that the annotations Given, When and Then don’t really matter in front of that regular expression. You have to have one of them, but it really can be any of them and Behat will match the regular expression. So what do they mean, and why do they matter? These keywords describe the state of the application and your interaction with it.
First start writing .feature files, likely to write them like basic examples in tutorials.
Writing an application to broadcast a video presentation
Every broadcast has a specific time at which it should start playing.
Write feature describing broadcasts having countdown timer before scheduled start time.
Click
Process
Author describes Feature with implementation specific example
Developer adds fixture data
Issues
Test only works before 6pm
What is the intent?
Confusion for business users
There are several Drivers that come with Behat. Drivers communicate with other libraries that either power or emulate a browser.
There is a great chart in the Mink documentation outlining what each driver can do. You can find it on Behat’s site, by going to the documentation and looking at the Drivers section.
For the most part, you’ll probably end up picking one driver as your main driver, and using another for special cases, if at all.
The Mink Driver is a go-between that standardizes the various methods on each of these other drivers for things like page navigation. There are two main types of these drivers: headless browser emulators and real browser controllers.
Since a browser emulator does not actually launch an instance of the browser, but just simulates it, they are much faster than the real browser controllers. They also have access to more HTTP information like response headers and status codes.
However, most browser emulators can’t execute JavaScript, tell you if an element is actually visible on the page, or interact with the browser window, for example switching tabs or windows when a link opens a new one. These actions can be done with the browser controllers.
Hooks are annotations you put on a method that can trigger these methods to be called at a specific point in your test suite. You can hook into events Before and After the entire test suite, every feature, every scenario and every step.
Here is an example of using the AfterScenario hook to capture and save a screenshot upon a test’s failure.
If the event result was failure, we use the driver to take a screenshot, and save it into a specific directory we’ve configured in our context.
You can add tags to limit a hook to run only when it matches that tag. For example, the getScreenshot function only works with actual browser drivers, not the emulators. So this function will throw an exception when triggered in the browser emulator. We can add the javascript tag to the hook so that it only gets triggered after Scenarios that also had the javascript tag
You can have multiple tags on a hook. Here I’ve added the @screenshot tag.
If your application has user interactions that depend on AJAX, you’ll need to add some logic to handle this. In order to ensure that an AJAX call triggered by a previous step was completed before starting the next step, I’ll add a beforeStep hook that waits a reasonable amount of time for any previous step’s AJAX to complete. If the AJAX takes longer than this time, that’s a signal there is a problem as well.
Once again, we’ll use the @javascript tag to tell Behat to only use this hook when we can use javascript.
This code basically checks if jQuery has an active request and waits for it to be completed. The code for angular is a lot longer, so I haven’t included it here, but if you would like to see it, it is available in a gist located at this url, so you can look that up later once these slides are online. These are the only two javascript frameworks I have personally done this in, but I’m sure it’s possible for many others.
Another great use for hooks is to load your fixture data. Your tests should be written so that they can run independently of each other, so if for example one test inserts a user, another test should still be able to pass with or without that user existing. But sometimes you'll want to reload the database before or after tests that cause any database changes. I like to use the Doctrine Data Fixtures library to do this, since we're using Doctrine in our software. You could do this without Doctrine, by just having a sql dump of your database schema and some core data, but the Doctrine Data Fixtures makes it much easier to add new data during development and quickly make changes.
To create some fixture data, you'll create some files with the data you want, using your models, and then persist the data as usual in Doctrine. This should just be the basic data you need to run your application like an Admin user, and data that doesn't typically change, such as states and countries.
Once you have several files with fixture data, you can set up a tagged hook to load the database. This will purge the existing database and reload the data, using the existing schema.
This is probably best as a Before Feature hook, or even a Before Suite hook, but if you need to, you could set this up to run on every scenario - but it would slow your test suite down a lot.
However often you reload your database, just make sure you write your features so they can run independently of each other, and in any order.
For every statement in a Scenario, there must be a method in a Context class that has a matching regular expression. This is called the Step Definition.
I’ll be honest, before I started using Behat, I could not write a regular expression to save my life. So I would always just copy and paste the example Behat gave me in the terminal and use it. But as I wrote more and more features, I started having steps that were similar enough that they could use the exact same PHP code, but I still wanted the step definition to be slightly different.
That’s when I discovered that you can have multiple regular expressions and definitions for a single function.
This is a really simple way to reuse methods, and keep your Contexts smaller and maintainable.
If you are familiar with regular expressions, you’re probably familiar with the case-insensitive flag, /i. You can apply this to your Step Definitions like this. I use this one a lot.
The downside to this is that while Behat will match it and run the step with no problems, if you’re using an IDE like PHP Storm and you use this flag, PHP Storm will tell you that a step is undefined and it can’t navigate to the definition. So that can be a little confusing or annoying. If you want it to be able to match within the IDE you can use the inline modifier instead:
The inline case insensitive modifier (paren, question, i, close paren) makes everything after it case insensitive, so either put it at the beginning to affect the entire step, or before the words you want case insensitive.
When you’re using Behat to generate your Step Definitions based off the feature file, you designate a string variable by putting quotes around it. Numbers are made variables automatically.
It’s really easy to change these to be variables without having to use quotes, which I think looks nicer in a lot of cases. I’ve removed the quotes, and changed regex to capture anything between the quotes to just capture anything.
You can also be more specific about what can be matched, like limiting it to a list of options, using the pipe within a group.
If you have multiple definitions for the same function, you might want to make the one or more arguments be optional. This basically works like any normal PHP function. Add a default value for the argument, and then handle the variable appropriately in the code.
The last regular expression tip I want to share is using non-capturing groups. This is great for consolidating multiple definitions when the words that differ don’t matter to the definition. We can turn this multiple line definition into one line with a non capturing group.
Here is the single line definition, using a non-capturing group by adding question mark, colon to the beginning of the group.
In Behat 3, a new way to define steps without regular expressions was added. In Behat 3, you can use tokens instead of regular expressions. Tokens are like named placeholders that are much simpler than regular expressions.
You can still use regular expressions however for more specific matching.
Behat 3 introduced some other changes in step definition
Once you have more than a few tests with custom step definitions, the FeatureContext file might get too big to easily maintain. You can break them out into multiple context files.
Context organization changed in Behat 3.0, so first I'll go over the version 2 way to do it, then we'll look at the version 3 way.
In version 2 of Behat, to use multiple contexts, you have your main FeatureContext and then everything else is a subcontext. These must be added to the Main Context by calling useContext. You pass in an alias for the sub context, then an instance of the subcontext class.
Every time you want to add a new Sub Context, you add it this way, so as you add more, this might get lengthy.
Because I like to be able to add new Contexts as often as I need to, and I like things to happen pretty automatically for me, I wrote a function for my test suite that adds any Context file in my Contexts directory as a sub context with an Alias so I don’t have to add it every time. This is basically to handle the task of registering the subcontext into the MainContext with an alias, and it assumes the file name is the same as the class name.
I'm using Symfony's Finder component to find all files ending in Context.php in the current directory, and this is recursive through all directories within this one. I've excluded two classes from from loading as a subcontext, one is the FeatureContext, another is a class called CoreContext. I like to have all of my Contexts extend a base class, where I put all of my assertion methods. You may be familiar with assertions from Unit Testing. Behat does not come with an Assertion Library, but lets you import an existing one, for example from PHP Unit, or write your own. I chose to write my own so that I could control the error messages and add some custom methods too, so I put all of these in the CoreContext. Using an existing library is always a great idea though, so definitely use one if it works well for you.
So now I can loop through the files that are actual sub context files. I get the class name, because in our codebase the file name is always the same as the class name, so I can just get the base file name excluding the file extension and that will be the class name.
Because this is recursive and we could have subdirectories and therefore some more namespaces, I've added a little bit of logic to add the namespaces for each directory, again assuming they're the same as the file structure.
Finally, use Reflection Class to instantiate the object and add it to the MainContext using useContext. I could generate an alias based off the name, but for this example it's even easier if I just use the same name as the alias.
In addition to the assertions, I also use my CoreContext class to define helper methods that I want all of my Contexts to be able to use. For example, in this code, I am looking for an element with a specific class, then getting the text of it to compare to the expected message.
If the code that displays that error message on the actual site is faulty, and no error message is found at all, the call to $this->getPage()->find() will actually return null. And what happens when you try to call a class method on null instead of on the object?
You might want a shortcut wrapper that finds an element, and automatically throws an exception if it doesn’t exist. This is really useful for preventing fatal PHP errors in your test suite, which is what will happen if you try to use an HTML element that doesn’t exist. This is especially bad because it kills your test suite - obviously, it’s a fatal error. So instead of having to check after each find to make sure the element was found, I use this method to find an element and throw an exception if it isn't found.
So now we just need to switch the basic find from the page to our custom method available from CoreContext, and this will prevent that fatal error.
An important thing to keep in mind if you extend a base Context class in your suite is that you cannot put any step definitions in it, or Behat will complain about redundant step definitions. That means no functions with annotated regular expressions.
Once you have written a few scenarios, you’ll probably run into a situation where you need to reuse multiple steps in the same order in a few scenarios to describe a common behavior. For example, let’s say you’ve written an application that includes the ability for the user to upload a csv file, then designate columns as properties on the User model, and use it to import User data. Here are some scenarios to describe the behavior of your new User Import Process.
(read scenarios)
So we have our first scenario that describes the first interaction in this process - uploading a file and seeing a file review screen. The second scenario describes the second interaction, where it’s Given that we have already uploaded a file, and we are now interacting with the review screen.
In a lot of cases, our Given can simply manipulate data behind the scenes, for example manually inserting data into a database. For this scenario we could mock the file upload process and force a post to that page - but sometimes it’s just easier to run through the steps again. We could write the steps again in the .feature file, but it’s not as elegant to read. So we can write this one new step that executes the first couple of steps from the first scenario.
The first method for doing this is called “Meta-Steps” - this is when your step defintion returns a Step Object or an array of Step Objects. This will then kick off those steps after your step, as if you had written them in the feature file. This includes firing before and after step hooks.
When doing this, you don’t put the keyword such as Given in the string - you’ll notice the class you’re actually instantiating is that keyword. There is no But or And here, and just like in the actual .feature file, it really doesn’t matter which one you use.
This functionality was actually removed in Behat 3, so you'll only be able to use this in Behat 2, the last version of which is 2.5
If the step you’re including uses a multi-line argument like a PyString or TableNode, you can still use Meta-Steps, and create those objects and pass them in as arguments.
Another way to re-use those steps is to call the method directly. This is easy if the method is in the same context. When calling the method directly, you pass in the arguments, and if you need a TableNode or PyString you create them in PHP just like in the Meta-Steps method.
So, now that we have defined steps in multiple contexts, if we want to call one of those from another context, this is pretty easy to do in Behat 2. You refer back to the main context, get the subcontext, and then call the step you want.
If you ever move a step definition from one context to another, you’ll have to find the places it was called and update the reference to the sub context. You can typically do this with a simple find and replace. If you used a wrapper method for the subcontext, it may be even easier to do this using your IDE’s refactoring tools.
If you used a wrapper method for the subcontext, it may be even easier to do this using your IDE’s refactoring tools. Plus, this will allow your IDE to autocomplete your subcontext's method calls.
In Behat 3.0, you can now assign specific contexts into different test suites. This basically means you can specify exactly which contexts a specific suite of tests needs to use. This is done via the config file instead of in the code.
So this also means you'll have to plan ahead a little more to use steps from another context directly. You can only access the other contexts from Behat's Environment, which is only available within Hooks. So you'll need to make a Before Hook that stores the context, similar to the way Version 2 used to in the MainContext. This is based on the example code from the Behat documentation, where they are demonstrating how to get the Environment from the Score, then get the Context to store in a property, just updated for my specific example.
Now you can call that other method directly.
So your options for reusing steps within other steps are Meta-Steps or Calling the method directly. While Meta-Steps can be slower due to the hooks, it's often easier to use them when you're first starting and you're getting an idea of how to organize your suite, since it's easier to reorganize steps that have been used this way.
If you notice that your tests using meta-steps are running too slow, or you're using Behat 3, you'll want to switch to direct method calls for this.
So far in these examples we have interacted with our application in the step definitions by referencing specific page elements directly. There is an extension for behat called the Page Object Extension. Page Objects are a way to separate the UI information from these steps. This helps make it faster to make changes to your UI and update your related Behat tests by keeping the UI information in one place.
Installing the extension is easy, just require it through composer and add it to your Behat configuration.
There is a stable version out for Behat 2, and a dev one for Behat 3.
Let’s go back to our first scenario we rewrote, about the Seminar, and add one that describes the seminar. Once we've added some more scenarios with custom steps for the seminar, we could end up with multiple steps with xpath or css selectors for the same element. For example, the steps to view the seminar page for each seminar might duplicate the URL, or the steps for the countdown time would probably have the same css selector.
We can create a Page Object that represents this seminar page. You can use variables in the path for the page url.
We can then add some elements to represent each of the important elements we want to test. The video, the author’s information, the countdown timer, and the message.
Now our context can assert that the message is or isn’t there, and if we ever want to change the html or css for that message, we only have to update it here in the Page Object, and not in multiple step definitions.
If an element on a page is complex or has a lot of interactions, you can move it into a custom Element class.
Because PageObjects are instances of the Mink DocumentElement class, and Page Elements are instances of the NodeElement, you also have access to all the same methods they provide, just like doing a Session - Get page, or find/findAll in your Context, and getting a NodeElement.
One thing to keep in mind is that the PageObject should not be doing any assertions, it’s just representing the Page to abstract our your UI and return elements and other information about the page.
There may be behaviors you need to describe that might not seem so easy to test.
Let’s say your application offers some reports for admins to download about your sites users, and these are all available as CSVs.
We want to describe the behavior that determines when a user’s information ends up on this report, as well as what columns and data should show up, and confirm that the user did show up on the report.
Part of the difficulty here is that this is a file download, not a page you can easily view on a site. Maybe we could let the browser download the file, then find the file in the downloads folder and open it - but this wouldn’t work very well if anyone running the test used Firefox on their computer and had a different setting for the downloads folder location, or had the prompt to save or open the file turned on, etc.
For testing a CSV, there is a really simple hack for this. If you use Firefox as your browser with a real browser driver, you can simply add view-source: to the start of the URL, and now you can easily read the CSV. Be sure to add that @javascript tag to force this test to use your real browser driver.
Now let’s try a similar task with a much larger report, that has to be compressed, and the user will download a zip file.
This is a little more tricky. This feature requires you to confirm the file was a zip, download it, open it, and then read the CSV. For this one, we’ll want to switch to a browser emulator, like Goutte.
For the export step, we create a new temporary file, and write the content of the browser’s response into it. In the next step we can confirm the content type and content disposition as a downloaded zip file.
The next part is easy, use a PHP Library to unzip the file, then php functions to read in the CSV contents. Now we can work with the csv contents the same as if we had the csv directly in the browser like earlier.
Another type of file we work with a lot on our website is PDFs. Since Firefox can load a PDF directly in the browser, if you just need to confirm that specific text appears on the PDF, this is easy to do using the regular page content assertions.
You can also easily use Behat to test things besides browser interactions. Our application has some command line processes that run via cron jobs at regular intervals. We can actually use Behat to test the behavior of these commands on their own and within the application.
Here is an example of a feature that describes how one of our command line processes should be able to batch insert at least 100 new users within a minute. This test was written to catch regressions with some complicated code for this process - we ultimately decided 100 users in 1 minute was acceptable. To accurately test the conditions, we needed the database to already have a large number of users, and then test uploading a large amount of users.
The first step is going to insert some users to the database. This method generates some raw SQL with random user data thanks to the Faker library, gets the Doctrine Connection and executes the raw SQL.
This method could benefit from checking first how many users already exist and only inserting as many as we really need, but this is the simplified version.
(Explain addslashes use)
Next, we'll create a new company, and add some of those users to it. We're using a mix of Doctrine and raw SQL here. The generateCompany function uses the Faker library again to generate a Company object, using a custom provider for Faker. We then run an insert query using raw sql, and then refresh the company to get the users. Confirm they were inserted successfull, then save the company and the actual starting number of users into a private variable.
The third step needs to generate a CSV, put it in the right directory for the batch process to find it, and insert the appropriate data into the database that an admin user would have selected when uploading the file. We are using Doctrine to save the model of the Batch Process to the database.
Next, we kick off the actual process that the cron job runs. Since we support our developers using Windows, Mac or Linux, the test suite needs to work on any of them, so we have a little code here that kicks off the process one way for Windows, or another for anything else.
Next, we’ll wait some time. We could just run sleep the entire time, but I wanted to add some output so if we used this step for longer than 1 minute, the developer didn’t think the terminal had frozen or something.
Now the confirmation that everything worked - ensure the process was updated accordingly, not in a failed status basically.
Lastly, check that we did have at least the amount of users we expected. Here's where we are going to use that company object and original count we stored earlier. Once we refresh the Company object, some basic math will tell us if the process is working ok.
Hopefully now you have a better understanding of how truly powerful Behat is, and how you can better leverage it to do true behavior driven development in your own projects. Behat can be much more than basic examples of clicking on specific buttons and reading text off the page. You can use Behat to document and test your frontend, your API, your command line processes and more.
Thank you very much for coming, and please remember to rate this talk on Joind in.