This document discusses automating testing using Gherkin scenarios in an agile environment. It provides examples of goal, task, and action level Gherkins and discusses automating Gherkins across browsers and environments. It also addresses challenges such as documentation, traceability, and gaining organizational buy-in for the Gherkin approach.
2024: Domino Containers - The Next Step. News from the Domino Container commu...
05/2012 - Automating testing in the iteration
1. AGILE
AVENGERS
Automating Testing in the Iteration
Terry Wiegmann, CBAP, CSQE, PSPO, ACG
Ayan Dave, PMP, PSM, OCMJEA
Quick Solutions, Inc.
2. There was an idea to bring together a group of remarkable people,
so when we needed them, they could fight the battles that we never could...
Nick Fury
3. You put those people together, you can't expect
what's going to happen...
Maria Hill
4. Steve Rogers: Have you got a suit?
Hawkeye: Yeah.
Steve Rogers: Then suit up
5. WHY GHERKINS?
– Simple Text
– Shareable
– Some
exposure to
6. HOW DO WE AUTOMATE GHERKINS?
• Using Java, Maven, cucumber-jvm, gherkinsalad
• Goal, Task and Action Level Gherkins
• Behavior and Specification
• Automation for
– 3 browsers
– 3 environments
7. This is nothing we were ever trained for...
Natasha
Romanof
to
Hawkeye
8.
9.
10.
11.
12. WHAT IS NOT WRITTEN IN THE GHERKINS?
• In the gherkin we do not write “Then” statements for
– UI effects like color, css, tooltip
– UI effects like scrolling, animation
– Any expectation that never changes with different
“Given” contexts or “When” actions
• Verifying field lengths
• Verifying enabled / disabled fields
• Verifying field validations
15. RUN AUTOMATED GHERKINS
• Executing a gherkin in
– 3 browsers: Firefox, IE, Chrome
• Execution plan comes to the rescue
– 3 environments: DEV, QA, UAT
• Data file comes to the rescue
24. World Security Council: Director Fury, the council has made a
decision.
Nick Fury: I recognize the council has made a decision, but given
that it's a stupid ass decision, I've elected to ignore it.
25. DOCUMENTATION...
• Needed end to end scenarios- context for
features being requested
• For client testing
• For customer support
• For Trainers
• For Sales to demo product
28. JIRA, CONFLUENCE & THE PICKLE JAR
Pickle Jar
Context: Release
Point in time snapshot
Context: Feature/Component Context: Usage overview
and Acceptance Criteria
Design, specifications and history scenarios for this release
29. HOW IS THE CLIENT REACTING TO OUR
USE OF GHERKINS?
– Excitement
– Concerns about
Long term
maintainability
– How to get
everyone onboard
– QTP
– Gherkins for legacy
code
30. WHAT CHALLENGES DID WE RUN INTO?
– Traceability
– Maintaining
history
– Consistency
– Architect for
domain
knowledge
– Reusability –
suite of “global”
gherkins?
31. WHAT’S NEXT?
– More
automation
– More reuse
– More value
Terry Assuming we will have an introducer, I will compare and contrast us….humorous…..gender, nationality, education, different sets of initials after our names….Penny/Sheldon….Coke/Pepsi!The point being and yet, we are collaborating, leveraging, learning
Terry:QSI Team of PM, BA and 5 devs including a Dev LeadWorking at client locationExisting Teams working in silosWaterfall modelAyan:Internet Application, 3 browsers, devicesHighly motivated team of engineersNo TDD practice
Who are we? Terry:QSI Team of PM, BA and 5 devs including a Dev LeadWorking at client locationExisting Teams working in silosWaterfall modelAyan:Internet Application, 3 browsers, devicesHighly motivated team of engineersNo TDD practiceWhat are we doing?TerryAgile Team in a Waterfall EnvironmentShow and TellsIdentify and Engage PO’sPutting ATDD practice in placeNot the record and play AutomationAyanWould like to do TDD, but using junit extensively takes more timeWith limited and evolving understanding of the system gherkins are a good choiceTDD & Taxation Analogy
TerryAgile Team in a Waterfall EnvironmentShow and TellsIdentify and Engage PO’sPutting ATDD practice in placeNot the record and play AutomationAyanWould like to do TDD, but using junit extensively takes more timeWith limited and evolving understanding of the system gherkins are a good choiceTDD & Taxation Analogy
TerryGherkins written at the start of feature development, tested at the endAll items in the definition of done can me mentionedHelps increase collaboration levels
We each brought what we could and just jumped in …. Skillsets, experience.It’s about the Goal, not the Role
TerrySimple to Read, Easy to ShareProvides a clear context, event and expectationsCaptures the behavior of the systemCaptures the business requirementsGoals Tasks ActionsAyanCan be automatedDocumentation gets out of sync and code is the system of truth TDD at a different level
We use Java, maven, WebDriver SeleniumLeverage Existing J2EE expertise
Ayan:Overview – online educationDiscussion thread, teachers and students
We keep the features in SVNApart from features we have execution plan and data files
Data File for each environment to enable execution of gherkins in different environments
Execution plan to tie the gherkin, environment url, browser and data file all togeatherOne execution plan per release
Screenshots taken with each action
Reports generated at the end of execution
Terry: Do/Improve, show PO something to get responseAyan:Definition of Done Coded Reviewed Unit Tested Executed Integrated RefactoredAutomation completed
Terry:Handoff to QA Team QTP licensesTheir integration testing with other productsWhen we add a gherkin to Dev, need to move it forwardAyan:Although not limitation of waterfall the emphasis was to just finish off the featureNow automation plays a role in definition of doneEven then we had a few instances where we had to cut down on time spent on automationSprint 0 for that
Terry:Code is the truth, but gherkins not accessible to everyone who needed them and difficult to string together in end-to-end scenarios.Gherkins in SVN may cause accessibility issuesGot feedback from other users of gherkins that long text file with a list of scenarios was difficult to follow, read and useAyan:Building on top of the cucumber syntax can be troublesome with little space, formatting issues etcAs much the idea of gherkins was welcomed there were more questions and concerns on “how much information to keep here"
Terry:Artifacts created by BA’s that are very usefulPossibility for automationA scenario for each transition may be createdAyan:Well, Visio cannot export this out as XML but this is a possibility.Perfect example where documentation may get out of sync and ATDD on such diagrams could be useful
Terry:Explain this format?Advantages over using feature files in SVN?Ayan:And this can be automated too.Plain Text .feature file can be generated at run time from this excel using a few groovy scripts and libraries like Apache POIFeed in these files to the framework of choice and the scenarios are automatedPossibility: The data file, execution plan, page structure may be consolidated out to this single file as well.
Terry:How do all these things tie together?
Terry:All of the aboveAyan:We are doing a lunch and learn series with the client QA team to explain and show what gherkins areHelping them write gherkins at task levelOutside In vs Inside Out approach
Terry:Requirements TraceabilityWriting gherkins ConsistentlyArchteching for domain knowledge – what to assume is generally known vs what to specify; e.g., 4000 charactersAyan:The power can be the curse: Gherkins can be written in plain english and can be automated, but this can cause problems with maintainenceGherkins are written for strictly technical tasks that can be automated without launching the browser. How to keep them separate yet in syncSeems like a need for a DSL on top of gherkins specific to an industry
Terry:Investigating the possibility of exporting from Visio to ExcelAyan:
Terry:Ayan:
Embrace and respond to change byAdopting the practices thatManifest the principles thatSupport agile values