This document provides an overview of automated testing in AngularJS, including unit testing, end-to-end testing, and acceptance testing using tools like Protractor and CucumberJS. It discusses the benefits of automated testing such as enabling safe refactoring and reducing bugs. It then demonstrates how to set up testing frameworks like Protractor and Karma and write tests using page objects and test-driven development. The document also covers acceptance testing with CucumberJS by writing step definitions and features in Gherkin and linking them to tests.

1. AUTOMATED TESTING
IN ANGULARJS
WITH JIM LYNCH
AUGUST 2016

2. OVERVIEW
▸ Why write tests?
▸ UI / E2e Testing
▸ Unit Testing & TDD
▸ Acceptance Testing

3. WHY USE AUTOMATED TESTING AT ALL?
IF YOU DON’T HAVE AUTOMATED TESTS…
▸ Refactoring becomes dangerous since you can break
something and not be alerted of it.
▸ The team must rely on heavily on manual testing to ensure
quality which is expensive and lengthens each iteration.
▸ Anyone reading your code has nothing to help him or her
understand the code besides that source code itself.
▸ Every push to production leaves you with fear and worry in
the back of your mind that something might go wrong.

4. “OUR HIGHEST PRIORITY IS TO SATISFY
THE CUSTOMER WITH EARLY AND
CONTINUOUS DELIVERY OF VALUABLE
SOFTWARE”
#1 Principle of Agile Manifesto 12 Principles
IT ALL COMES BACK TO THIS

5. MANUAL TESTING
▸ Can you think of a manual test that can’t be automated?
▸ Manual Testing is the Devil!
▸ Quickly becomes expensive, error prone, time consuming…
▸ Don’t rely only on “____ serve” for development because
then you need to test manually!
▸ Used only for exploratory / usability testing and
should NOT be a step in the deployment process.
AVOID IT!

6. PROTRACTOR

7. SETTING UP PROTRACTOR
‣ Create a config file:
‣ You might also need to update webdriver-manager:
‣ Install the library:
‣ And then run it:

8. AN EXAMPLE OF A PROTRACTOR TEST

9. PAGE OBJECTS
▸ These are the mysterious .po.js files.
▸ They allow you to separate the DOM selection code from
the core test logic.
▸ Existed way before Protractor was even a thing. Tons of
info has been written on the web about page objects.
▸ Example:
KEEP YOUR UI TESTS MAINTAINABLE!

10. PROTRACTOR IS AWESOME!
▸ Test on virtually any browser via Sauce Labs!
▸ Allows you to control Selenium WebDriver with friendly,
JavaScript API.
▸ Tip: Start with functional testing of user interactions
and flow through the app.
▸ Keep it real! Instead of mocking backend services, just
run your tests on the real app and call it e2e testing too!
WE ARE SO LUCKY.

11. TEST ON ALL THE BROWSERS WITH SAUCE LABS!
ALL THE THINGS!

12. PROTRACTOR REPORTS
Install the package:
And then add this to your protractor config file
exports.config object:
Then on every
run you’ll get a
generated
report with
screenshots!
FOR WHEN YOU CARE WHAT ACTUALLY HAPPENED

13. KARMA

14. SETTING UP KARMA
Install Karma:
Install any plugins you’ll need:
Create a
config file:
And run it like this:

15. AN EXAMPLE OF A UNIT TEST

16. THE CODE COVERAGE REPORT

17. TEST DRIVEN DEVELOPMENT

18. IF YOU WANT TO REFACTOR,
THE ESSENTIAL PRECONDITION
IS HAVING SOLID TESTS.
Martin Fowler
author of Refactoring: Improving the Design of Existing Code

19. TDD: WHAT IS IT?
▸ TDD is all about writing unit tests for your source code.
▸ When you write the failing test first and then make it pass
with source code you can be confident that the test is
really covering the piece of functionality that it should.
▸ Ideally you should avoid writing testless lines of code.
▸ Write the tests first, inventing the names of controllers,
services, and methods in your tests. Then generate them
(or create them by hand), and implement them in a way
that makes the tests pass.
LET THE TESTS DRIVE THE DEVELOPMENT (DUH)

20. WHY IS TDD SO AWESOME?
▸ Unit tests can expose runtime errors immediately after
every change of the code which helps to keep quality high
and prevent regression.
▸ Seeing a unit test fail, making a change in the source, and
seeing it pass helps show that the test is indeed checking
what it should.
▸ You don’t have to “go in after you’re done” and add in
unit tests.
▸ It encourages prolific refactoring with confidence.
TDD: YOU SHOULD LOVE IT
▸ You appreciate the importance of DELETING code.

21. TIPS FOR TDD & UNIT TESTING
▸ TDD is hard. Getting into the right mindset comes with practice.
▸ Try “Reverse TDD” (green-red-green-refactor) or “Transplant TDD”.
▸ Use mocks and spies. Check only one thing per test.
▸ Imagine testing each of your Angular objects in a black hole in
outer space separately from the rest of the application.
▸ Don’t worry about the UI details. Focus on testing the functions,
logic, and calculations of your source code.
▸ Focus on explicitly calling your “public methods” which in turn will
use the private methods to do it’s work.

22. THE HOLE IN PURE TDD
▸ You can sometimes find that it’s unclear “what test to write
next”.
▸ TDD Tests only that that software is free of errors but doesn’t
necessarily test that it provides real business value.
▸ It’s unclear how or when ui testing tools like Protractor fit
into the TDD development cycle.
▸ It says nothing about how to actually turn the requirements
into unit tests.
WHAT’S MISSING WITH TDD?

23. WOULDN’T IT BE NICE IF:
▸ we could save our requirements in source control also have them
accessible, readable, and easily understandable for all
stakeholders?
▸ the requirements were able to be run against our
codebase quickly and easily, recording which ones passed
and which ones failed?
▸ recordings could be visualized in a report similar to a
code coverage report but for show if our requirements
tests are passing?
IF ONLY WE HAD TDD FOR OUR REQUIREMENTS

24. YES!
IT IS A THING!

25. TERMS THAT ALL REALLY REFER TO THIS SAME IDEA:
▸ Specification by Example
▸ Behavior Driven Development
▸ Acceptance-Test Driven Development
▸ Agile Acceptance Testing
WHAT SHOULD WE CALL IT?

26. “SPECIFICATION BY
EXAMPLE IS TDD APPLIED TO
BUSINESS PROBLEMS”
Gojko Adzic
from Specification by Example

27. OK,
SHOW ME AN
EXAMPLE.

28. A SNIPPET OF GHERKIN CODE

29. GHERKIN
▸ Meant to be easily readable by non-programmers.
▸ Aids in building a common understanding of the requirements
between all parties and is accessible to non-programmers.
▸ Becomes incredibly nice as documentation and progress
reporting once the project begins.
▸ Meant to be written during a “Three Amigos” meeting.

30. Scenario: eat 5 out of 12
Given there are 12 cucumbers
When I eat 5 cucumbers
Then I should have 7 cucumbers

31. Scenario: Adding items to basket
Given I have no items in my shopping cart.
When I add 2 pumpkins
And remove two pumpkins
Then I should no items in my shopping cart.

32. FOR TEAMS PRACTICING BDD, THE
REQUIREMENTS AND THE EXECUTABLE
SPECIFICATIONS ARE THE SAME THING.
John Ferguson Smart
from BDD In Action

33. OK... BUT HOW IS AN
ENGLISH SENTENCE
“EXECUTED”?
A. STEP DEFINITIONS

34. STEP DEFINITIONS
▸ Uses a Given-When-Then syntax which separate the test logic into setup
code, action code, and assertion code, respectively..
▸ The Gherkin is mapped to a
step definition function that
uses a regex to match the
Gherkin text.
▸ These functions then
implement what the Gherkin
says should happen and
checks these business
requirements against the
source code.
MAPPING GHERKIN STEPS TO JAVASCRIPT FUNCTIONS

35. BUT HOW DOES ONE SIMPLY
“IMPLEMENT A STEP DEFINITION”
IN ANGULARJS?

36. UNDERSTANDING CUCUMBERJS
▸ Instead of running the acceptance tests through cucumber
itself, we can “cucumberize” or existing test runners.
▸ The most common JavaScript acceptance test running
framework is called, “Cucumber-JS”.
▸ Running CucumberJS itself might be fine for a NodeJS
project, but AngularJS testing requires special setup that
CucumberJS does not know how to do.

37. CUCUMBERIZING YOUR TEST RUNNERS
▸ For Karma I recommend, “karma-cucumber-js”:
https://github.com/eugene-sea/karma-cucumber-js
▸ Karma and Protractor both know how to execute gherkin.
▸ Both have a “frameworks” option that gets changed
▸ For Protractor, check out “protractor-cucumber-framework”:
https://github.com/mattfritz/protractor-cucumber-framework

38. CUCUMBERIZING PROTRACTOR
Install the library via npm:
And then edit the framework parameter in your config block:

39. RUNNING CUCUMBERIZED PROTRACTOR TESTS
▸ You can also set up gulp / grunt tasks for acceptance tests.
▸ Tell Protractor to only
look for the files with
BDD suffixes
like .feature
or .step.js
▸ Run the Protractor executable and pass in your config file:

40. CUCUMBERIZING KARMA
‣ Notice that this project uses a slightly different syntax
from Protractor step definitions:
▸ I recommend “karma-cucumber-js” (by eugene-sea) and not
“karma-cucumberjs” (works with newer versions on cucumberjs).

41. WRITING KARMA STEP DEFINITIONS
▸ The Then is where you do the assertion that what you
expected to happen did happen.
▸ The Given is similar to your “beforeEach” regular unit
tests. It is where you can inject Angular objects and create
mocks.
▸ The When is where you do some action, in this case
normally calling a public method of some controller,
service, etc.

42. THE CUCUMBER “WORLD”
▸ Cucumber has the concept of a “World object” that is
somewhat of a global object for your acceptance tests.
▸ You can set frameworks like chai to the World object,
allowing you to use chai API in your step definitions.
▸ Depending on your cucumber framework it’s either an
object or a function that you can implement.
▸ It’s purpose is to allow you to define “utility properties”
that can be used in step definitions.

43. WHY YOU SHOULD ALSO STILL USE YOUR NON-
CUCUMBERIZED TEST RUNNERS
▸ If you’re worried about having two separate code coverage
reports, you can combine them with “Instanbul-combine”.
▸ Use cucumberized karma when you don’t need the
UI to test the BUSINESS CASE of the scenario.
▸ It’s important to be able to increase your code coverage
without needing to write “boring scenarios”.

44. TEXT
A CUCUMBER REPORT EXAMPLE

45. GOOD REPORTS ARE KEY TO SUCCESS
▸ Cucumber reports allow you to see the progress of the
application at any point in time.
▸ The reports should be seen as incredibly useful.
▸ Reports are all just-in-time generated (ideally from a
build server) giving everyone up-to-date reports
available at any time without the devs ever needing to
“write documentation”.
▸ Code coverage report lets you know what has’t been
covered by unit tests.
YOUR LIVING DOCUMENTATION!
▸ Great for the BAU team trying to understand your code.

46. THE MOST SUCCESSFUL TEAMS DISCOVERED
THE LONG-TERM BENEFIT ON
IMPLEMENTING SPECIFICATION BY EXAMPLE
COMES FROM LIVING DOCUMENTATION.
Gojko Adzic
from “Specification by Example”

47. A LOT OF PREPARATION GOES A LONG WAY
▸ But it’s worth it!
▸ Jenkins pipeline to run all of these tests, hosts the
reports, and deploy the final build.
▸ Creating a cucumberized Protractor config file.
▸ Creating a cucumberized Karma config file.
▸ Learning and understanding gherkin / step definitions.
CAN WE START YET?
▸ More complicated file globbing patterns.

48. TEXT
EXAMPLE OF A TRAVIS.YML FILE

49. BDD WITHOUT CUCUMBER
▸ Some downsides to not cucumberizing your runners
include:
▸ You can still practice some form of Behavior Driven
Development even without cucumberized test runners.
▸ You can still have “Three Amigos” conversations about the
requirements and even create Gherkin feature files without
executing them.
KEEP AN ACCEPTANCE TESTING MINDSET
1) You don’t get reports (ie living documentation).
2) You don’t know if your code passes or fails the scenarios.
3) You don’t have explicit step definition functions.

50. ADDITIONAL BENEFITS OF ACCEPTANCE TESTS
▸ Misunderstandings get thrashed out early.
▸ Better team communication and ubiquitous language.
▸ Requirements and code are always in sync
▸ Removes the manual testing bottleneck.
▸ JIT documentation for development and beyond.
IF YOU’RE STILL NOT CONVINCED
▸ It helps developers think more in the business domain
than just falling back to computer science patterns.

51. BUT WE DON’T HAVE TIME FOR TESTING
▸ Remind him that with the all of the rework, regression, and
manual testing it could easily take longer to build the
project without tests.
▸ Remind the boss that this is a long-term project, and the
reports increase in value as the project ages.
▸ The 99% perfect rule: Nobody wants 99% perfect
software. If it’s not bug-free then it’s not really finished.
▸ Is it that your team doesn’t have time, or is it that they just
don’t know how and no one wants to pair program on it?
BUILD IT QUICKLY VS BUILD IT FOR LONGEVITY

52. HIGH QUALITY -> HIGH PERFORMANCE
PERFORMANCE TESTING
▸ Manual performance testing available through Chrome
dev tools, WebPageTest, Google Pagespeed…
▸ Command line utility “Benchpress” that can be used for
getting stats on and benchmarking any web application.
▸ In general, favor automated performance tests for the
same reasons we automate our other tests.
▸ You could even host your benchpress report(s) on every
build as part of your living documentation!

53. WHERE DO I PUT ALL OF THESE FILES?
▸ Avoid parallel directory structures
or you might find yourself with
“scrolling like mad syndrome”.
▸ It usually makes sense to work
on one component at a time,
and it’s nice to have all the
things relating to that
component in one place.
DIRECTORY DIVING

54. IT’S “LIVING” DOCUMENTATION, SET IT LIVE!
THE IMPORTANCE OF HOSTING YOUR REPORTS
▸ Without hosting your reports at a url for everyone to see,
no one on your team can see the reports!
▸ Reports should NOT be generated and hosted manually.
Generating the reports (by running the tests) and hosting
them should be steps in your CI pipeline.
▸ As a CEO or business leader, if you can create a culture
that highly values and utilizes these three types of reports,
everything else will* fall into place.
▸ Access to up-to-date reports for non-programmers is
CRITICAL for them to do their job properly.

55. GREAT BOOKS ON BDD / AUTOMATED ACCEPTANCE TESTING

56. UGAT
github.com/JimTheMan/AngularJS-UGAT

57. AN EXAMPLE PROJECT
OFFICIAL UGAT PROJECT: NG-NJ

58. THANKS!!
Send any questions to twitter.com/webWhizJim
(or just ask them right now!)