What is UI Testing? How does it work with Xcode 7? What is XCTest? Comparison with Calabash

1. UI Testing in Xcode 7
@roland9
roland@kitmanlabs.com
Xcake Dublin, 13th October 2015

2. UI Testing
• Find UI elements
• Interact with the app the same way as user does
• Validate changes through assertions

3. Why would I do it?

4. XCTest
• Xcode’s testing framework
• Apple introduced it in Xcode 5 - keeps adding:
• Unit Tests
• Performance Tests
• UI Tests
• Supports Objective-C and Swift

5. How does it work?
• New target type
• Executed in separate process
• New:
• XCUIApplication: proxy for tested app
• XCUIElement: proxy for elements
• XCUIElementQuery: resolves to collections of
elements

6. Documentation
http://masilotti.com/xctest-documentation

8. Demo
• Using Shwopping http://shwopping.herokuapp.com
• Test target for UI Test
• Recording (new test; but also enhance existing test)
• Manually add assertions
• Debugging:
• print how query gets resolved:
• p print(XCUIApplication().tables.cells.textFields.debugDescription)
• Accessibility Inspector ⌘F7: highlight button, …
• Screenshot - quick look
• Code coverage

9. Queries - Filtering
• Filter by Element type (button, table, …)
• Filter by Identifier (label, title, …)
• Predicates (value, partial matching, …)
• Remember: Queries can be chained

10. Queries - Details
• Find your UI elements by combining
Relationships and Filtering
• Use .element property to get XCUIElement
• Elements must be unique (use .exist to check
whether elements exists)
• Queries are evaluated on demand; will be re-
evaluated when UI changes

11. Example
let app = XCUIApplication()
let table = app.tables["Shopping List"]
app.launch()
XCTAssert(table.cells.count == 4, "wrong number of
cells")
table.cells.elementBoundByIndex(0).tap()
app.buttons["Clear text"].tap()
table.cells.elementBoundByIndex(0).textFields.elem
entBoundByIndex(0).typeText("xcake")
table.swipeDown()

12. Wait for Element

13. Comparison Calabash
Calabash
• Adds server to your
iOS app
• View hierarchy is
exposed and events
are generated
• API, DSL (Ruby) for
writing tests
Xcode 7
• App runs in separate process
• UI elements & queries (relationship
& filtering); assert values
• Events are synthesised on low level
of OS
• Integrated into Xcode (test target,
recording, code completion, running
tests, code coverage)

14. Pros / Cons: Xcode 7
• Nice: fully integrated into Xcode (debugging,
code coverage, …)
• Access to advanced scenarios: pickers, tap links
in a webView, reordering of table cells
• Backgrounding / foregrounding -> test app
lifecycle
• Not really expressive: similar to writing unit tests
with Kiwi / Spectra vs. XCTest

15. Pros / Cons: Calabash
• More expressive - you write tests in DSL
• More flexible (e.g. backdoor feature; get view
hierarchy)
• But NOT integrated…
• Similar to Xcode view debugging & Reveal.app:
• Xcode has superior integration (auto layout)
• but Reveal has great features

16. References
• WWDC https://developer.apple.com/videos/play/wwdc2015-406
• Xcake Calabash http://www.slideshare.net/roland99/ios-and-
android-acceptance-testing-with-calabash-xcake-dublin
• Automate https://krausefx.com/blog/run-xcode-7-ui-tests-from-
the-command-line
• Documentation http://masilotti.com/xctest-documentation
• Tips & Tricks http://masilotti.com/ui-testing-cheat-sheet
• App I used for demo: http://shwopping.herokuapp.com

17. Summary
• Recording is huge, but not perfect -> use it to learn
how to write tests
• Queries are evaluated on demand -> keep it DRY
• Relationships & filtering -> chain
• Integration into Xcode: code coverage, performance
testing, screenshots

19. UI Testing in Xcode 7
@roland9
roland@kitmanlabs.com
Xcake Dublin, 13th October 2015