Deploy distributed systems faster and safer with contract tests It’s 2018 and we still rely on integrated environments and large end-to-end test suites to release complex ecosystems, also called “software". In this talk, Matt breaks down the arguments for such nonsense and provides a better, faster, safer alternative. From https://www.meetup.com/sfjava/events/255379906/

1. PRESENTED BY Matt Fellows (@matthewfellows)
Test S-M-R-T not HARD
Deploy distributed systems faster with contract tests
and safer!
v

2. ABOUT ME
2
1. Principal Consultant at DIUS
2. Pact since 2014
3. Would work in sports/fitness if
not IT
@matthewfellows

3. A story

4. Soundify
®
A STORY

5. Everyone is doing microservices APIs
A STORY

6. A STORY

7. Metric Value
No. teams 1
No. components 1
Test environments 1
Time in pipeline (commit to prod) 5 minutes
Risk 2.5%
Deployments per day 10
A STORY > CD HEALTH CHECK

8. A STORY

9. Metric Value
No. teams 2
No. components 2
Test environments 4
Time in pipeline (commit to prod) 10 minutes
Risk 5%
Deployments per day 10
A STORY > CD HEALTH CHECK

10. A STORY

11. A STORY

12. Metric Value
No. teams 3
No. components 6
Test environments 18
Time in pipeline (commit to prod) 30 minutes
Risk 10%
Deployments per day 1
A STORY > CD HEALTH CHECK

13. ACQUIRED
A STORY

14. A STORY

15. Metric Value
No. teams 10
No. components 20
Test environments 20 200
Time in pipeline (commit to test) 120+ minutes
Risk 20%
Deployments per month 1
A STORY > CD HEALTH CHECK

16. Lessons?

17. So what did we learn?
A STORY > LESSONS

18. A STORY > LESSONS

19. Want: O(1) or O(n)
Got: O(n2
) or worse
A STORY > LESSONS

20. Where did we go wrong?
A STORY > LESSONS

21. Cohesive services, loosely coupled (want)
vs
Cohesive teams, tightly coupled by services (got)
A STORY > LESSONS

22. Previous tooling and strategies were not good enough
A STORY > LESSONS

23. “Integration tests are a scam”
- JB Rainsberger
A STORY > LESSONS > INTEGRATION TESTS

24. A STORY > LESSONS > INTEGRATION TESTS
Scam, you say? Justify!
Integrated tests are:
● Slow
● Fragile
● Hard to manage
When they fail, you can’t point to the problem!

25. “But my integration tests run in Docker, why can’t I use
them?”
- People
A STORY > LESSONS > INTEGRATION TESTS

26. “Because Maths”
- Me
A STORY > LESSONS > INTEGRATION TESTS

27. A STORY > LESSONS > INTEGRATION TESTS

28. A STORY > LESSONS > INTEGRATION TESTS

29. Branches per box vs test cases required
2 code branches = 128 tests
5 code branches = 78,125 tests
10 code branches = 10M tests
A STORY > LESSONS > INTEGRATION TESTS

30. Good tests have the exact opposite properties
A STORY > LESSONS > INTEGRATION TESTS

31. Mocks to the rescue?
A STORY > LESSONS > INTEGRATION TESTS

32. A STORY > LESSONS > INTEGRATION TESTS

33. A STORY > LESSONS > INTEGRATION TESTS

34. Dictator Driven Contracts
A STORY > LESSONS > API DESIGN

35. Dictator Driven Contracts
A STORY > LESSONS > API DESIGN

36. A STORY > LESSONS > API DESIGN
How to: dictator driven contracts
1. Sit in ivory tower and postulate
2. Document perfect API (Swagger/OAS etc.)
3. Create said API
4. Publish said document to consumers
5. Repeat steps 1-4

37. A STORY > LESSONS > API DESIGN

38. A STORY > LESSONS > API DESIGN

39. A STORY > LESSONS > API DESIGN

40. Dictator Consumer Driven Contracts
A STORY > LESSONS > API DESIGN

41. A STORY > LESSONS > API DESIGN

42. Benefits?
A STORY > LESSONS > API DESIGN

43. You’ll know when you break a consumer
A STORY > LESSONS > API DESIGN

44. You have a form of documentation
A STORY > LESSONS > API DESIGN

45. You can test things independently
A STORY > LESSONS > API DESIGN

46. Pact
www.pact.io

47. Evolved from combining these two principles
PACT

48. ● Open source (MIT)
● Multiple languages
○ Javascript
○ Go
○ Python
○ JVM
○ .NET
○ + more
pact.io
● HTTP contracts
● Message contracts
● … protocol agnostic!
PACT

49. PACT > WHO’S USING IT?

50. PACT > STEP 1: WRITE A CONSUMER TEST

51. PACT > STEP 1: WRITE A CONSUMER TEST

52. PACT > STEP 1: WRITE A CONSUMER TEST

53. Given “User A exists”
When I Receive “a GET request for user A”
With “content-type: application/json”
Respond with “200 OK”
And “User A’s details in the body”
PACT > STEP 1: WRITE A CONSUMER TEST

54. Step 2: Share your pacts
PACT > STEP 2: SHARE CONTRACT

55. PACT > STEP 3: VERIFY PROVIDER

56. PACT > NO SILVER BULLET
No Silver Bullet

57. Are not functional tests
Contracts tests
PACT > NO SILVER BULLET

58. Are not load
performance tests SLAs
Contracts tests
PACT > NO SILVER BULLET

59. Are not good for “public”
APIs (many, unknown
consumers)
(Consumer)
Contracts tests
ANOTHER WAY @bethesque

60. PACT > NO SILVER BULLET

61. PACT > NO SILVER BULLET

62. PACT > NO SILVER BULLET
Pact

63. PACT > NO SILVER BULLET
Do I still need integration tests?

64. PACT > NO SILVER BULLET
Probably, maybe, yes?

65. PACT > NO SILVER BULLET
Integration test coverage
Consequences of bug
x
Time to find bug
x
Time to release fix

66. DEMO

67. BEST PRACTICES
Integrating with your CICD processes

68. BEST PRACTICES
68

69. BEST PRACTICES > CHALLENGES
69
1. Sharing and collaborating on pacts between teams
2. Managing contracts across code branches and environments
3. Orchestrating builds to know when it is safe to deploy

70. Introducing...
The Pact Broker

71. BEST PRACTICES > PACT BROKER
71
Integrated collaboration tool to and share and manage pacts* between
teams.
● API documentation that is guaranteed to be up-to date
● Visualisations of the relationships between your services
● Dashboards containing contract verification status
● Pact tagging and versioning
● Webhooks for integration and communication
● View pact diffs
● ...and more!
All powered by a RESTful API

72. BEST PRACTICES > PACT BROKER
72
Did I mention README badges and integrations?

73. DEMO
BEST PRACTICES > PACT BROKER > VISUALISATIONNS
73

74. DEMO
BEST PRACTICES > PACT BROKER > VISUALISATIONNS
74

75. Replace lengthy integration test pipelines with
“The Matrix”
BEST PRACTICES > PACT BROKER > PIPELINES
75

76. BEST PRACTICES > PACT BROKER > CAN I DEPLOY?
76
● Verify that a Pacticipant is safe to deploy prior to release
● Enables “matrix” style testing
Consumer Head (1.0.1) Consumer Prod (1.0.0)
Provider Head (2.0.0) ? ?
Provider Prod (1.1.13) ? Already tested

77. BEST PRACTICES > PACT BROKER > CAN I DEPLOY?
77
Can also do with tags...
Consumer Head (‘dev’) Consumer Prod (‘prod’)
Provider Head (‘dev’) ? ?
Provider Prod (‘prod’) ? Already tested

78. BEST PRACTICES > PACT BROKER > “THE MATRIX”
78
$ pact-broker can-i-deploy
--app A --version 1
--app B --version 4

79. BEST PRACTICES > PACT BROKER > CAN I DEPLOY?
79

80. BEST PRACTICES > PACT BROKER > CAN I DEPLOY?
80
Computer says yes o/

81. DEMO
BEST PRACTICES > PACT BROKER
81

82. BEST PRACTICES > CI/CD
82

83. BEST PRACTICES > CI/CD > PROVIDER
83

84. BEST PRACTICES > CI/CD > CONSUMER
84

85. Best Practices: Summary

86. Summary
Best Practices
● Pact Broker is key for scaling
● Use webhooks for team communication
○ But… keep build triggers simple!
● Tag pacts to mirror your git branches
● Use can-i-deploy tool for deployment
pre-flight checks
● Update pact tags after deployment to
environment (e.g. prod)

87. Recapping...
● Business impact of integrated tests and environments
● Cause and explanation of those effects
● Alternative approach - isolation + contracts
● Contract-testing as an approach, Pact as a tool (in this order!)
● CDC has its own challenges
○ Not suitable for external/public APIS
○ effective collaboration is the key

88. Thank you
- @matthewfellows
- @pact_up
Given “The presentation is over”
Upon Receiving “A request for an answer”
With “A valid question”
Respond With “A valid answer”

89. What about Swagger / Open API?

90. A B
mock/write
C P
pact swagger
verify
verify
Pact+Swagger
OTHER TOOLS

91. No Silver Bullet

92. What about systems maintained by other teams?

93. What about systems built in outdated technologies?

94. Scary outside
world!
Websphere?
Mainframe

95. Thank you
- @matthewfellows
Given “The presentation is over”
Upon Receiving “A request for an answer”
With “A valid question”
Respond With “A valid answer”