1. qaware.de
Endlich gute API Tests
Boldly Testing APIs Where No One Has
Tested Before
Ildikó Tárkányi
ildiko.tarkanyi@qaware.de

2. Ildikó Tárkányi
Senior Software Engineer @ QAware GmbH 2
QAware

3. Sonja Wegner
Lead Software Architect @ QAware GmbH 3
QAware

4. API Tests

5. 5
QAware

6. 6
QAware

7. 7
QAware

8. 8
QAware

9. 9
QAware

10. 10
QAware

11. 11
QAware

12. 12
QAware

13. Spezifikation/Schema Testing

14. 14
QAware

15. openValidation

16. 16
QAware

17. 17
QAware
openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
responses:
'200':
description: Speaker added successfully

18. 18
QAware
openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
x-ov-rules:
culture: en
rule: |
IF Name IS NOT Ildikó Tárkányi
AND Email DOES NOT CONTAIN @qaware.de
THEN Please register with your full name and business e-mail address
responses:
'200':
description: Speaker added successfully

19. openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
x-ov-rules:
culture: en
rule: |
IF Name IS NOT Ildikó Tárkányi
AND Email DOES NOT CONTAIN @qaware.de
THEN Please register with your full name and business e-mail address
responses:
'200':
description: Speaker added successfully
19
QAware
openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
responses:
'200':
description: Speaker added successfully

20. openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
IF Name IS NOT Ildikó Tárkányi
AND Email DOES NOT CONTAIN @qaware.de
THEN Please register with your full name and business e-mail address
responses:
'200':
description: Speaker added successfully
20
QAware
Schlüsselwörter
openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
x-ov-rules:
culture: en
rule: |
responses:
'200':
description: Speaker added successfully

21. openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
IF Name IS NOT Ildikó Tárkányi
AND Email DOES NOT CONTAIN @qaware.de
THEN Please register with your full name and business e-mail address
responses:
'200':
description: Speaker added successfully
21
QAware
Schlüsselwörter
Schema-Attribute
openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
x-ov-rules:
culture: en
rule: |
responses:
'200':
description: Speaker added successfully

22. openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
IF Name IS NOT Ildikó Tárkányi
AND Email DOES NOT CONTAIN @qaware.de
THEN Please register with your full name and business e-mail address
responses:
'200':
description: Speaker added successfully
22
QAware
Schlüsselwörter
Schema-Attribute
Operanden
openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
x-ov-rules:
culture: en
rule: |
responses:
'200':
description: Speaker added successfully

23. openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
IF Name IS NOT Ildikó Tárkányi
AND Email DOES NOT CONTAIN @qaware.de
THEN Please register with your full name and business e-mail address
responses:
'200':
description: Speaker added successfully
23
QAware
Schlüsselwörter
Schema-Attribute
Operanden
domänenspezifische Operatoren
openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
x-ov-rules:
culture: en
rule: |
responses:
'200':
description: Speaker added successfully

24. openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
IF Name IS NOT Ildikó Tárkányi
AND Email DOES NOT CONTAIN @qaware.de
THEN Please register with your full name and business e-mail address
responses:
'200':
description: Speaker added successfully
24
QAware
Fehlermeldung
Schlüsselwörter
Schema-Attribute
Operanden
domänenspezifische Operatoren
openapi: 3.0.3
info:
version: 1.0.0
title: Conference Management Service
paths:
/speakers:
post:
requestBody:
content:
application/json:
schema:
properties:
Name:
type: string
Email:
type: number
x-ov-rules:
culture: en
rule: |
responses:
'200':
description: Speaker added successfully

25. Stoplight - Spectral

26. 26
QAware

27. 27
QAware
extends: ["spectral:oas", "spectral:asyncapi"]
formats: ["oas3"]
documentationUrl: https://www.example.com/docs/api-style-guide.md
parserOptions:
duplicateKeys: warn # error is the default value
aliases:
Paths:
- "$.paths[*]~"
rules:
paths-kebab-case:
description: Paths should be kebab-case.
message: "{{property}} should be kebab-case (lower-case and separated with hyphens)"
severity: warn
formats: ["oas3"]
given: "#Paths"
then:
function: pattern
functionOptions:
match: "^(/|[a-z0-9-.]+|{[a-zA-Z0-9_]+})+$"

28. 28
QAware
extends: ["spectral:oas", "spectral:asyncapi"]
formats: ["oas3"]
documentationUrl: https://www.example.com/docs/api-style-guide.md
parserOptions:
duplicateKeys: warn # error is the default value
aliases:
Paths:
- "$.paths[*]~"
rules:
paths-kebab-case:
description: Paths should be kebab-case.
message: "{{property}} should be kebab-case (lower-case and separated with hyphens)"
severity: warn
formats: ["oas3"]
given: "#Paths"
then:
function: pattern
functionOptions:
match: "^(/|[a-z0-9-.]+|{[a-zA-Z0-9_]+})+$"

29. 29
QAware
extends: ["spectral:oas", "spectral:asyncapi"]
formats: ["oas3"]
documentationUrl: https://www.example.com/docs/api-style-guide.md
parserOptions:
duplicateKeys: warn # error is the default value
aliases:
Paths:
- "$.paths[*]~"
rules:
paths-kebab-case:
description: Paths should be kebab-case.
message: "{{property}} should be kebab-case (lower-case and separated with hyphens)"
severity: warn
formats: ["oas3"]
given: "#Paths"
then:
function: pattern
functionOptions:
match: "^(/|[a-z0-9-.]+|{[a-zA-Z0-9_]+})+$"

30. 30
QAware
B:.
├───stoplight
│
└───spectral
.spectral.yaml
myOpenAPI.yaml
B:stoplightspectral> spectral lint myOpenAPI.yaml
8:10 warning operation-description Operation "description" must be present and non-empty string.
paths./speakers.post
8:10 warning operation-operationId Operation must have "operationId".
paths./speakers.post
8:10 warning operation-tags Operation must have non-empty "tags" array.
paths./speakers.post
✖ 3 problems (0 errors, 3 warnings, 0 infos, 0 hints)

31. 31
QAware

32. Contract Testing

33. 33
QAware

34. 34
QAware

35. pact

36. 36
QAware
{
"metadata": {...},
"provider": {
"name": "speaker-provider"
},
"consumer": {
"name": "speaker-consumer"
},
"interactions": [
{
"description": "A GET request for a speaker",
"request": {
"method": "GET",
"path": "/speakers/some-name"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json"
},
"body": {
"name": "some-name",
"email": "some-email"
}
}
},
{...}
]
}

37. 37
QAware
1
2
3
4

38. 38
QAware
@PactTestFor(providerName = "speaker-provider", hostInterface = "localhost")
public class SpeakerProviderPactTest {
@Pact(consumer = "agenda-consumer", provider = "speaker-provider")
public V4Pact createPactForSpeakerInteractions(PactDslWithProvider builder) {
PactDslJsonBody newSpeakerRequestBody = new PactDslJsonBody();
newSpeakerRequestBody.stringValue("name", "new-name")
.stringValue("email", "new-email")
.closeObject();
return builder
.uponReceiving("A POST request to add a speaker")
.path("/speakers")
.method(HttpMethod.POST)
.headers("Content-Type", "application/json")
.body(newSpeakerRequestBody)
.willRespondWith()
.status(204)
.toPact(V4Pact.class);
}
}

39. 39
QAware
@Test
@PactTestFor
public void testSpeakerInteractions() {
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.setContentType(MediaType.APPLICATION_JSON);
String jsonBody = "{"name": "new-name", "email": "new-email"}";
ResponseEntity<String> postResponse = new RestTemplate().exchange(
mockProvider.getUrl() + "/speakers",
HttpMethod.POST,
new HttpEntity<>(jsonBody, httpHeaders),
String.class
);
assertThat(postResponse.getStatusCode().value()).isEqualTo(204);
}

40. 40
QAware

41. 41
QAware
@Provider("speaker-provider")
@PactFolder("pacts")
class SpeakerProviderPactTest {
@BeforeEach
void setup(PactVerificationContext context) {
context.setTarget(new HttpTestTarget("localhost", 8080));
}
@TestTemplate
@ExtendWith(PactVerificationInvocationContextProvider.class)
void pactVerificationTestTemplate(PactVerificationContext context) {
context.verifyInteraction();
}
}

42. 42
QAware

43. 43
QAware

44. 44
QAware
@Provider("speaker-provider")
@PactBroker(url = "http://localhost:9292")
class SpeakerProviderPactTest {
@BeforeEach
void setup(PactVerificationContext context) {
context.setTarget(new HttpTestTarget("localhost", 8080));
}
@TestTemplate
@ExtendWith(PactVerificationInvocationContextProvider.class)
void pactVerificationTestTemplate(PactVerificationContext context) {
context.verifyInteraction();
}
}

45. 45
QAware

46. Trace-Testing

47. 47
QAware

48. Tracetest

49. 49
QAware

50. 50
QAware

51. 51
QAware

52. 52
QAware

53. 53
QAware

54. 54
QAware
type: Test
spec:
id: Ugtm74WIg
name: speakers - POST
description: Create new speaker
trigger:
type: http
httpRequest:
method: POST
url: http://provider-service:8080/speakers
body: "{n "name": "some-name",n "email": "some-email"n}"
headers:
- key: Content-Type
value: application/json
specs:
- selector: span[tracetest.span.type="http"]
name: "Http span: response status code is 204"
assertions:
- attr:http.status_code = 204
- selector: span[tracetest.span.type="database"]
name: "Database span: processing time is less than 100ms"
assertions:
- attr:tracetest.span.duration < 100ms
- selector: span[tracetest.span.type="general" name="Tracetest trigger"]
name: "Trigger span: response time is less than 200ms"
assertions:
- attr:tracetest.span.duration < 200ms

55. 55
QAware
B:.
├───tracetest
speakers_post.yaml
PS B:api-testing-demotracetest> tracetest run test --file speakers_post.yaml --output pretty
✔ speakers - POST (http://localhost:11633/test/Ugtm74WIg/run/42/test) - trace id: af60f652b4097a13120f001df3da4edd
✔ Http span: response status code is 204
✔ Database span: processing time is less than 100ms
✔ Trigger span: response time is less than 200ms

56. 56
QAware

57. qaware.de
QAware GmbH
Aschauer Straße 32
81549 München
Tel. +49 89 232315-0
info@qaware.de
twitter.com/qaware
linkedin.com/company/qaware-gmbh
xing.com/companies/qawaregmbh
slideshare.net/qaware
github.com/qaware
Conclusion and Q&A