Continuous Lifecycle Online 2021, May 11th 2021, online: Vortrag von Mario-Leander Reimer (@LeanderReimer, Principal Software Architect bei QAware) == Please download slides if blurred! == Abstract: Continuous delivery is everywhere. Well, not quite! Many teams still fail to continuously deliver well tested and stable product increments to production, especially when it comes to its non-functional attributes. Usually with the same old excuse: these high-level tests are too laborious and expensive to implement. But the opposite could be the case! This slides will show how easy it is to implement continuous performance, security and acceptance tests based for microservices on Kubernetes using well-known open source tools.

1. TITELTEXT
Continuous (Non)-Functional
Testing of Microservices on K8s
Continuous Lifecycle Online 2021, May 11th 2021
@LeanderReimer #cloudnativenerd #qaware @ConLifecycleLon

2. Mario-Leander Reimer
Principal Software Architect
@LeanderReimer
#cloudnativenerd #qaware

3. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
3
"Software Is Eating the World.“
Marc Andreessen, 20th August 2011

6. Dev
Ops

7. https://commons.wikimedia.org/w/index.php?curid=2223771

9. The Software Assembly Line

10. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
10
Continuous delivery (CD) is a software engineering
approach in which teams produce software in short cycles,
ensuring that the software can be reliably released at any
time .… It aims at building, testing, and releasing software
with greater speed and frequency.

11. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
11
Continuous
Delivery
Low Risk
Releases
Less
Rework
Fast Time
to Market
Better
Products
Lower
Costs
Happier
Teams
Happier
Users
Loosely Coupled Architectures
Maintainable Code
Empowered Teams
Continuous Security from Day 1
Test
Automation
Continuous
Integration
GitOps
Deployment
Automation
Monitoring and Alerting
Input Output

12. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
12
Continuous
Delivery
Low Risk
Releases
Less
Rework
Fast Time
to Market
Better
Products
Lower
Costs
Happier
Teams
Happier
Users
Loosely Coupled Architectures
Maintainable Code
Empowered Teams
Continuous Security from Day 1
Test
Automation
Continuous
Integration
GitOps
Deployment
Automation
Monitoring and Alerting

13. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
13
Continuous
Delivery
Low Risk
Releases
Less
Rework
Fast Time
to Market
Better
Products
Lower
Costs
Happier
Teams
Happier
Users
Loosely Coupled Architectures
Maintainable Code
Empowered Teams
Continuous Security from Day 1
Test
Automation
Continuous
Integration
GitOps
Deployment
Automation
Monitoring and Alerting

14. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
A single, linear CI/CD pipeline is suboptimal and will
result in delayed feedback and long release cycles.
14
Check-In Build Test Integrate
Deploy to
Production
Automated
Acceptance Tests
Manual
Auto Auto Auto Auto?
Acceptance Security
Performance
These high level tests are usually delayed until the end of sprint or the release.
Which one
fi
rst? Is functionality more important than security or performance?

15. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
A microservice architecture with many down
stream dependencies is complex and hard to test.
15
Cluster
Microservice
A
Microservice
B
External
System B
TEAM A TEAM B UNKNOWN
Microservice
C
TEAM C
External
System A

16. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
Solution: Why not continuously run (non)-functional
tests against a complex microservice architecture?
16
Check-In Build Integrate
Auto Auto Auto
Smoke
Tests
Quality
Acceptance
Security
Performance
Auto Auto?
Check &
Report
Continuous
Runs inside
the cluster
Test
Deploy to
Production

17. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
Architectural Fitness Functions continuously validate
the (non)-functional requirements of the system.
17
https://www.thoughtworks.com/de/radar/techniques/architectural-
fi
tness-function

18. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
Examples of Architectural Fitness Functions
18
https://www.thoughtworks.com/de/insights/articles/
fi
tness-function-driven-development
describe "Performance" do
it "completes a transaction under 2 seconds" do
expect(k6.check_transaction_round_trip_time()).to < 2
end
it "has less than 10% error rate for 10000 transactions" do
expect(k6.check_error_rate_for_transactions(10000)).to < .1
end
end
describe "Security" do
describe “Static Analysis” do
it "should not have plaintext secrets in codebase" do
expect(code.has_secrets_in_codebase()).to_not be(true)
end
end
describe “Dynamic Analysis” do
it "should not have any of the OWASP Top 10" do
expect(zap.has_owasp_top_10_vulnerabilities()).to be(false)
end
end
end

19. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
19
Code & Demos
https://github.com/qaware/continuous-k6k8s
https://github.com/qaware/continuous-zapk8s
https://github.com/qaware/continuous-atdd

20. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
Load Testing for Engineering Teams
• Made for Developers. The k6 API and CLI are easy to use and
fl
exible.
• Out-of-the-box Docker support. Can be run in a container easily.
• Suitable for Smoke Tests, Load Tests, Stress Tests and Soak Tests.
• Supports REST, GraphQL, gRPC and Websockets.
• Result visualisation via DataDog, CloudWatch, In
fl
uxDB+Grafana, New
Relic and many others
• Extensive documentation and good community.
20

21. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
Continuous K6 on K8s - Conceptual View
21
Load Test
Cluster
Microservice
Namespace
Pod
CronJob
Pod
Deployment

22. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
OWASP Zed Attack Proxy (ZED)
• Widely used, popular, free and open source web app scanner.
• ZAP provides powerful APIs for security automation.
• Intercepting proxy, active and passive scanner, traditional and AJAX
spider, brute force scanner, port scanner, OpenAPI v3, web sockets.
• ZAP's docker images provide an easy way to run ZAP, especially in a
CI/CD environment: baseline, full scan, API scan, Webswing UI.
• Extensive documentation and international community.
22

23. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
Continuous Zap on K8s - Conceptual View
23
API Test
Cluster
ZAP API
ZAP
GUI
HTML
REST
CronJob
Pod Pod
Microservice
Deployment
Namespace

24. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
Very Groovy Browser Automation
• Cross Browser Automation: Chrome, Firefox, Safari, Selenium Hub, …
• Powerful, easy to learn Groovy based DSL. Easy content navigation and
access inspired from jQuery.
• Built-in support for Page Object Pattern for maintainable, stable UI tests.
• Supports asynchronicity and dynamic content.
• Provides integration modules for popular test frameworks: Spock, Unit,
TestNG and Cucumber
• Good and easy build system integration for Maven and Gradle
24

25. // Continuous Lifecycle Online 2021 —> Continuous (Non)-Functional Testing of Microservice on K8s // @LeanderReimer #cloudnativenerd #qaware
Continuous ATDD on K8s - Conceptual View
25
UI Test
Cluster
Browser
REST
CronJob
Pod Pod
Microservice
Deployment
Namespace
Selenium
Hub

26. Mario-Leander Reimer
Principal Software Architect, QAware GmbH
mario-leander.reimer@qaware.de
https://www.qaware.de
https://speakerdeck.com/lreimer/
https://github.com/lreimer/
&