Mark59 – A Jmeter/Selenium/CI Open Source P&V Solution

1. Mark59 : A Jmeter-Selenium-CI Open Source Solution
By Philip Webb

2. Overview
• The Team
• Problems and progression through solutions
- ‘traditional P&V’ to CI P&V to ‘Mark59’
• A technical overview of Mark59
- scripting

3. Design Influences
• My background
• Mainframe applications, odds and ends, Java development, Functional Testing, P&V
• Our IT environment
• Functional (QA) testers executing Regression Suites
• Server-based technologies
• Cloud migration

4. SourceControl
• Some-project_dir/My_favourite_LR-script_20180909_v3

5. Jenkins
• Running individual LoadRunner scripts
• Executing LoadRunner scenarios and report generation

6. Jenkins Reports

7. DataHunter
• Test data management
• Mark59 development begins with the ‘DataHunter’ application
• … Later also becomes the Mark59 ‘sample applicaton’

8. DataHunter Usage: HomeScreen

9. DataHunter Usage: AddPolicyScreen

10. DataHunter Usage: AddPolicyResults Screen

11. Baselines
• Comparisons in Excel
• Baseline to baseline variation
• ‘Mark59’ : Creation of the ‘Trend Analysis’ graphical application

12. Trend Analysis – The Graph
The graphic spins ….

13. Trend Analysis – The Graph : Hover Over and Data Table

14. Trend Analysis – SimpleFiltering

15. Trend Analysis – Advanced Filters

16. SLAs

17. Transactional SLAData Entry
• Note the SLA Bulk Edit functions

18. ApplicationDashboard (SLA summaries)

19. UsingJmeter with Java Selenium ?
• Triggered by a ‘problem project’
• .. but with a full java selenium regression suite available
• Alignment with design philosophy
• Skills growth and knowledge sharing
• Justified by cost savings
• Jmeter
• Java friendly, well-known established open source tool
• ‘Mark59’ : Selenium-Implementation for Jmeter built

20. Mark59 Selenium Implemenation
• Eclipse based development
• Maven builds
• Central

21. Report Changes for JmeterImplementation in Mark59
• Reports Generation

22. Henrik and theNeoload Selenium Recipe

23. Phil’s Mark59 Selenium and JmeterRecipe
Ingredients
• 1 x Fully cooked Selenium DSL
• 1 x Raw Apache Jmeter
• Min 1 fresh Web Browser
(Chrome preferred)
• Min 1 good Selenium Driver
(ChromeDriver preferred)
• 1 x Fully prepared Eclipse
(Springboot Plugin optional)
• 2 x Jars of Mark59
• Min 1 DIY script sauce
Method
• Mix together Ecplise, DSL and Driver
• Pour in the jars of Mark59
• Add your script sauce and use a Maven to
place the results an a clean jar
• Add all into the raw Jmeter
• Switch on the Jmeter and Run
• Note: kitchen is assumed to have a git
• Advanced Chefs
• Should consider combining all ingredients
in a Jenkins

24. Mark59 Data Flow Overview (Jmeter)
Diagram Excludes:
LoadRunner Processing,
Logging, Archiving, API
Target
System

25. Scripting

26. Script-Jmeter Parameter Interface (1 of 2)

27. Script-Jmeter Parameter Interface (2 of 2)

28. Scripting- Script Structure and Inline Execution
• jm.startTransaction("DH-lifecycle-0200-addPolicy");
protected Map<String, String> additionalTestParameters() {
public class DataHunterLifecyclePvtScript extends SeleniumAbstractJavaSamplerClient
{
protected void runSeleniumTest(JavaSamplerContext context, JmeterFunctionsForSeleniumScripts jm, WebDriver driver) {
jm.endTransaction("DH-lifecycle-0200-addPolicy");
public static void main(String[] args) throws InterruptedException{
Log4jConfigurationHelper.init(Level.INFO) ;
DataHunterLifecyclePvtScript thisTest = new DataHunterLifecyclePvtScript();
//1: single
thisTest.runSeleniumTest(KeepBrowserOpen.ONFAILURE);
//2: multi-thread
thisTest.runMultiThreadedSeleniumTest(2, 2000);
//3: multi-thread with parms
Map<String, java.util.List<String>>threadParameters = new java.util.LinkedHashMap<String,java.util.List<String>>();
threadParameters.put("USER", java.util.Arrays.asList( "USER-MATTHEW", "USER-MARK", "USER-LUKE", "USER-JOHN"));
threadParameters.put(SeleniumDriverFactory.HEADLESS_MODE, java.util.Arrays.asList( "true" , "false" , "true" , "true"));
thisTest.runMultiThreadedSeleniumTest(4, 2000, threadParameters);
}
}
}
You normally only chose
one of the 3 options !
Mark59 instantiates the
Selenium WebDriver

29. Selenium DSL basics ( 1 of3 )

30. Selenium DSL basics (2 of3)

31. Selenium DSL basics (3 of3)
Method called by script

32. Selenium – Beyondthe Basics

33. Scripting– Logging andLog Control

34. Scripting: JavaDocs are awork inprogress…..

35. Scripting: Documentation(@ Beta4 release)
• The Sample project : https://github.com/iagcl/mark59/tree/master/dataHunterPVTest
• The “QuickStart”notes : https://github.com/iagcl/mark59
• https://mark59.com status : “Coming soon”
That’s a tumbleweed

36. Jenkins CI

37. Jenkins JobFlows
Build and Deploy Jmeter Test
Executing a Single Injector Jmeter Test
Executing a Jmeter Distributed Test

38. AJenkins Check Job: Parameters

39. AJenkins Results Job (1 of2)
(fudged) links to logging directories on injectors

40. AJenkins Results Job (2of2 )

41. Injector LoadCapacity Data (Virtual /non-AWSServers)
• All injectors :
• Inter(R) Xeon(R) E7-2870 @ 2.40GHz 16 CPU (Virtual) 24 GB Ram 200GB Disk Win ‘16
• Largest current single server Selenium-only Test
• 13,000 txns/hr , 45 threads : CPU Util Average 30%
• ‘Just to Big’ for single server Selenium-only Test (10% degradation in responses)
• 27,000 txns/hr , 60 threads ,
• When executed on a single injector : CPU Util Average 50 – 60 %
• Distributedacross 2 injectors : CPU Util Average 20 – 30%
• Largest distributed (2 servers) Selenium-only Run
• 44,000 txns/hr , 100 threads : CPU Util Average 30-45%

42. LoadCapacity: historical CPU variance distributedinjectors
• Largest distributed (2 server) Selenium-only Run (44000 txns/hr , 100 threads)
Injector CPU Utilization Averages (last 10 soak runs)
• Guesstimate max practical capacity using similiar sized VMs : ~ 10 servers => ~ ~ 200,00 txns/hr
Injector 1
Injector 2

43. Thank You
• Follow-up
• If ‘Mark59’ gets traction we’ll monitor on Stack Overflow
• Please watch for updates and progress at https://github.com/iagcl/mark59
• and soon at https://mark59.com