Dolphin is a regression test infrastructure built from ground up for Latitude system testing. It is a major initiative to improve test efficiency and meet test automation objective for Latitude system evaluation department. Latitude system testing involves in large varieties of devices and device interactions, including: PG, communicator, external sensors, heart simulator, TTM, PRM, etc, as well as, different type of application server environments with different geographic configuration, which impose great engineering challenges. Dolphin answers those challenges by standardizing test station hardware and software configuration for hardware virtualization. Each virtualized test station is then brought into Dolphin service cloud (private cloud) and becomes virtual service end point. Dolphin manages test stations farm and configures them automatically per test request. Dolphin exposes its services via RESTful JSON web service, including: Regression Testing support, Device configuration and control, Lease out a virtual test station on demand as a hardware resource. Besides, Dolphin provides an integrated web application for all of its services, including services provided by Panda. It is a one stop shop for all Latitude system testing needs. Methods: 1. Hardware virtualization to bring virtualized test station into cloud (cloud computing). 2. On demand web application for all exploratory testing needs. 3. Service is available through thin wrapper API (Python, Java, Javascript) for automated test development. 4. Integrated with Panda service for enrollment and data browsing. Results: 1. Simplify hardware configuration: all test stations are standardized and no need to physically switch parts and reconnect wire. 2. On demand web application greatly simplify device setup and control. 3. Integrated with Panda for automatic enrollment. Manual test can achieve the efficiency of automated test. 4. Centralized management reduces the maintenance cost. 5. Flexibility: automated test can be developed in any language and run on any platform because virtual test station can be leased out from test farm as hardware resource.

1. Dolphin: Regression Test System for Latitude
Tao Jiang
www.linkedin.com/in/taojiang2000/
Yongjin Yan
Rohitasva Rohitasva

2. Objectives
• Infrastructure as a Service
• Latitude Regression Test System
2

3. Latitude System
3

4. Challenges in Latitude System Testing
4

5. First Step
Standardize Test Station (Dolphinize)
• Create private network for each test station
• Extensively Use PG system board
• Eliminate moving parts to reduce error
• Set it, and forget it!
5

6. Dolphin Test Station Configuration
6

7. Infrastructure as a Service
Dolphin delivers Testing Infrastructure as a Service (IaaS)
• Virtualized Test Station (Hardware Virtualization)
• A Controller (virtual test station manager)
– Manage test farm machines
– Testing Service Provider
• Device configuration and control
• Regression Testing
• On demand testing hardware resource
7

8. Dolphin: System Overview
8

9. What Dolphin does?
Dolphin is a collection of:
• Cloud Infrastructure: Testing Infrastructure
– Test Station on Demand
– Regression Testing
• Cloud Applications: Software as a service over Internet
– Dolphin Web Application: Software on Demand
• Automated Device Setup & Test Control
– Integrate with Panda Web Application
• Automated Account Enrollment
• Dolphin API: Client interface to Dolphin Cloud Service
– Get Virtual Test Station Instance via API
– Test can be run on any machine not just on test station
9

10. Device Setup using Dolphin
10

11. Test Control for Exploratory Testing
11

12. Demo: System BAT
• Run Real Device
• End to End
• Acceptance Testing Effort
Latitude 6.0: ~ 1 week
Latitude 7.0: ~ 1 day
Dragon 1.0: ~ 1 hour on a Dolphinized test station
12

13. Dolphin: Regression Test System
• Run test on different test station without manual intervention.
• Auto configure hardware per test requirement.
• Auto configure and update supporting software packages.
• Work order scheduling.
• Test result reporting.
13

14. Work Order Example
workorder = {
"Submitter": "fortyTwo/g041145",
"Test": {
"Arguments": ["-t", "Test1_Supported_PGs", "-c", "Case1_Supported_PGs", "-x", "JP_1.0_JAGJP", "-i", "W172-400-0", "-p", "pgsim=false,enroll=false,wac=10057,+auto"],
"Name": "Case1_Supported_PGs/W172-400-0/6443/JP",
"Protocol": "SDVT_DeviceData_Supported_Devices",
"Stream": "DVT_System_Dragon",
"TestScript": "SDVT_DeviceData_Supported_Devices.py"
},
"Environment" : { # environment to run against
"Type" : "REG", # Clock, Biz , When type is Clock but different region, regression system will do a deployment.
"Region" : "JP“ # region field is a required.
},
"Hardware": { # Hardware information in order to run the test.
"Setup" : “true", # True if the test script want s regression system setup hardware for it. False, test script will setup hardware by itself.
"Communicator": {
"ModelNumber": "6443",
"SerialNumber": "5077766",
"ServerConnectionType": "ethernet"
},
"PulseGenerator": {
"ModelNumber": "W172-400-0",
"SerialNumber": "5077766",
"PGFamily": "Ingenio",
"Type": "PGBoard"
}
},
# URL for test result notification
"NotifyUrl" : "http://yourTestManagementSite.com/regression/results/"
}
14

15. Implementation Strategy
• Dolphin Controller and Agent Service
• .Net application written in C#.
• Windows Communication Foundation (WCF): for building service-oriented applications.
• C# Selenium 2.0 WebDriver for driving Dragon App.
• Expose functionalities as a set of web services.
• Automated device setup and account enrollment
• Simplify test script development
15

16. Apache Server as a Cluster Service
• Cluster N + 1 application servers
– N farm machines and 1 controller
– Base URL: http://dolphin/dolphinws
– Forward command to target automatically
• Deliver static page content
16

17. Dolphin Web Service API
• Schedule regression test by submitting work order
• Test station rental service
• Configure and Control devices
17

18. Technology Highlights
• C# .NET 4.0
• WCF: Windows Communication Foundation
• Selenium 2.0 and WebDriver (C# version)
• jQuery and jQuery UI
• Apache HTTP Server
18

19. Results
• Standard hardware configuration
• On demand web application
• Integrated with Panda for automatic enrollment
• Centralized management reduces the maintenance cost
• Flexibility
– Hardware on demand
– Language-Neutral Solution to automated test development
– Platform-Neutral Solution to test execution
19

20. Conclusions
• Dolphin continues Panda's successful adventure of cloud
computing (Software as a Service)
• Dolphin brings hardware resource into cloud
(Infrastructure as a Service)
• Greatly simplifies Latitude system-level end-to-end
testing
• Reduces cost
20

21. 21
Thank You