This is my slides' content when I visit cusotmer for first time! You also can understand what my comapny's do via this presentation!

1. Standards of Excellence
TestShell Framework
Introduction
Tuesday, December 20, 2011
Richard Hsu
QualiSystems Proprietary & Confidential

2. QualiSystems – Company Overview
• Global Software Company
•Santa Clara, USA
•London, UK
•Munich, Germany
•Tel-Aviv, Israel
•APAC
• Singapore , APAC HQ
• Shanghai, China
• Taipei, Taiwan
• Seoul, Korea
• APAC Channels, Australia, Japan , India
• Founded 2004
9001:2000 90003:2004
• Flag product – TestShellTM Test-Automation Framework
• “Customer First” Culture
QualiSystems Proprietary & Confidential

3. What we do - TestShell Framework
• End-to-End framework for Test Automation
that supports the entire life cycle of a test for components,
devices, or systems
• TestShell automates and simplifies the process of testing,
allows you to focus on pure testing, and thus substantially:
 Improve Product Quality
 Shorten Time to Market
 Reduce Expenditures
QualiSystems Proprietary & Confidential

4. Testing Challenges without Automation
• Lengthy test processes that can take up to
several days
• The wide array of benchmark
applications/Scripts include Hardware and
Software Time Lost
• The variety of vendors and models requiring
Quality Impacted
diverse specifications
• The need to control and manage many Expense Overhead
distributed test stations
Resources
• Time-consuming test result collation and Unexploited
analysis
• Compound aggregation of extensive amounts
of test data
QualiSystems Proprietary & Confidential

5. Proven ROI within the first year of use
• Significantly reduce Time-To-Market
• Full regression cycle 64 hours → 24 hours at USI (環隆電氣)
• Single test cycle 15 days → 1.5 day at Cameo (友勁科技)
• Significantly reduce Operational Expenditures
• Resources involved in test creation down by 50% at Deutsche Telekom
• Single test cycle effort 2 Days → 5 hours at Telstra
• Managing and Improving Quality Control
• Daily Build and regression at Ceragon Networks
• Customer service centers at Crow
• Saving X10 on RMA at Syneron
QualiSystems Proprietary & Confidential

6. Our Customers – across the value chain
QualiSystems Proprietary & Confidential

7. End-to-End Test Automation Framework
Schedule Execute
Run Tests & Regressions
Coordinate Test Execution
Collect Logs & Results
Launch Test Stations
Monitor Real-time
Tests & Regressions
Assets
Create Equipment
Analyze
Test Results
Write Tests & Regressions Aggregate Results
Create reusable functions Central Customized Reports
Control Test Assets
Management Customized Dashboards
QualiSystems Proprietary & Confidential

8. TestShell Platform Details
QualiSystems Proprietary & Confidential

9. End-to-End Test Automation Framework
Schedule Execute
Create Analyze
Central
Management
QualiSystems Proprietary & Confidential

10. TestShell System
Interfaces to Software and hardware compnents (Any many more..)
Scripts & • Exe, DLLs, CLI, VIs • Mail & SMS, File Engine
Software Utilities
• Script-Engines • GUI Automation, Attributes
• TCL, Perl, Python, VB, Java • Requirements
• MatLab, OCR Management, QTP ,LR , Etc..
Networking
• Telnet, SSH, TCP, UDP Mobile • Symbian
Modules
• SNMP (& Traps), FTP • Windows Mobile
• Traffic Generators, Switches • iPhone, Blackberry
• etc. • Android
T&M • Traffic generators • Messages, Instructions
Modules Dialogs
• Network analyzers • Alerts, Remarks
• Spectrum, scope • Watchers, Graphs
• Generators, etc. • etc.
Logic • Variables Web
Modules • Web GUI
• Cases, Loops, Delays Modules
• Web Services
• Parallelization, events
• etc.
• etc.

11. Testing in the All IP World - Example Scenario
Start • Typical test flow for validating :
• Allocate all testing resources
• Configure the Device Under Test
Setup Topology • Configure Traffic Generator(s)
• Applying traffic to the DUT and
Configure DUT sniffing /getting statistics
• Analyzing
Configure Traffic
Generator (ex. IXIA
, SPIRENT) • Typical tests:
• Traffic blasting, protocol emulation, load & stress,
conformance , acceptance, interoperability, etc
Run Traffic
Analyze Results • Typical validations:
• End-to-End: Path Recovery, Clock Recovery,
Delay, Latency, Throughput, Loss Ratio,
End Information-leakage, routing/switching performance, etc
QualiSystems Proprietary & Confidential

12. Testing in the All IP World - Example Scenario
Start
Multiple Network Elements from
different vendors such as:
Setup Topology
• Ethernet Switch
Configure DUT
• OSS
Configure Traffic • Wireless devices (Access Points)
• WiMAX/4G (STB)
Run Traffic
• Multi Service Router
Analyze Results • High Speed Router
• BBRAR
Store & Report Data
• DSLAM
• Gateway
End • Modem
QualiSystems Proprietary & Confidential

13. Testing in the All IP World - Example Scenario
Start
Various network protocols and
interfaces
Setup Topology
SSH Telnet SMTP
Configure DUT
SOAP TCP SNMP Traps
Configure Traffic
Run Traffic TFTP SNMP VISA
Analyze Results Serial SFTP GPIB
Store & Report Data
UDP HTTP FTP
End
QualiSystems Proprietary & Confidential

14. Testing in the All IP World - Example Scenario
Start Multiple test equipment types
from different vendors
Setup Topology
• Traffic Generators
Configure DUT • Network Emulators
Configure Traffic • Network Sniffers
• Network Analyzers
Run Traffic • Baseband Emulators
Analyze Results • Logic analyzers
• Signal analyzers
Store & Report Data
• Oscilloscopes
• Signal generators
End
QualiSystems Proprietary & Confidential

15. Testing in the All IP World - Example Scenario
DEMO
QualiSystems Proprietary & Confidential

16. TestShell Common
Scenario for Network
QualiSystems Proprietary & Confidential

17. DEMO 1: Router Testing via Traffic Generator
- Configure DUT by Telnet
Traffic Data Data Traffic
Sender Receiver
Configure Check alive
Router/Switch Command line
Run Traffic Telnet Analyze Results
Driver Driver

18. DEMO 1: Router Testing via Traffic Generator (cont.)
Code Free Test Creation as easy as Visio
Visio Flow Studio Flow
QualiSystems Proprietary & Confidential

19. DEMO 2: AP Router Testing via Traffic Generator
- Configure DUT by Website(GUI)
Traffic Data Data Traffic
Sender Receiver
Configure Router Check alive
Website(GUI) Command line
Run Traffic Analyze Results
Driver Driver

20. DEMO 2: AP Router Testing via Traffic Generator (cont.)
- Configure DUT by Website(GUI)
• Special features for Wireless Device:
1. Use different configuration files to complete diverse
testing, ex. different mode(802.11n,b,g), different
channel(1~11) or different encryption type(WEP, TKIP).
2. Use excel file to maintain the path of configuration files
and import data of excel file by “File Engine” on
Testshell.
3. TestShell’s GUI Automator can provide “Inspect” and
“Valuate” feature to check states of web screen.
4. TestShell can get snapshoot of testing result and export
to report automatically.
QualiSystems Proprietary & Confidential

21. Tests Creation - Benefits
• Drag & Drop Test Creator – non programmers can join the automation
• Reusable Building Blocks
• Record and Replay Actions
• Integration with both Traffic Generators and T&M
• Supported interfaces:
• Network Protocols (Telnet, SSH, Serial, TCP, UDP, SNMP, etc)
• GUI – Web, Java, Win app
• Logic and Dialog
• Integration with external files (configuration and results)
• Resource Asset Management & Deployment
• Documentation
• Scripts
• Executables
• Dlls

22. End-to-End Test Automation Framework
Schedule Execute
Create Analyze
Central
Management
QualiSystems Proprietary & Confidential

23. Scheduler Snapshot
Resource
management &
Equipment
utilization
Tests scheduling –
overnight & weekend
QualiSystems Proprietary & Confidential

24. Tests Scheduling –
Scheduler
• Automatic scheduling according to time, station and priority
• Test scheduling across single or multiple stations
• Advanced "Outlook-like" interface
• Virtual station locking
• Fully customizable execution rules
• Automatic real-time execution conflict handling

25. End-to-End Test Automation Framework
Schedule Execute
Create Analyze
Central
Management
QualiSystems Proprietary & Confidential

26. Tests Execution –
Runner
• Execution planning
• Select Test or Regression
• Select Unit Under Test and Topology
• Record Admin information
• Serial, Version, Batch etc… (Dynamic)
• Run
• Complete Run
• Step by Step
• Automatic troubleshooting and action execution
• Real-time data tracking
• Standardize data collection
• Data transfer guarantee

27. End-to-End Test Automation Framework
Schedule Execute
Create Analyze
Central
Management
QualiSystems Proprietary & Confidential

28. Test Analysis - InSight
Central data collection
Data is then ready
Test Lab 1 Test Lab 2 21,6 C Test Lab n for comprehensive
128 ms 19 W 101,3 3E-07
102,1
81,9
MHz Melbourne Canbara analysis, using
51 kb/s 3,2 V
MHz 11,9 mA 57 kb/s 69% 12 5,3 TestShell InSight
21 C Sydney 3,3 V mA
www
Data from all
test labs and
stations is
Multi-Site
collected and Repository
stored in a
central server

29. Test Analysis - InSight
Online customizable display
www
The dashboards are
completely customizable
according to user
The analayzed data is
preferences, supporting
presented via online & display
multiple analysis
Dasboards: including OLAP
options,
• Providing real time analysis
• Reviewed any time anywhere

30. End-to-End Test Automation Framework
Schedule Execute
Create Analyze
Central
Management
QualiSystems Proprietary & Confidential

31. Test Management modules - Snapshot
Tests and Regression
management
DUT
management
Central repository across the organization
Tests version
control Fully scalable and openand user control
Permission framework
QualiSystems Proprietary & Confidential

32. TestShell Components
QualiSystems Proprietary & Confidential

33. TestShell Applications
Analyze Create
TestShell InSight TestShell Studio
TestShell Server
Manage Execute
TestShell Scheduler TestShell Runner
TestShell Controller
QualiSystems Proprietary & Confidential

34. Thank you
To Schedule a Live Demonstration go to:
www.qualisystems.com/demo
QualiSystems Proprietary & Confidential