Dr. Yongyan Wang presented on a test automation framework (TAF). Some key points:
- TAF is an open source framework for automating repeatable tasks across different languages/tools. It uses a distributed architecture.
- Challenges addressed include making automation independent of test/tool types and ensuring consistency. TAF's design focuses on repeatability, adaptability, and a simple web interface.
- A demonstration showed how TAF can simplify test management and provide pass/fail reports. It also integrates other tools like AutoIt and allows command-line usage.
- TAF aims to make automation easy, reliable, reusable, and independent through its structured approach and extens
2. XBOSoft info
•
•
Founded in 2006
Dedicated to software quality
•
•
•
Software QA consulting
Software testing services
Offices in San Francisco, Beijing, Oslo and
Amsterdam
4. Housekeeping
•Everyone except the speakers is muted
•Questions via the gotowebinar control on the right side of your
screen
•Questions can be asked throughout the webinar, we’ll try to fit
them in when appropriate
•General Q & A at the end of the webinar
•You will receive info on recording after the webinar
5. About the Speaker
Dr Yongyan Wang is senior Software QA
for iovation in Portland, Oregon.
He developed the "Inverse Modeling theory"
in his PhD program to guide numerical
simulation to understand generic linear and
no linear problem.
After receiving his PhD, Dr Wang started
working in the software quality assurance
field focusing on test automation.
Dr Wang has extensive experience applying
automation tools, like QTP, Expect and
autoIt to address industrial automation
needs.
6. Test Automation Framework Demo
by Yongyan Wang PhD
• Test Automation Framework (Google : CPAN TAF)
• Test Automation Challenges/Framework Design Principles
• Test Automation Framework Demo for Principles
– Simplicity/Easiness - point & click, cut & paste
– Repeatability/Reliability/Reusability/Universal
– Independency/Adaptable/Extensibility
– Self-contained IDE /w command-line developer mode
6
7. Test Automation Framework Demo
by Yongyan Wang PhD
• Test Automation Framework (Google : CPAN TAF)
• Test Automation Challenges/Framework Design Principles
• Test Automation Framework Demo for Principles
– Simplicity/Easiness - point & click, cut & paste
– Repeatability/Reliability/Reusability/Universal
– Independency/Adaptable/Extensibility
– Self-contained IDE /w command-line developer mode
7
8. Test Automation Framework Demo
• Test Automation Framework (Google : CPAN TAF)
• Test Automation Challenges/Framework Design Principals
• Test Automation Framework Demo for Principles
– Simplicity/Easiness - point & click, cut & paste
– Repeatability/Reliability/Reusability/Universal
– Independency/Adaptable/Extensibility
– Self-contained IDE /w command-line developer mode
8
9. Test Automation Framework
Google keyword: CPAN TAF
•
•
•
•
•
•
TAF is an open source tool and free to everyone
TAF is a generic framework for any repeatable tasks
TAF collaboration tool (QA, Developer, Mgr, Executives)
TAF has a distributed Peer to Peer, plug-in architecture
TAF is language/script/application independent
TAF is a self-contained IDE for test automation dev.
9
10. Test Automation Framework Demo
• Test Automation Framework (Google : CPAN TAF)
• Test Automation Challenges/Framework Design Principals
• Test Automation Framework Demo for Principles
– Simplicity/Easiness - point & click, cut & paste
– Repeatability/Reliability/Reusability/Universal
– Independency/Adaptable/Extensibility
– Self-contained IDE /w command-line developer mode
10
11. Test Automation Challenges & Design Principals
• TC Auto is unpredictable: Independency/Adaptive for Change
– Test Type independent (unit test, func test, accept T.,UI test, perf test)
– Test Content independent (Jira, Lab Update, Test env, etc)
– Test Tool independent (QTP, AutoIt, Expect, Perl, Selenium, codeUI)
• Automated TC lacks consistency & hard to debug: Repeatable
– TC code struc to ensure repeatability (TC pre, exec, verify, post Proc)
• Too complicate to run: web UI for everyone
– TAF Web UI is simple and easy to use for everyone from anywhere
(Developer/QA/Support/Mgr/Executives)
11
12. Test Automation Framework Demo
• Test Automation Framework (Google : CPAN TAF)
• Test Automation Challenges/Framework Design Principals
• Test Automation Framework Demo for Principles
– Simplicity/Easiness (Installation, TC magnt, and TC Report)
– Repeatability/Reliability/Reusability/Universal
– Independency/Adaptable/Extensibility
– Self-contained IDE /w command-line developer mode
12
13. Simplicity/Easiness (Installation, TC magnt, and TC Report)
• Install Strawberry Perl (youTube: CPAN TAF)
• Install TAF from CPAN (youTube: CPAN TAF)
perl –MCPAN –e “install Test::AutomationFramework”
• Install Test Bed
(youTube: CPAN TAF)
perl –MTest::AutomationFramework –e “help”
13
14. Simplicity/Easiness (Installation, TC magnt, and TC Report)
•
•
•
•
Automation Organization
Automation Remote Display
Automation Remotely Execution
Report Test Result (P/F, performance and TC logs)
14
15. Test Framework Web GUI (by going to http://qtp1)
SAE Test Automation Framework by Yongyan Wang
15
16. Test Framework and different Test Case Types
11/20/13
SAE Test Automation Framework by Yongyan Wang
16
20. Test Automation Framework Demo
• Test Automation Challenges/Framework Design Principals
• Test Automation Framework (Google : CPAN TAF)
• Test Automation Framework Demo for Principles
– Simplicity/Easiness (Installation, TC magnt, and TC Report)
– Repeatability (Test case structure ensure repeatable)
– Adaptable/Independency/Extensibility (autoIt integration)
– UI user mode and command-line developer mode
20
21. Repeatability (Test case structure ensure repeatable)
Test case definition (tc.pl)
tc.pl print
“pass|fail|null|number”
generate log “_tcLogAppend.txt”
setup(), execution(), verification(), teardown()
Test suite definition (index.pl)
index.pl [enter] displays each TC description
index.pl [index] executes test case [index]
webUI execution Pattern
Run Test case once or 24/7 till stopped
Run Test suite once or 24/7 till stopped
21
22. Test Automation Framework Demo
• Test Automation Challenges/Framework Design Principals
• Test Automation Framework (Google : CPAN TAF)
• Test Automation Framework Demo for Principles
– Simplicity/Easiness (Installation, TC magnt, and TC Report)
– Repeatability/Reliability/Reusability/Universal
– Independency/Adaptable/Extensibility
– Self-contained IDE /w command-line developer mode
22
24. Independency/Adaptable/Extensibility
QTP based GUI
Test
Non-GUI Test
TAF
Ta
sk
s
Test Suit (Acceptance Test)
Tools: QTP, AutoIt, Expect,
Selenium, Perl, JavaScript…
Defect Certification
Performance Test
Test Environ Monitoring
Equipment (CMTS/CM)
Monitor
Build Push/Installation
Quality Method: Req Mapping
Quality Method: Jira TC
Metrics
25. Test Automation Framework Demo
Test Bed Real Time Demo
•
Test Automation Challenges/Framework Design Principals
• Test Automation Framework (Google : CPAN TAF)
• Test Automation Framework Demo for Principles
– Simplicity/Easiness - point & click, cut & paste
– Repeatability/Reliability/Reusability/Universal
– Independency/Adaptable/Extensibility
– Self-contained IDE /w command-line developer mode
25
26. Test Automation Framework Demo
• Test Automation Challenges/Framework Design Principals
• Test Automation Framework (Google : CPAN TAF)
• Test Automation Framework Demo for Principles
– Simplicity/Easiness - point & click, cut & paste
– Repeatability/Reliability/Reusability/Universal
– Independency/Adaptable/Extensibility
– Self-contained IDE /w command-line developer mode
26
27. IDE command-line developer mode
Create Test Suite hook (index.pl)
c:_TAFtaf.pl testsuite=c:testsuite1;generateTAFTestsuite
Integrate Test Suite “e.g. calc test suite”
c:_TAFtaf.pl testsuite=c:autoIttestsuite;generateTAFTestsuite
Execute Test Suite/Case
c:_TAFtaf.pl testsuite=autoIttestsuite;exec
c:_TAFtaf.pl testsuite=autoIttestsuite;testcase=testcase0002;exec
Delete Test Suite from webUI
c:_TAFtaf.pl testsuite=c:autoIttestsuite;deleteTAFTestsuite
Capture Test Commands from webUI command
27
28. Test Suite Integration Example
Test Purpose: calc multiplication works correctly (2*2=4)
autoIt TS format: autoIt3 calc.au3 num1 num2
calc.au3 prints pass/fail and generate _tcLogAppend.txt
TAF test suite hook (index.pl)
Index.pl and index.pl 1
Integrate Test Suite
c:_TAFtaf.pl testsuite=c:autoIttestsuite;generateTAFTestsuite
28
29. TAF Test Bed Revisited
One click access to
• Test suite/case execution, test results and summary
• Test suite/case code, which are “hide” in webUI
• Test case logs, log in logs, url in logs
• TAF configuration and Test bed scripts
• TAF activity commands
• Access other TAF host on the internet
29
30. Test Automation Framework Demo
• Test Automation Challenges/Framework Design Principals
• Test Automation Framework (Google : CPAN TAF)
• Test Automation Framework Demo for Principles
– Simplicity/Easiness - point & click, cut & paste
– Repeatability/Reliability/Reusability/Universal
– Independency/Adaptable/Extensibility
– Self-contained IDE /w command-line developer mode
30
Clareice: Master’s Degree PMP certification from the Project Management Institute and is a Certified Professional Contracts Manager through the National Contract Management Association (NCMA). She has presented at the National Contract Management Association World Congress.
Clyneice:
She has participated as an examiner for state quality awards for Georgia and Virginia. She is currently an instructor for the International Institute for Software Testing and has presented technical papers at the Software Engineering Institute: SEPG Conference, American Society for Quality: Quality Manager's conference, Quality Assurance Institute International Testing Conference, International Conference on Software Process Improvement and Software Test and Performance Testing Conferences.
TC failure is a blue-screen, exception, unknown DB status, etc. Throw {} catch {} is hard to implement.