2. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 break
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube
14:30 Extensibility - test Scribe, TCMIMport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
25. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 break
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube
14:30 Extensibility - test Scribe, TCMIMport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
33. VS2010 Test Tools
UI Test Automation Capabilities
Fast Forward
CodedUI
Test Infrastructure
34. Test Automation Investment Levels
0 No investment in any kind of automation. Just click and test.
1 Clean up of the Action log maybe re-execute, for more smooth Fast Forward.
2 Basic CodedUI. Only use the default generation, add your own assertion.
3 Advanced CodedUI. Customize the CodedUI and UIMap for optimization.
39. Cleaning the action recording
When: executing the same test case on
multiple environments or when using test data
iteration or … and don’t forget the shared
steps
45. Lab 12
Create a CodedUI from Test Case
Create a CodedUI by using the CodedUI Builder
Timebox: 10 minuts
46. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 break
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube
14:30 Extensibility - test Scribe, TCMIMport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
51. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 break
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube
14:30 Extensibility - test Scribe, TCMIMport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
54. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:10 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 break BREAK TILL:
13:15 MTLM, Scrum and other project Methodologies
13:45 11:00
Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube
14:30 Extensibility - test Scribe, TCMIMport
(extra time to finish the labs :)
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
55. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:10 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 break
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube
14:30 Extensibility - test Scribe, TCMIMport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
56. CodedUI Common Questions
• In what version what functionality?
• How to debug?
• Where can I find test run results for a test run?
• Where do I see these assertions are passed or failed?
• For which properties can I make an assertion?
• …
57. CodedUI Advanced Questions
• How does CodedUI work?
• Can I use CodedUI for performance testing?
• How to maintain CodedUI tests?
• How to execute in the build?
• …
58. CodedUI Challenges
• The playback failed to find the control with the given search
properties.
• Identifying controls with no name search criteria or changing
ID’s
• Selecting correct Search Configurations for a control
• How to select the particular value from combo box
• …find controls
59.
60. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:10 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 Lunch
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube
14:30 Extensibility - test Scribe, TCMIMport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
61. Execute CodedUI
During the build with Visual Studio 2010
For the next 1,5 hour, you must have:
- A successful running codedui
- Test agent and controller
- A build
- Testplan and a test case
62. Configurations for automated test execution:
C
Flavor C: Execution during build with Test controller… B
Preferred configuration above flavor
Part of Build Verification Tests on multiple test agents
(triggered by: build)
Configure Test Controller (don’t register it with a project collection )
http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestControllers
Configure Test Agents on clients (interactive mode)
http://msdn.microsoft.com/en-us/library/dd648127.aspx#TestAgents
Configure *.Testsettings file in solution to use Test Controller and Test Agents
Configure the build to run the tests (see B)
Pro: Con:
Test run on test environments No collection of test results in TFS
Tests run on multiple environments Harder to configure
Test Results in Build result Need for specific test client environments
Test Settings from VS
64. 2
Add new Test Settings file
Set remote execution and
test controller.
65. 3
Create a new build definition
Set the Automated Test Properties
66. 4
Queue a new Build
Watch the Test Agent and
Test Environment
67. Lab 15
Run the automated test from the build.
Look for the test results
Timebox: 10 minuts
68. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 break
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube
14:30 Extensibility - test Scribe, TCMIMport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
70. Configurations for automated test execution:
D
Flavor D: Execution from Microsoft Test Manager…than BVT
Other type of test
Part of Regression Tests
(triggered by: MTM user, right mouse click on test case, run)
Configure Test Controller (register it with a project collection )
Configure Test Agents on clients (interactive mode, can be the same as MTM)
Configure Lab Center in MTM to use test controller and create test ‘agent’ environment.
http://msdn.microsoft.com/en-us/library/ee390842.aspx
http://msdn.microsoft.com/en-us/library/dd293551.aspx
Associate CodedUI test with WI Test Case from VS.
http://www.richard-banks.org/2010/11/how-to-use-codedui-tests-watin-and-mtm.html
Pro: Con:
Test run on test environments Manually triggered by Tester
Tests run on multiple environments Hard to configure
Test Result in MTM and TFS Need for specific test client (same as MTM?)
Test Settings from MTM
79. Configurations for automated test execution:
E
Flavor E: Execution from MTM duringPreferred configuration above flavor C
Build…
Part of BVT Flavor D and E can be configured together
(triggered by: Build)
Configure Test Controller (register it with a project collection )
Configure Test Agents on clients (interactive mode, can be the same as MTM)
Configure Lab Center in MTM to use test controller and create test ‘agent’ environment.
Associate CodedUI test with WI Test Case from VS.
Create Build task to run MTC or MSTEST task for Test Plan
http://blogs.microsoft.co.il/blogs/shair/archive/2010/10/30/how-to-run-coded-ui-tests-from-command-line.aspx
Pro: Con:
Test run on test environments Hard to configure
Tests run on multiple environments Need for specific test client (same as MTM?)
Test Result in MTM and TFS
Triggered by build
Test Settings from MTM
82. 1
Configure additional environments in Test Plan properties.
Win 7 IE9 configuration
Win 7 IE8 configuration
TC MTLM Client
Win 7 IE8 configuration
Physical
Test Client
TFS 2010
Win 7 IE9 configuration
Physical
Test Client
83. 1
Get Test Plan ID
Get Test Suite
Get Test Configurations
with TCM.exe
84. 2
Create command line for Test Run for the different Test Configurations
"run /create /title:""Run on Host for specific Test Point"" /planid:16
/querytext:""SELECT * FROM TestPoint WHERE ConfigurationId=13""
/settingsname:""Automated Test on Windows 7 Environment IE9""
/testenvironment:""Win 7 IE9"" /collection:http://clemenes-
pc:8080/tfs/defaultcollection /teamproject:EmployeeApplication"
"run /create /title:""Run on test env for specific Test Point"" /planid:16
/querytext:""SELECT * FROM TestPoint WHERE ConfigurationId=12""
/settingsname:""Automated Test on Windows 7 Environment IE8""
/testenvironment:""Win 7 IE8"" /collection:http://clemenes-
pc:8080/tfs/defaultcollection /teamproject:EmployeeApplication"
86. 6
Queue a new Build
Watch the Test Agent and
Test Environment
87. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 Lunch LUNCH TILL:
13:15 MTLM, Scrum and other project Methodologies
13:45 13:15
Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube
14:30 Extensibility - test Scribe, TCMIMport
(extra time to finish the labs :)
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
88. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 Lunch
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube
14:30 Extensibility - test Scribe, TCMIMport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
95. # Release planning
meeting
Estimating and prioritizing the Product Backlog
# Product Backlog
Evolving
User Needs
Owned by Product Owner
Prio by Business Value
Test during the Release Planning meeting.
100. ‘only execute tasks who make an addition to
the product are worthy to execute’
101. # Sprint planning
meeting
Add PBI to sprint
Decomposition
Commitment
# Sprint Backlog
Tasks
Related to PBI’s
Hours remaining
Test during the Sprint Planning meeting
106. # Daily scrum
Track
# Sprint Review
# Sprint
Demo the value
Retrospective
Agile Practices – Improvement
Test early and often
Build and deploy continuously
Acceptance Test Driven Development
Emergent Architectures
Refactor
Test Driven Development
Agile Database Development
Pair Programming
107. Undone
End to End
Testing
http://www.scrum.org/scrumguides/
108.
109. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 Lunch
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube
14:30 Extensibility - test Scribe, TCMIMport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
110. Build
application
Provision
Run tests
‘clean’ env
Deploy
application
111. VS2010 Lab Management
• Reproduce the exact conditions of a bug or other development
issue.
• Build, deploy, and test applications automatically in a clean
environment.
• Reduce the time required to create and configure machines for
testing an application.
• Run multiple copies of a test or development at the same time.
• Enable members of a team to create and manage virtual
environments without requiring system administrator privileges.
112. MTLM Client
TFS 2010 System Center Virtual Machine
Manager (SCVMM)
Hyper-V Library
Host Shares
VM share
VM share
115. TEAM FOUNDATION SERVER INTERNET LAB INFRASTRUCTURE Prerequisites ?
SAN
Domain Controller 1. TFS Application Tier (AT)
• Windows Server 2003 SP2, 2003 R2, 2003 R2, 2008 SP2, 2008 R2
• Windows Vista SP2, 7
• Ports used by services 8443 team web access
8080 server
Internet LAB DOMAIN (could be same as CORP DOMAIN) 2. TFS Data Tier (DT)
• Windows Server 2003 SP2, 2003 R2, 2003 R2, 2008 SP2, 2008 R2
NLB External Users • Windows Vista SP2, 7
• Ports used by services 1433 SQL
Walkthrough VHD
Lab Management
3. TFS Build Controller
• Windows Server 2003 SP2, 2003 R2, 2003 R2 SP2, 2008 SP2, 2008 R2
• Windows XP SP3, Vista SP2, 7
TFS 1 VS/MTM Clients • Ports used by services 9191 controller
Application Tier (AT)
4 4. TFS Clients
• Windows XP SP3, Vista SP2, 7
• Windows Server 2003 SP2, 2003 R2, 2008 SP2, 2008 R2
VMM Server 5. VM Factory
6 • Windows Server 2003 SP2, 2003 R2, 2008 SP2, 2008 R2
• Windows XP SP2, Vista, 7
• Ports used by services 3389 RDP
137 UDP
9 139 TCP
VMM Library • For more information, refer to the quick reference poster:
8 Visual Studio 2010 LAB Management Guide – VM Factory Checklist
3
Test Controller 6. VMM Server
• Windows Server 2008 or 2008 R2. R2 is recommended.
• Ports used by services 8100 VMM Agent
TFS 135 WMI (49152-65535)
Build Create LabMan template 3389 RDP
Servers from golden image 7. VMM Hyper-V Host
• Windows Server 2008, 2008R2. R2 is recommended.
Build
VS/MTM 8. Test Controller
Deploy and • Windows Server 2003 SP2, 2003 R2, 2003 R2 SP2, 2008 SP2, 2008 R2
Client Build Golden Images
• Windows XP SP3, Vista SP2, 7
Machines Host VMs
• Ports used by services 6901 .NET Remoting
9050 Lab Agent
9. VMM Library
Primary Factory • Windows Server 2008 or 2008 R2. R2 is recommended.
2
TFS 5 • Ports used by services 443 BITS
Data Tier (DT) 80 WinRm
VMM Hyper-V Host
7
Secondary Factory
VM FACTORIES LAB MANAGEMENT
References
• Visual Studio 2010 Lab Management Guide
CORP DOMAIN • http://go.microsoft.com/fwlink/?LinkID=206935&clcid=0x409
• Visual Studio ALM Rangers VM Factory Guidance
• http://rangersvsvmfactory.codeplex.com/
Domain Controller • Capacity Planning Quick Reference Poster
Network Storage
Team Project Collection Planning Quick Reference Poster
CORP INFRASTRUCTURE • http://vs2010quickref.codeplex.com/
Visual Studio 2010 LAB Management Guide – Infrastructure
Visual Studio ALM Rangers
2011-03-23
118. Start Topics
9:00 Recap
9:15
9:45
BREAK TILL:
Create Basic CodedUI, from Test Case and Manual
UIMap and customize the code for optimization
10:15 Data driven Tests and assertions 14:30
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI(extra time settings fromlabs :) Execute
tests, test to finish the VS2010,
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 Lunch
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube Break
14:30 Test Scribe, TCMImport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
122. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 Lunch
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube Break
14:30 Test Scribe, TCMImport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
123.
124.
125.
126. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 Lunch
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube Break
14:30 Test Scribe, TCMImport
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing
128. Start Topics
9:00 Recap
9:15 Create Basic CodedUI, from Test Case and Manual
9:45 UIMap and customize the code for optimization
10:15 Data driven Tests and assertions
10:45 break
11:00 Troubleshoot CodedUI, Common Practices and questions
11:30 Configure the build to execute the CodedUI tests, test settings from VS2010, Execute
from VS2010 with test agents and analyze the results
11:30 Associate test automation with test case, execute automated test case from MTM,
configure the build to run from the build within MTM and analyze the results
12:30 Lunch
13:15 MTLM, Scrum and other project Methodologies
13:45 Lab Management - Virtualized Test Environment and execution Demo
14:15 Test info in the OLAP Cube Extensibility - test Scribe, TCMIMport
14:30 break
14:45 Create your own Data Diagnostic Adapter
15:15 MTLM usages with Azure projects
16:00 Closing