O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

upload ppt by browse button

470 visualizações

Publicada em

Publicada em: Negócios
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

upload ppt by browse button

  1. 1. Automated Testing with Team Test Unit, Web, Performance, Load, Manual, and Ordered Tests Code Coverage and Code Analysis
  2. 2. Quality Challenges “ Software bugs, or errors, are so prevalent and so detrimental that they cost the U.S. economy an estimated $59.5 billion annually, or about 0.6 percent of the gross domestic product…an estimated $22.2 billion, could be eliminated by an improved testing infrastructure that enables earlier and more effective identification and removal of software defects.” (Source: NIST 2002) Satisfying users Business interruption risks Ongoing maintenance costs rising Less avail. resources
  3. 3. How Does VSTS/TFS Help? <ul><li>Increased Communication and Integration… </li></ul><ul><li>Code Analysis Tools </li></ul><ul><li>Code Profiling Tools </li></ul><ul><li>Unit Testing and Code Coverage </li></ul><ul><li>Load Testing </li></ul><ul><li>Other Testing Tools </li></ul><ul><li>Test Case Management </li></ul>
  4. 4. Unit Testing <ul><li>Each unit is tested independently </li></ul><ul><li>A large percentage of defects are identified during unit testing </li></ul><ul><li>Automatic and repeatable code testing </li></ul><ul><li>Simplifies integration </li></ul><ul><li>Self documenting </li></ul><ul><li>Auto-generate </li></ul>Isolate small portion of code & determine whether it works correctly
  5. 5. Benefits of Automated Unit Testing <ul><li>Repeatable unit that verifies code still works </li></ul><ul><li>Confirm code is tested </li></ul><ul><li>Build a set of regression tests </li></ul><ul><li>Make changes with confidence </li></ul><ul><li>Aid in understanding code </li></ul><ul><li>Limits: integration, web and Windows UI testing, load, performance testing </li></ul>
  6. 6. Unit Test Types <ul><li>Class (standard) </li></ul><ul><ul><li>Test the properties and methods of a class </li></ul></ul><ul><li>Data-driven </li></ul><ul><ul><li>Bind unit test parameters to a datasource </li></ul></ul><ul><li>ASP.NET </li></ul><ul><ul><li>Test classes (or business logic) inside an ASP.NET application </li></ul></ul><ul><ul><li>Run in the context of the web server (ASP objects available) </li></ul></ul><ul><li>Web Services </li></ul><ul><ul><li>Define a web reference to the service </li></ul></ul>
  7. 7. Extensions of Unit Testing <ul><li>Regression Testing </li></ul><ul><ul><li>Modifications are validated with regression tests </li></ul></ul><ul><ul><li>Write new code and verify system continues to function correctly </li></ul></ul><ul><li>Integration Testing </li></ul><ul><ul><li>Helps validate how tested components interact with one another </li></ul></ul><ul><ul><li>Can identify problems that occur when units are combined </li></ul></ul><ul><li>Scenario Testing </li></ul><ul><ul><li>Combine sequence of unit tests to cover a scenario </li></ul></ul>
  8. 8. Writing Effective Unit Tests <ul><li>Atomic </li></ul><ul><ul><li>Should not require others tests to be run first </li></ul></ul><ul><li>Cover all cases </li></ul><ul><ul><li>One test for each sceneario </li></ul></ul><ul><ul><li>Cover all conditions, exceptions, nulls, etc. </li></ul></ul><ul><li>Able to re-run without configuration </li></ul><ul><ul><li>Database create/read/update/delete without having to modify the database before or after the test is run </li></ul></ul><ul><li>Test a common application state </li></ul>Unit tests should be …
  9. 9. Unit Test Conditions <ul><li>Success baseline </li></ul><ul><ul><li>Common, successful, most-likely call to your code </li></ul></ul><ul><li>Parameter mix </li></ul><ul><ul><li>Pass a varied set of parameters </li></ul></ul><ul><ul><li>Data-driven </li></ul></ul><ul><li>Bounds checking </li></ul><ul><ul><li>Stress the upper and lower limits of your parameters </li></ul></ul><ul><li>Null values </li></ul><ul><ul><li>Determine outcome of passing nulls to parameters </li></ul></ul><ul><li>Error conditions </li></ul><ul><ul><li>Trigger error conditions and validate expected outcomes </li></ul></ul><ul><li>Code coverage scenarios </li></ul><ul><ul><li>Test all conditions in your code </li></ul></ul>
  10. 10. Creating Unit Tests (1 of 2) <ul><li>Attributes are used to denote which methods and classes should be loaded for tests </li></ul><ul><ul><li>TestClass – denotes a class for testing </li></ul></ul><ul><ul><li>TestMethod – denotes a method for testing </li></ul></ul><ul><ul><ul><li>must return void (Sub for VB.NET) and have no parameters </li></ul></ul></ul><ul><li>Certain attributes are associated with initialization </li></ul><ul><ul><li>TestInitialize – run before each test </li></ul></ul><ul><ul><li>TestCleanup – run after each test </li></ul></ul><ul><ul><li>ClassInitialize – run once before running any tests in the class </li></ul></ul><ul><ul><li>ClassCleanup – run once after running all tests in the class </li></ul></ul><ul><li>Assert – static type used for asserting test values </li></ul><ul><li>ExpectedException – attribute used for determining the exception that should be thrown from the test </li></ul>
  11. 11. Creating Unit Tests (2 of 2) <ul><li>Methods on the Assert type </li></ul><ul><ul><li>AreSame / AreNotSame </li></ul></ul><ul><ul><li>AreEqual / AreNotEqual </li></ul></ul><ul><ul><li>IsNull / IsNotNull </li></ul></ul><ul><ul><li>IsInstanceOfType / IsNotInstanceOfType </li></ul></ul><ul><ul><li>IsTrue / IsFalse </li></ul></ul><ul><li>Other Assert types </li></ul><ul><ul><li>CollectionAssert – collection equivalency </li></ul></ul><ul><ul><li>StringAssert – string comparison/regex matching </li></ul></ul><ul><li>Web test types </li></ul><ul><ul><li>HtmlDocument – grants access to tags in the html document </li></ul></ul><ul><ul><li>Validate*/Extract* - types used for reading field values from the request and response </li></ul></ul>
  12. 12. Unit Tests
  13. 13. Code Coverage <ul><li>Improves effectiveness of tests </li></ul><ul><li>Show a measurable indication of code that was covered by unit tests </li></ul><ul><ul><li>Strive for 75% and higher </li></ul></ul><ul><li>Shows the ratio of executed logic to the total logic </li></ul><ul><li>Can be used in conjunction with unit and load testing </li></ul><ul><li>Helps to find unused code </li></ul>Determine code that is exercised by tests
  14. 14. Code Coverage
  15. 15. Static Code Analysis <ul><li>Provides a means to enforce coding standards </li></ul><ul><li>Configurable </li></ul><ul><li>Reduction in code reviews for small issues </li></ul><ul><li>Provides guidance on how to fix </li></ul><ul><li>Define a check-in policy to enforce code analysis pass </li></ul><ul><li>Coverage includes issues with design, globalization, interoperability, maintainability, naming, performance, reliability, security, and usage </li></ul><ul><li>Warning: developers can supress messages with attirbute [SuppressMessage(&quot;AdventureWorks.Rules&quot;, &quot;AW14441&quot;)] </li></ul><ul><li>MORE INFO see, “Writing Quality Code” in TFS documentation </li></ul>
  16. 16. Static Code Analysis
  17. 17. Web Testing <ul><li>Record page navigation and user interactions with web pages </li></ul><ul><li>Run tests back to determine errors </li></ul><ul><li>Seed tests from a database </li></ul><ul><li>Run tests from a group of users </li></ul><ul><li>Validation rules </li></ul><ul><li>Extractions </li></ul>
  18. 18. Web Testing
  19. 19. Load Testing <ul><li>Load Testing: ensure application works under expected, concurrent user load </li></ul><ul><li>Create test cases that simulate real user conditions </li></ul><ul><li>Distribute tests across cases based on user behavior </li></ul><ul><li>Assume standard think times </li></ul><ul><li>Distribute across connection types </li></ul><ul><li>Simulate an application performance in a production environment </li></ul><ul><li>Provide repository to look at a performance trend over time, to see if changes are helping or hurting </li></ul>Simulate multiple users against an application simultaneously
  20. 20. Stress Testing <ul><li>Stress Testing: determine breaking points in your application </li></ul><ul><li>Find scalability issues before the application is deployed </li></ul><ul><li>Step (increase) user load over time, monitor </li></ul><ul><li>Find breaking point and use to monitor application </li></ul><ul><ul><li>Change software </li></ul></ul><ul><ul><li>Add hardware </li></ul></ul>
  21. 21. Performance Testing <ul><li>Test at single user </li></ul><ul><li>Test at normal load </li></ul><ul><li>Test at peak load (1.5x normal load) </li></ul><ul><li>Measure against expected performance </li></ul><ul><ul><li>Peak load should allow for a 20-30% increase in metric </li></ul></ul><ul><li>Set thresholds on the output to avoid digging through data </li></ul>
  22. 22. Load, Stress, and Performance Testing
  23. 23. Manual Test <ul><li>Tracked like other tests when executing test groups </li></ul><ul><li>Presented to the tester to enter and confirm results </li></ul><ul><li>Defined as text file or Word document </li></ul>
  24. 24. Generic Test <ul><li>Wrap existing code and have that code executed as part of the testing process </li></ul><ul><li>Centralized results of all tests </li></ul>
  25. 25. Ordered Test <ul><li>Test to group (and order) other tests </li></ul><ul><li>Results of ordered tests either succeed or fail as a group </li></ul><ul><li>Used for scenario and module-level testing </li></ul><ul><li>Add any test except a load test to an ordered test </li></ul>
  26. 26. Manual Test
  27. 27. Test Case Management <ul><li>Organize tests into lists </li></ul><ul><li>Run tests </li></ul><ul><li>Filter and group the display of the tests </li></ul><ul><li>Import additional tests </li></ul><ul><li>Export tests </li></ul>Organize and manage test cases
  28. 28. The Test Lab <ul><li>Clients </li></ul><ul><ul><li>Tester machine </li></ul></ul><ul><ul><li>Used to create and edit tests / view results </li></ul></ul><ul><li>Servers </li></ul><ul><ul><li>Deployed test environment (web servers, load balancer, database server, etc.) </li></ul></ul><ul><li>Agents </li></ul><ul><ul><li>Computers running tests (multiple-threads, each a user) </li></ul></ul><ul><ul><li>Listen for commands from controller </li></ul></ul><ul><ul><li>Execute tests and pass results back to controller </li></ul></ul><ul><li>Controller </li></ul><ul><ul><li>Central computer that administers agents and collects results </li></ul></ul><ul><ul><li>Distributes load by applying weights to agents </li></ul></ul>
  29. 29. The Test Lab Database <ul><li>Run against a test version of the database </li></ul><ul><li>Test data should be same nature and size as production </li></ul><ul><li>Test a steady state of the database </li></ul><ul><li>Automate the initialization of the database </li></ul><ul><li>Extract data using rules to obfuscate personal information </li></ul><ul><ul><li>Visual Studio Database Developer </li></ul></ul>
  30. 30. Test Lab Database Initialization