My lecture from Israel ALM UG about Testing in Visual Studio 2012. This was a overview for all the test abilities in Visual Studio 2012 include a demo project.
2. Visual Studio Testing Tools
Unit Test
Fakes
Coded UI Test
Getting Started
Advanced Coding
Web Testing
Load Testing
Questions
Summary
3. Team Foundation Server Reporting
Generalist Specialist
Manual Testing Some scripting
Creates scripts to set up
lab, create data
Strong scripting skills
Some
coding skills
Strong coding
Develops fully
automated testing
procedures
Expert
coding skills
Other
Automation
Tools
4. The primary goal of unit testing is to take the smallest
piece of testable software in the application, isolate it
from the remainder of the code, and determine whether
it behaves exactly as you expect.
• Testers Or Developers?
• Documentation
• Change Existing Code
• Avoid (Truck) Factor
5. • Multiple test projects
• Test Adapters
– Nunit , Gtest , xUnit++
– Chutzpah – JavaScript
• UITestMethod
• DataRow
• .NET regardless the target code
• Run Every Build
• Grouping – Outcome, Duration…
6. • Shim – A shim modifies the compiled code of your
application at run time so that instead of making a
specified method call, it runs the shim code that your
test provides.
• Stub - A stub replaces another class with a small
substitute that implements the same interface.
7.
8. • Functional Testing
• Generate code in C#/VB
• Intent aware recording and resilient playback
• Integrated with ALM story
– Build, deploy & test in lab or as part of build
– Local, remote runs, data collection
• Rich Extensibility
9. Fully supported platform
Partial solution. Further work
required to complete
Best efforts. May work with
known issues. No major
ongoing investment
Currently no support but on
the roadmap for future
releases
Currently no support and none
planned for now. Opportunity
for partners
Platform 2012 Update 1 Notes
IE7/8 – HTML/AJAX
Windows Forms 2.0+
.NET Fx Controls fully
supported, working on 3rd
party support
WPF 3.5+
.NET Fx Controls fully
supported, working on 3rd
party support
Windows Win32
MFC
HTML 5/AJAX
Chrome / Fire Fox Update 1
SharePoint
Silverlight
Citrix/Terminal Services
Office Client Apps
IE 6
Opera/Safari
Flash/Java
SAP
http://tinyurl.com/CuitCrossBrowser
10.
11. • Performance
• Security Testing
• No User Interface Complexity
• Faster Testing
• Easy Manipulation
• Data Driven Test
• Load Test uses Web Test
13. Load Test is used in
several types of testing:
Smoke Test.
Load Test.
Stress Test.
Capacity Test.
Performance Test.
Compatibility Test.
Hello ladies and gentleman, I'm very excited to be here in Moscow and presenting the topic of Applied software testing…My name is Shai, Shai Raiten I’m a ALM Senior Architect and Trainer working for Sela Group, I’m Microsoft Regional Specialist and for the past 4 year I’m VS ALM MVP.I would like to start my session with a short story….Once a poon a time when I just started to work as a QA engineer my boss came to me and said, this is your first day so you’ll learn how to use our Bug and Test Cases systems. The systems he was talking about was Excel and Word documents. So I’m familiar with Excel and Word and so I learn very easily how to use them.Later on my boss came a said that he bought a new system for Bugs management and another system for Test Case management and he told me to learn how to use them, of course we spend couple of weeks migrating all our word and excel files manually to those systems.Couple of months later he came again but now he said because he is so pleased with the new systems he bought he bought another system for automated testing and he told me to learn how to use them and then to write automated tests. So I learn and wrote tests, but when I complete writing tests on one system my company developed we went to write tests on another system and then we find out this automation tool isn’t good enough for the other system, and so my boss went and bough another system and of course I had to learn another tool and another script language.Again couple weeks later my boss came and said he very pleased with those automation tests and he want to move to the next stage – And he bought a tool for load testing – again I had to learn how to use this tool and learn another script language just for load testing.Software Testing הוא עולם רחב מאודמהיום שהתחלנו להיות אנשי בדיקות ביקשו מאיתנו כל הזמן ללמוד כלים ומוצרים חדשים. כלי חדש לניהול תסריטים עוד כלי לניהול תהליכי הבדיקות ואפילו עוד כלי על מנת שנוכל לבדוק את כיסוי ואייכות הבדיקות שלנו, אבל פה זה לא נגמר – אז אחרי כמה זמן התחילו לבקש מאיתנו ללמוד שפות סקריפט על מנת שנוכל להריץ בדיקות אוטומטיות (כמובן כל כלי והשפה הייחודית שלו) אבל גם פה זה לא נגמר כי עכשיו אנחנו רוצים גם לדעת להריץ בדיקות עומסים וגם פה אנחנו צריכים ללמוד עוד כלי חדש , אהה וכמובן אף אחד מהכלים אין אינטגרציה עם הכלי האחר.היום אנחנו הולכים להכיר את Visual Studio 2010 ואת סוויטת הכלים לעולם הבדיקות – מבדיקות ידניות לאוטומטיות וכמובן גם עומסים והכל בכלי אחד שתוכנן עבורנו.שלום שמי שי בלה בלה.....
אז על מה אנחנו הולכים לדבר היום:נדבר על הכלי החדש לעולם הבדיקות הידניות, ונראה איך בדיקה ידנית הופכת לאוטומטית ללא כל מאמץ.נדבר על Data Adapters ואיך הוא עוזרים לנו, וכמובן איך לכתוב משלנו.נעבור על CodedUI מההתחלה ועד הסוף.לקראת הסוף יהיה זמן לשאלות ואז נסכם.אז בואו נתחיל.
עכשיו לפני שמתחילים מה זה Visual Studio 2010?????אתם מבולבים? מה כבר מיקרוסופט עשתה עבור עולם הבדיקות? הרבה....
UITestMethod – Force unit tests to run on the main thread.DataRow – Specific data entries for the unit tests
Performance. Shims run slower because they rewrite your code at run time. Stubs do not have this performance overhead and are as fast as virtual methods can go.Static methods, sealed types. You can only use stubs to implement interfaces. Therefore, stub types cannot be used for static methods, non-virtual methods, sealed virtual methods, methods in sealed types, and so on.Internal types. Both stubs and shims can be used with internal types that are made accessible by using the assembly attributeInternalsVisibleToAttribute.Private methods. Shims can replace calls to private methods if all the types on the method signature are visible. Stubs can only replace visible methods.Interfaces and abstract methods. Stubs provide implementations of interfaces and abstract methods that can be used in testing. Shims can’t instrument interfaces and abstract methods, because they don’t have method bodies.In general, we recommend that you use stub types to isolate from dependencies within your codebase. You can do this by hiding the components behind interfaces. Shim types can be used to isolate from third-party components that do not provide a testable API.
Performance – check much more easily for response time from server site.Security – See client – server request and inject whatever you like.
מקליט את האתר IBuySpy מבצע את אותם הפעולות שעשינו ב – MTMמסביר על - > Detecting Dynamic Parametersמריץ ומסביר למה זה נכשל (היוזר כבר קיים – Validation Rules) – משנה את היוזר ומריץ שוב.עובר על היכולות של Web TestValidation Rule (מוסיף אימות על הצלחת קנייה ואחד נוסף לבדוק שהכמות שמופיעה אכן תואמת – זה יכשל כאשר נריץ מרשימת הערכים בגלל הערך הוא קבוע ומראה איך משנים).Request Properties – מסביר גם על Think TimeCall 2 Test – מפצל למספר בדיקות – Enter Home Page והרשמה לאתר – Registration.Loop - רק מדבר על Conditionמוסיף DataBinding ומחבר אל הבדיקותמראה את ה – Test Settings ואיך אפשר לשנות את סוג הדפדפן בבדיקה
Smoke How your application performs under light loads for short durations.Performance To determine how responsive your application is.Load testing. Use load testing to verify application behavior under normal and peak load conditions. This allows you to verify that your application can meet your desired performance objectives; these performance objectives are often specified in a service level agreement. It enables you to measure response times, throughput rates, resource utilization levels, and to identify your application's breaking point, assuming that breaking point occurs below the peak load condition. Stress testing. Use stress testing to evaluate your application's behavior when it is pushed beyond the normal or peak load conditions. The goal of stress testing is to unearth application bugs that surface only under high load conditions. These can include such things as synchronization issues, race conditions, and memory leaks. Stress testing enables you to identify your application's weak points, and how it behaves under extreme load conditions. Capacity testing. Capacity testing is complementary to load testing and it determines your server's ultimate failure point, whereas load testing monitors results at various levels of load and traffic patterns. You perform capacity testing in conjunction with capacity planning. You use capacity planning to plan for future growth, such as an increased user base or increased volume of data. For example, to accommodate future loads you need to know how many additional resources (such as CPU, RAM, disk space, or network bandwidth) are necessary to support future usage levels. Capacity testing helps you identify a scaling strategy to determine whether you should scale up or scale out.
דקה עד פה!מתחיל באשף הבסיסי עובר על האפשריות , מוסיף את הבדיקות Web וקובע את העומס הרצוי.אז ראינו כמה פשוט וקל לבנות בדיקת עומסים – בלי עוד כלים מיוחדים ולא צריך ללמוד הרבה בשביל זה.8.5 דקות דמו.