Recomendados
Mais conteúdo relacionado
Semelhante a How to Add Perfecto to Your CI
Semelhante a How to Add Perfecto to Your CI (20)
Mais de Lizzy Guido (she/her)
Mais de Lizzy Guido (she/her) (20)
Último
Último (20)
How to Add Perfecto to Your CI
- 1. How to Add Perfecto to Your CI Partners Tech Talk Webinar David Broerman Partner Enablement Manager Email: davidb@perfectomobile.com
- 2. Agenda • Intro to Perfecto • Who is Perfecto • Basic Architecture/Setup • Digital Challenges • Overview of the Perfecto CQ Lab • Continuous Integration Overview • What and why • CI workflow and tools • CI Example • Perfecto Solution © 2016, Perfecto Mobile Ltd. All Rights Reserved. 2
- 3. Intro to Perfecto Perfecto Offers: A Cloud Based Continuous Quality Lab enabling testing and monitoring of all digital platforms on real devices and real end user conditions, for enterprises adopting agile delivery processes.3
- 4. Intro to Perfecto Dominant leader & visionary Strong Ecosystem Perfecto @ a glance Working with Enterprises and Partners to deliver exceptional digital experiences. “Perfecto Mobile marks shift in mobile apps QA with Continuous Quality Lab” 4 Setting the bar for market success • Founded in late 2006, Business HQ in Boston • Over 300 employees globally • Fortune 500 customer base
- 5. Cloud Based Quality Platform For The Digital Era 5© 2017, Perfecto Mobile Ltd. All Rights Reserved. Open & Integrated into Dev/Test/Ops Any test tool Any CI server Any test framework Any IDE Any Location Code/Debug Functional Test Performance Test Production Insight One Lab for all Digital Platforms Reporting & Analytics Wind Tunnel Layer Customer Support Browsers Desktops Mobile Devices IoT
- 6. Architected for Digital Quality at High Velocity 5/1/2017 6© 2017, Perfecto Mobile Ltd. All Rights Reserved. Maximize Digital Test Coverage Maximize % of Automation 1. Cloud-first architecture total reliability, unlimited scale, secure, global and available 24/7 2. Platform connectivity layer™ (PCL) architecture • Making open source enterprise grade • Enables high test automation coverage 3. One Script, One Lab for all digital platforms 4. Embedded into your toolchain to streamline activities 5. WindTunnel™ Layer persona based user environment testing for true digital coverage 6. DigitalZoom™ Reporting provides cross platform visibility and fast drill down Quality Across Dev- QA-Ops Fast Feedback Loop FOUR PILLARS OF SUCCESS
- 7. Perfecto PCL™ - Architectural Workflow 5/1/2017 7© 2017, Perfecto Mobile Ltd. All Rights Reserved. Standard Open Source Platform Under Test Standard Appium Server/ Selenium Grid Standard OSS Test Code In Dev IDE’s Platform Under Test Standard OSS Test Code In Dev IDE’s Standar d OSS Standard Appium Server/ Selenium Grid Perfecto PCL™ Benefits Standard Open Source With Perfecto PCL™ Test across platforms Same day support for any platform using any test framework Enhanced test coverage • Full system level control • Handle interruptions: text msg, system msg • Test audio (e.g. voice commands) • Test image (e.g. check deposit) • Test environment conditions (Wind Tunnel™) Visual validation of screen layout Always On lab
- 9. Global Data Centers Network 95/1/2017
- 10. The Need for Speed is Driving DevTest Adoption Now 10 • Collaboration between Dev and Test is increasing • Continuous integration is accelerating the move towards DevTest • Performance testing is starting earlier and occurring more frequently • DevOps adoption is less mature compared to the evolution of DevTest • Automation is a fundamental enabler Dev Test Stage Deploy Unit Test Fn Test Perf Test UAT Monitoring Continuous Integration Continuous Delivery ChangingSDLC Perf Test
- 11. The Challenges – Velocity vs. Quality • Pain: • Market pressure compressing delivery schedules • Growing App complexity driving test case growth • Rapidly evolving test matrix • Goal: • Deliver feedback to developers faster • Complete test coverage within each sprint cycle • Strategy • Increase automation • Adopt continuous integration 11 And Or
- 12. Perfecto Continuous Quality Lab One Script, One Lab for all Digital Platforms © 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 13. User Centric Quality Lab with End User Conditions Using a Mobile App 1 2 Network Quality Cellular, Wi-Fi 3 Varied Device Conditions 4 Multiple devices Specific profiles 5 Location 6 Resource Conflicts 7 Interacting with sensors Peak time - Millions of transactions 8 Simple Functional Testing Is Not Enough 13
- 14. CI Overview
- 15. Agile software development: • Is a SW development method based on iterative and incremental development where requirements and solutions evolve through collaboration between cross functional teams. Continuous integration (CI): • Was created for agile development to enable better collaboration, speed up the development process and improve the product quality. CI Trends
- 16. Continuous Integration and Testing • Continuous Integration • Frequently merging all developer changes into a shared mainline to avoid merge conflicts • Continuous Testing • Applying test automation to the continuous integration process • Continuous Delivery • Extending the automation paradigm across the entire software delivery chain 5/1/2017 16© 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 17. Continuous Integration workflow 5/1/2017 17© 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 18. Continuous Integration and Testing with Perfecto 5/1/2017 18© 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 19. Perfecto Continuous Quality Lab Code Test Resource Reports SCM CI GRID PhonesDevices BrowserBrowser Debug Commit Build Parallel execution SLA Security Error Handling System LevelReal Devices Enterprise Grade Continuous Quality Lab IDE
- 20. CI Example Putting it all together with the tools and process © 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 21. Example CI Architecture – VS/TFS 21
- 22. Example CI Architecture – ALM
- 23. CI Tools – Test Engineer • Selenium test • Eclipse • CI Tool • Jenkins • Build Tool • Maven • SCM Tool • Git 5/1/2017 23© 2015, Perfecto Mobile Ltd. All Rights Reserved. plan code build test release deploy operate
- 24. CI Process – Test Engineer • Create automation tests • Maven Project in Eclipse (see references for how to install) • Create a pom file • Containing test build & dependencies (see resources for an example) • Run test on mvn locally • Using Selenium Maven plugin, or • Running local mvn command line: mvn test -Dtest=<filename> • Commit mvn project to SCM • Git code source repository (see reference for how to install) • Create a job in Jenkins (see next slide) • Run Jenkins job according to build trigger 5/1/2017 24© 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 25. Create a job in Jenkins - build steps • Upload & install native application • Copy the app artifact from the previous job & upload app to the lab (see next slide) • Set the source code management URL • For example Github • Build Maven test project • Specify the Root POM and the maven command • For example, test -Dtest=<filename> • Define a build trigger • Run your build 5/1/2017 25© 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 26. Native applications – build steps • Jenkins Build Job (on the dev side of things…) • Dev commits a change to git • Jenkins identifies a change in development • using “Poll SCM” as a “Build Trigger” configuration • Jenkins starts a build job • to build a new app (apk or ipa file) • once completed it calls the test step • Jenkins Test job • Copy the app from the build job to the test job • using build step “Copy artifact plugin" • The test job uploads the artifact to the Perfecto lab using the Perfecto plugin • using build step “Upload files to MobileCloud” • Deploy new app on all devices 5/1/2017 26© 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 27. Resources • Perfecto Jenkins Plugin wiki • https://wiki.jenkins-ci.org/display/JENKINS/MobileCloud+for+Jenkins+Plugin • Jenkins in the Perfecto community • https://community.perfectomobile.com/series/26788 • Working with Maven • https://community.perfectomobile.com/posts/915224 • What is Maven • https://maven.apache.org/what-is-maven.html • Plug Real Devices into Your Jenkins Process and Automate Testing • https://community.perfectomobile.com/posts/974054 • Example POM file • https://community.perfectomobile.com/series/26788/posts/1010021 • Perfecto Required Jars https://community.perfectomobile.com/posts/915224 • Maven Surefire Plugin • https://maven.apache.org/surefire/maven-surefire-plugin/ • Using ReportNG • https://community.perfectomobile.com/series/26788/posts/1101298
- 28. Reference - Maven & Git installation • Maven installation • Maven Eclipse plugin (M2Eclipse) http://www.eclipse.org/m2e/ • Maven local https://maven.apache.org/download.cgi • Setting up windows environment variables https://community.perfectomobile.com/series/267 88/posts/1010021 • Git installation • Git Eclipse plugin (EGit) http://www.eclipse.org/egit/download/ • Github Setup https://community.perfectomobile.com/series/267 88/posts/1010021 5/1/2017 28© 2015, Perfecto Mobile Ltd. All Rights Reserved.
- 29. Question/Answer David Broerman Partner Enablement Manager Email: davidb@perfectomobile.com
- 30. Wrap-Up Thank You for attending! David Broerman Partner Enablement Manager Email: davidb@perfectomobile.com
Notas do Editor
- Welcome everyone and thanks for attending today’s Tech Talk Session for Partners on How to Add Perfecto to Your CI My name is ….
- Before we get started with the Agenda, just a quick housekeeping item…at the end we should have a few mins for questions and answers. There is a panel in the webinar named Q&A to submit questions during the webinar, so if you think of something, please ask. If we are unable to get to finish questions for any reason, then we will send out the info to all attendees. Here’s the agenda…
- The Perfecto CQ (Continuous Quality) Lab platform is a cloud architected quality platform. One lab for all digital platforms, from browsers, desktops to mobile to IoT We have capabilities to mimic the real user environment with our Wind Tunnel Including Location, orientation, network conditions like 3g 4g, and everything you need to reproduce your users environment While providing Quality visibility and trending with our reporting and analytics for fast feedback loops Completely accessible/embeddable in your SDLC process and tools Accessible from coding to customer support To integrated in Any test tool, CI server, test framework, IDE and location
- How is our perfecto hosted cq lab different? There are six key capabilities that make our lab uniquely capable to satisfy all four pillars of success They are: Cloud is not hosting devices. It’s much more than that. We connect differently to platforms than anyone else. This removes instability and barriers to high automation. One lab for responsive web is an example, multiple screen sizes can be tested using the same script across both mobile and desktop browsers. Our solution is architected to allows us to detach our lab from the interface – allowing the lab to be embedded across tools We are the only vendor with the ability to perform persona based testing, defined as a parameter in you test scripts. We decouple the persona conditions form the test case, allowing the same test to be run across different personas (with different network, location, background apps, etc). It is unique in that no reporting tool on the market provides a comprehensive digital view with the flexibility to see digital platform in one view, and zoom in to the point of failure, by platform/and step in the flow.
- Our platform connectivity layer provides a many benefits over standard open source. Same day support was a big pain encountered in 2017 for those not using Perfecto. lack of support for latest iOS platform (iOS 10, iPhone7) - It took 4 months for Appium to support iOS 10 in a GA release. iOS10 adoptions was at 54% of the market within the first 6 months. With Perfecto, our customer saw 0 interruption. You can even test on beta platforms the day they are released. Open source has no beta platform support. This can be the difference between testing on new platforms a month or more before they are released to the public. The Perfecto hosted cloud has a built in implementation of Remote Web Driver. So you are able to use Selenium and Appium through Perfecto’s Server. So for example, instead of having to set up an Appium Server locally, and then tether devices (1 iOS device per mac or a couple of Android devices to a windows machine), you can just connect to Perfecto’s server instance through the cloud and interact with the devices like you had them in your hand. So the benefit there is you or your team not having to deal with a local server instance, plus not having to manage physical devices or plugging them into machines locally. A few other benefits with using Perfecto’s instance, is being able to interact with objects using visual, setting end user conditions (like GPS location, phone calls and sms messages), network virtualization (3G 4G), and capturing device vitals (CPU, Memory and Bytes in/Bytes out). The bottom line is that it’s the same Selenium/Appium that users are used to working with, plus the additional Perfecto functionalities that we bring to the table. This setup allows you and your teams to be more productive, test on real devices with those all important real end user conditions PCL (platform connectivity layer) provides control over how we interact with the device, without modifying the interface or the jailbreaking or rooting the device itself, while providing system level control to the device. It is this capability, the PCL, that enables complex flows to be automated, enabling you to get to 80- 90% automation.
- The first picture is inside one of Perfecto’s Secure Datacenters, with racks of servers and devices locked in cabinets. Notice in the second picture, that we have real devices connected, via a USB cable, to the Perfecto Handset Server (or HSS). The HSS, which is responsible for managing all device connectivity, gets its commands from the MobileCloud manager (MCM) – which is a server. The MCM is the component responsible for executing all of the scripts as well as storing within its repository and reports. Through this USB cable we get a live streaming from the device and are able to control the device (what we call system level control); including any device real user commands/gestures (such as tap, swipe, click etc.). We don’t jailbreak/root or take the devices apart. We purchase them directly from the manufacturer, or we can take devices from you or a customer and also put them in the cloud.
- Perfecto operates 14 global data centers, including North America, South America, Canada, Israel, Europe, India (Gujarat - Guu Ju Rat) , Australia, and China. Photos: Top Left & Right – WiFi paddles/repeaters enabling increased device density | Top Right – Another Picture inside one of our Datacenters Bottom Left, FCC approved cellular tower repeater | Bottom Right, GPS signal repeater Center: Global NOC located in Wakefield, MA
- The most common debate especially in mobile is the balance and tradeoff between velocity of app releases and the quality which impacts the end UX. Having the ability to bridge between these 2 is a key to keep up with the dynamic and fragmented mobile market. Perfecto Mobile enables mobile app developers and testers through its MobileCloud platform and the CI solution to assure both Velocity and quality – all from the Developer or testers environment.
- Perfecto can Help with our Cloud Based Continuous Quality Lab – with One script and one lab for all digital platforms
- Let’s dig into Quality. Mobile quality differs dramatically from web. Specifically it is the user conditions. Therefore Quality and Dev teams must look beyond testing just a function toward testing all functions across all user conditions. The objective is to set defining and controlling user conditions as a key required capability. This can be expressed as a day in the life of how you use your favorite app. Think about traveling from point a to b and various network, device, device resources consumed and potentially conflicting apps that are also running on your device. App performance is influenced by a variety of factors, such as degraded network conditions, frequent network transitions back and forth between cellular and WiFi, on board sensor or resource conflicts along with a non-stop stream interruptions – SMSs, notifications, system pop-ups and phone calls. These factors impact app performance. Mobile app quality programs must account for these factors during testing to consistently deliver high-quality end user experience.
- Utilizing automation testing and automated reporting we can deliver quicker feedback to the development teams as part of the CI lifecycle. Keeping pace with the mobile market requires the ability to quickly deploy new mobile apps and updates. DevOps team must stay on top of the continuous demands for new features, fixes and platform support. To stay ahead, you must adopt an agile testing cycle which is built into your application lifecycle management (ALM) process - delivering continuous integration. By implementing a continuous integration approach you will accelerate release cycles and deliver better quality apps.
- The workflow of a CI process Dev check in code Unit, system, smoke tests run Results should be fed from the Jenkins run to a process that distributes meaningful results to the managers, Dev, and QA teams
- Here’s how Perfecto adds to CI
- Here’s a more in depth example showing the Perfecto CI workflow There are many options to choose from when implementing your CI Next we will see an example CI process including specific tools. Example Tools in the industry include: CI Tools: Jenkins, TeamCity, Bamboo Build Tool: Maven, Ant, Gradle SCM code repository: Git, Subversion, TFS, SubVersioN
- The next few slides will demonstrate where the automation test fits into the CI process, what tools can be used, and how the different tools fit together. IMPORTANT NOTICE: The following slides contain examples on how you can implement your CI with various tools. Keep in mind that there are many ways that you can implement.
- In our CI example we will be using: Jenkins server as our CI tool Maven as our build tool Git code source repository as our SCM (source control manager) tool (in this example both the developers code and the test code are stored in the git repository)
- Develop your automation test. For this example, these are contained within a Maven project. (see the references slide/links for how to instructions) Create a POM file and add to your test project (see the references slide/links for how to instructions) Run test on mvn locally, to test/debug out your build & dependencies, and see that your Maven project compiles correctly. * This is a recommendation. This can be done using the Selenium Maven plugin or by running a Maven command line. Commit your test project, in your SCM, enabling your build server to access it. In our example, this is done with Git. (* If you would like to work with EGit (see the references slide/links for how to instructions) Create a job in Jenkins (see next slide for details) Run your job
- Build Steps : In the case of a native app: upload the app to the lab & install it on the devices you are testing (the next slide describes this in more detail) In the case of a web app, this step is not necessary Set the source code management URL and credentials This is where your project is located. For example in your git repository. This will enable Jenkins to retrieve the repository files to run your test. Build the Maven project (which contains the build process and project dependencies) Notice, in our example here we are using the Maven Surefire Plugin, if you use this, make sure to add it to your dependencies (see references for link) Define a build trigger You can define different build triggers depending on the type of test you are running. For example, your test can be triggered by a build of a new application which now needs to be tested. This would be defined as “Build after other projects are built”. Run your build to make sure your test is running as expected.
- For native applications there is an additional layer to the process. Since there is an application that requires a new build job every time there is a dev change. The Jenkins Build Job is part of the application developers part of the CI process. Build Job The build job will be invoked every time the developer makes changes to the native mobile application. It can be triggered for example by configuring “Poll SCM” to periodically check for code changes that the developer makes and create a new build. Once the job is complete: a new native application is created the test job is initiated Test Job This Jenkins job is triggered by the build job which results with a new native application. The app should be uploaded to the Perfecto lab repository, and then installed on all the devices being used for testing. You can do this using the Perfecto plugin (see resources for the wiki documentation on how to do this) As part of your test project, you will also need to install the new app on all the devices you are testing on.
- A few links to get you started with installing Maven and Git Eclipse plugins and on your local machine. Once you install M2Ecliplse, you can create Maven projects that include your tests. EGit will allow you to create a code repository with your test, which can be shared and used within your Jenkins test job.
- Let’s take a few minutes to see if we have any questions, and answer them.
- Thanks everyone for attending today…Please let us know if we can help with any additional information on these topics. Have a good day, evening or night wherever you may be.