Baixado 29 vezes
![© 2013 IBM Corporation
Software and Systems Engineering | Rational
IBM Quality Management Portfolio
Rational Quality Manager
Test Planning
Test Construction
Test Execution
Team Collaboration
Reporting & Dashboard
Gerenciamento
de Requisitos
Gerenciamento
de Defeitos
Rational Requirements Composer
Rational DOORS
Rational RequisitePro
Rational Team Concert
Rational ClearQuest
Teste
Functional e
Mobile
Teste de
Performance Teste de Integração e
Virtualização em Testes
Dados de
Testes
Segurança e
Conformidade
Rational Test Workbench
Rational [Performance] Test
[Virtualization] Server
AppScan
Optim](https://image.slidesharecdn.com/rationaltestvirtualizationsolutionfabioport-131001150611-phpapp02/85/Teste-Continuo-de-Integracao-e-Virtualizacao-de-Servicos-13-320.jpg)
Teste Contínuo de Integração e Virtualização de Serviços
- 1. © 2013 IBMCorporation Testes Contínuos – Automação/Virtualização Fábio de Souza (fabiossa@br.ibm.com) Especialista Rational, Software Group, IBM Setembro/2013
- 2. © 2013 IBMCorporation Software and Systems Engineering | Rational 2 Agenda Como garantir melhor Qualidade de Software? – Do “teste” ao gerenciamento de qualidade inteligente Solução IBM Rational de Virtualização em Testes – Rational Test Workbench – Rational Test Virtualization Server – Rational Performance Test Server Referências – Casos de Sucesso Requirements Uma Solução Inteligente para Gerenciamento de Qualidade
- 3. © 2013 IBMCorporation Software and Systems Engineering | Rational Hoje, Software conduz inovação para um mundo mais inteligente Transformando a forma que nós vivemos, trabalhamos, e atuamos 3
- 4. © 2013 IBMCorporation Software and Systems Engineering | Rational Qualidade do software é o principal problema entre todas as indústrias Software é mais culpado pelos principais problemas nos negócios do que qualquer outro produto fabricado pelo Homem. Qualidade ruim do software passou a ser um dos tópicos mais caros na história da humanidade – $150+ bilhões por ano nos EUA. – $500+ bilhões por anos em todo o mundo. Projetos cancelados devido a qualidade ruim são 15% mais custosos do que projetos com sucesso do mesmo tamanho e tipo. Fonte: Capers Jones, 2011 Based on 675 companies, 35 government/military groups, 13,500 projects, 50-75 new projects/month, 24 countries, 15 lawsuits
- 5. © 2013 IBMCorporation Software and Systems Engineering | Rational Desequilíbrio entre Qualidade e Velocidade Crescente Custo da Qualidade Crescente Complexidade do Desenvolvimento Custo, complexidade e velocidade tornam a qualidade impraticável De 60% a 80% dos custos estimados de desenvolvimento de software está no retrabalho* * Source: http://www.sei.cmu.edu/about/message/ Mão de obra terceirizada não é um modelo sustentável com o aumento dos salários globais Complexidade e tamanho do produto e da aplicação estão crescendo Produtividade é inibida já que equipes de testes não podem mais se manter num desenvolvimento ágil Heterogeneous Environments Public Cloud Private Cloud Data Warehouse Mainframe Enterprise Service Bus Directory Identity File systems Collaboration Web/ Internet Routing Service Third-party Services Portals Content Providers EJB Shared Services Archives Business Partners Messaging Services Traditional Testing Other costs Testing costs
- 6. © 2013 IBMCorporation Software and Systems Engineering | Rational Melhor qualidade do software requer novo foco Evoluindo de Testes para Gerenciamento da Qualidade Uma técnica de investigação feita para expor informação relacionada a qualidade sobre o produto ou serviço sob teste Gerenciamento da QualidadeTestes Monitoração sistemática e avaliação dos vários aspectos de um produto ou serviço para maximizar o potencial e atingir padrões de qualidade almejados “Eu tenho centenas de testadores & muita automação, mas tudo que faço é descobrir mais defeitos. Eu não tenho o problema de testar, Eu tenho um problema de qualidade.” – de um grande banco global
- 7. © 2013 IBMCorporation Software and Systems Engineering | Rational 7 Soluções do Passado e Limitações Custos de Mão de obra, de hardware e software para gerenciar labs de testes e ambientes Enormes Custos de Lab Dias/semanas desperdiçados esperando pela disponibilidade de um ambiente de testes Longos Ciclos de Testes Disponibilidade de ambientes de testes ocultam a habilidade dos desenvolvedores de testar apropriadamente Altíssimos Riscos Muitos recursos do lab de testes sub-utilizados e custosos Desenvolvimento e QA desperdiçam muito tempo em atividades improdutivas: instalação, configuração, tentativa/erro, etc. Uma porção significativa dos esforços de testes é impelida para tarde no processo resultando em custos de correção de defeitos na ordem de 10-100x
- 8. © 2013 IBMCorporation Software and Systems Engineering | Rational Tempo/ Prazo Qualidade Custo Risco Automação de Testes Multicanais 1 Continuous Integration Testing & Delivery 2Smarter Testing 3 O que é mandatório para um efetivo Gerenciamento de Qualidade?
- 9. © 2013 IBMCorporation Software and Systems Engineering | Rational 9 Significativament e Menos Custos de Labs • Redução em até 90% dos custos de infra-estrutura do laboratório de testes • Redução em 80% ou mais dos custos com a mão de obra envolvida na configuração dos ambientes de testes • Redução ou eliminação dos custos de invocar sistemas de terceiros para uso que não seja de produção, como web services pagos Reduzidos Ciclos de Testes • Ambientes de Testes podem ser configurados em minutos versus semanas • Mais testadores podem estar focados em testes, e não na configuração do ambiente de testes • Mais testes de regressão podem ser realizados independentemente da interface do usuário final, durante o desenvolvimento Menores Riscos • Desenvolvedores tem como testar o software antes no nível do Serviço/API • Equipes extensas trabalhando em diferentes partes de uma aplicação ou sistema podem efetivamente trabalhar/desenvolver em paralelo, virtualizando diferentes partes do sistema Solução de Virtualização em Testes da IBM Rational Uma Solução Inteligente para Melhor Qualidade
- 10. © 2013 IBMCorporation Software and Systems Engineering | Rational 10 Agenda Como garantir melhor Qualidade de Software? – Do “teste” ao gerenciamento de qualidade inteligente Solução IBM Rational de Virtualização em Testes – Rational Test Workbench – Rational Test Virtualization Server – Rational Performance Test Server Referências – Casos de Sucesso Requirements Uma Solução Inteligente para Gerenciamento de Qualidade
- 11. © 2013 IBMCorporation Software and Systems Engineering | Rational 11 O que é Virtualização em Testes? Virtualização em Testes permite a criação de “ serviços virtuais”: –Serviços Virtuais simulam o comportamento de uma aplicação ou sistema inteiro durante testes –Serviços Virtuais podem rodar em hardware comoditizado, nuvem privativa, núvem publica –Cada desenvolvedor, testador pode facilmente ter seus próprios ambientes de testes –Desenvolvedores e testadores continuam a usar suas próprias ferramentas de testes (Manual, Performance web, automação de testes na Interface do Usuário) Captura &Modela Captura &Modela Dependências do sistema são um desafio chave na configuração de ambientes de testes: Indisponível/inacessível: Testes são limitados por agendamentos de produção, restrições de segurança, contenção entre equipes, ou porque eles ainda estão sob desenvolvimento Taxas custosas para acesso a sistemas de terceiros: Desenvolvimento ou testes de serviços baseados em nuvem ou compartilhados podem resultar em taxas de utilização custosas Impraticável virtualização baseada em hardware: Sistemas são ou muito difíceis (mainframes) ou remotos (serviços de terceiros) para replicar via abordagem tradicional de virtualização baseada em hardware Heterogeneous Environments Public Cloud Private Cloud Data Warehouse Mainframe Enterprise Service Bus Directory Identity File systems Collaboration App Under TestRouting Service Third-party Services Portals Content Providers EJB Shared Services Archives Business Partners Messaging Services Databases Mainframe applications App Under Test Third-party Services Packaged apps, messaging services, etc. Virtual Services
- 12. © 2013 IBMCorporation Software and Systems Engineering | Rational Solução de Virtualização em Testes da IBM Rational Uma Solução Inteligente para Melhor Qualidade Rational Test Workbench é uma solução desktop que possibilita testadores/desenvolvedores a: – Capturar e modelar serviços virtuais – Testar serviços e aplicações antesque suas interfaces se tornem disponíveis e fazer testes de sistemas integrados (SOA, BPM) Rational Test Virtualization Server é uma solução de servidor que: – Fornece um ambiente central para virtualizar hardware, software e servicços heterogêneos para fornecer capacidades de teste 24x7 – Reduz os custos de infra-estrutura dos ambientes de testes tradicionais – Serviços virtuais podem ser construidos a partir da definição da interface do sistema para uma variedade ampla de protocolos, incluindo HTTP, web services, SOA, JMS, TIBCO, IBM WebSphere MQ, Oracle, etc. Rational Performance Test Server permite que os usuários do Rational Test Workbench reusem scripts de testes para conduzir testes de performance – Pde ser usado combinado com Serviços Virtuais – Coleta de estatísticas para identificação de gargalos Databases Mainframe applications App Under Test Third-party Services Packaged apps, messaging services, etc. Rational Test Virtualization Server Rational Test Workbench Developers & Testers Rational Performance Test Server
- 13. © 2013 IBMCorporation Software and Systems Engineering | Rational IBM Quality Management Portfolio Rational Quality Manager Test Planning Test Construction Test Execution Team Collaboration Reporting & Dashboard Gerenciamento de Requisitos Gerenciamento de Defeitos Rational Requirements Composer Rational DOORS Rational RequisitePro Rational Team Concert Rational ClearQuest Teste Functional e Mobile Teste de Performance Teste de Integração e Virtualização em Testes Dados de Testes Segurança e Conformidade Rational Test Workbench Rational [Performance] Test [Virtualization] Server AppScan Optim
- 14. © 2013 IBMCorporation Software and Systems Engineering | Rational As Aplicações Modernas •Leves •Atualizadas •Novas •Simples …mas ainda infelizmente conectadas a mais de 20 anos de legado de TI
- 15. © 2013 IBMCorporation Software and Systems Engineering | Rational Desafio: Limitações no ciclo de vida de desenvolvimento e de testes Ambiente Físico de Testes Sistema Sob TesteSistema Sob Teste Volume Constraints Volume Constraints 2hr. access window Maintenance Constraints Database / JDBCDatabase / JDBC Server/ Web ServicesServer/ Web Services Mainframe / MQ/CICSMainframe / MQ/CICS Smart Device / SD-IOSmart Device / SD-IO Problema: Restrições que penetram no desenvolvimento e limitam a produtividade dos testes
- 16. © 2013 IBMCorporation Software and Systems Engineering | Rational Solução: Virtualização em Testes Virtualização ouve e captura o comportamento e características dos sistemas back-end Ambiente Físico de Testes Volume Constraints Volume Constraints 2hr. access window Maintenance Constraints Database / JDBCDatabase / JDBC Server/ Web ServicesServer/ Web Services Mainframe / MQ/CICSMainframe / MQ/CICS Smart Device / SD-IOSmart Device / SD-IO MQ/CICS Web Services Smart Devices JDBC Ambiente VirtualAmbiente Virtual Captura & Modela Sistema Sob TesteSistema Sob Teste Ouve / CapturaOuve / Captura ArmazenaArmazena Test Case Repository
- 17. © 2013 IBMCorporation Software and Systems Engineering | Rational Solução de Virtualização em Testes possibilita a implantação de serviços virtuais em múltiplos ambientes MQ/CICS Web Services Smart Devices JDBC AV - SITAV - SIT Captura & Modela Sistema Sob TesteSistema Sob Teste Modela e implantaModela e implanta Physical Test Environment Volume Constraints Volume Constraints 2hr. accesswindow Maintenance Constraints Database / JDBC Server/Web Services Mainframe/ MQ/CICS SmartDevice / SD-IO MQ/CICS Web Services Smart Devices JDBC AV - FTEAV - FTE Captura & Modela MQ/CICS Web Services Smart Devices JDBC AV - UATAV - UAT Captura & Modela Real / VirtualReal / Virtual Repositório de Test Cases
- 18. © 2013 IBMCorporation Software and Systems Engineering | Rational Testes de Integração – Contínuos e Incrementais Interface do Usuário Lógica de Negócio Camada Técnica Serviço Backend/ Bases de Dados Test Driver Detecta defeitos antes e de forma mais barata Acelera a resolução Simplifica questões de regressão (criação de defeitos, priorização) Abordagem completa para desenvolvimento paralelo de componentes com um mínimo de problemas de integração
- 19. © 2013 IBMCorporation Software and Systems Engineering | Rational Define-se sistemas com restrições e serviços para virtualização Integration OperabilityUser AcceptanceSystem IntegrationSystemUnit Integration OperabilityUser AcceptanceSystem IntegrationSystemUnit Performance Desenvolvedor usa serviços virtuais para testes unitários do código na sua workstation – nenhum ambiente elaborado é requerido Equipes de Desenvolvimento e de Testes usam serviços virtuais cedo para testes avançados O sistema todo ou componentes do sistema podem ter a performance testada bem cedo no ciclo. Serviços virtuais podem ser usados quando componentes não estiverem disponíveis Mais tarde no ciclo, versões finais de pré-produção dos serviços são trocados a medida que se tornam disponíveis. Os testes finais são feitos utilizando os serviços reais Solução de Virtualização em Testes fornece benefícios através do ciclo de vida de entrega de software
- 20. © 2013 IBMCorporation Software and Systems Engineering | Rational Incrementalmente Testando a IntegraçãoIncrementalmente Testando a Integração Serviço/App Real Serviço/App Virtual Virtualização habilita Teste Contínuos de Integração Serviços, aplicações, sistemas são introduzidos dentro do ciclo de integração contínua de forma priorizada, controlada. Solução de Virtualização em Testes tem papel chave para os Testes Contínuos de Integração
- 21. © 2013 IBMCorporation Software and Systems Engineering | Rational 21 Testes de Performance com Solução de Virtualização em Testes Configure & Control Execute Virtualize Analyse 1700 New Orders Type A per second 5 Cancelled Orders per second 50 Order Edits per second 1 Heartbeat per second 750 New Orders Type B per second
- 22. © 2013 IBMCorporation Software and Systems Engineering | Rational Ambiente e Tecnologias Suportados • ActiveMQ • Email (SMTP, IMAP) • Files • FTP/S • HTTP/S JMS (JBOSS et al) IBM WebSphere MQ • JBoss MQ • SAP IDoc, BAPI, RFC & XI/PI • Software AG’s IB & IS • Solace • Sonic MQ • TCP TIBCO Rendezvous, Smart Sockets & EMS • Custom • CentraSite • Oracle Fusion • SCA Domain • Software AG IS, BPMS • Sonic ESB • TIBCO ActiveMatrix • UDDI • Web Services • WebSphere RR • WSDL • BPM • Databases • Log Files • .Net Objects • Bytes • COBOL Copybook • ebXML • EDI • Fixed Width • HL7 • IATA • Java Objects • MIME • OAG • SOAP • Software AG Broker Docs • SWIFT • TIBCO ActiveEnterprise • XML (DTD, XSD, WSDL) • Custom Protocolos de Mensagens SOA, ESB, Outros Formatos de Mensagens Nota : Suporte a protocolo customizado pode ser desenvolvido
- 23. © 2013 IBMCorporation Software and Systems Engineering | Rational 23 Tipos de Aplicações Virtuais Simples Resposta Hard-coded retornada para qualquer entrada Não-determinísticos Uma-de-várias respostas hard-coded Data driven Dado de entrada e/ou saída especificado numa fonte de dados externa (Excel, arquivo, banco de dados) Model driven, stateful Dado de entrada e/ou saída mantido num modelo de dados com relacionamentos complexos. Suporta CRUD e outros comportamentos com estado Comportamental Estende model-driven para fornecer funcionalidade pré- empacotada, p.ex.: carrinho de compras, feed de dados em tempo real, trocas, ordenação
- 24. © 2013 IBMCorporation Software and Systems Engineering | Rational 24 Workflow da Solução de Virtualização em Testes Usando Rational Integration Tester (parte do Rational Test Workbench) – Define-se a arquitetura do sistema na “architecture school” – Define-se protocolos • Importa-se especificações, p.ex.:. WSDL para web services • Grava-se conversas/comunicações – Define-se stubs e dados • Dados capturados durante gravação ou fornecidos em arquivo ou banco de dados – Publica-se os stubs para o Rational Test Virtualization Server (RTVS) Dinamicamente configura-se roteamento de mensagens pelo console web do RTVS Testa-se usando sistemas virtuais a partir de qualquer ferramenta
- 25. © 2013 IBMCorporation Software and Systems Engineering | Rational Solução de Virtualização em Testes: Como tudo se encaixa…
- 26. © 2013 IBMCorporation Software and Systems Engineering | Rational 26 Agenda Como garantir melhor Qualidade de Software? – Do “teste” ao gerenciamento de qualidade inteligente Solução IBM Rational de Virtualização em Testes – Rational Test Workbench – Rational Test Virtualization Server – Rational Performance Test Server Referências – Casos de Sucesso Requirements Uma Solução Inteligente para Gerenciamento de Qualidade
- 27. © 2013 IBMCorporation Software and Systems Engineering | Rational Solução de Virtualização em Testes da IBM Rational tem provido resultados mensuráveis e valor ao negócio Major telecom carrier Multiple test releases required additional test resources, increasing testing costs Solution: Virtualized services and created reusable test cases across multiple environments Results: Reduced testing time by 50%, saving $800K over 3 years A leading global financial services firm with assets of over $2 trillion Many disparate legacy format to be migrated to next gen payments system Solution: Virtualized third-party systems otherwise unavailable for testing Results: Reduced manual testing from 10 days to 10 minutes, saving over $7 million to date Major U.S. insurer Recognized manual efforts insufficient for complex SOA and web services Solution: Agile middleware solution created to match the legacy systems’ functionality Results: Reduced testing by 95% to 2 hours; reduced ‘rate filing’ validation by 94% to 320 hours Global manufacturer acquires competitor Migrate off rented infrastructure onto company’s standardized middleware platform Solution: Virtualized critical, unavailable systems during migration Results: Fully integrated in 6 months – 2 months ahead of schedule; saved significant rental costs and dependencies on third-party systems
- 28. © 2013 IBMCorporation Software and Systems Engineering | Rational 28 © Copyright IBM Corporation 2011. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others. www.ibm.com/software/rational
Notas do Editor
- #4 Software is the invisible thread woven through systems, products and services, helping companies bring new – smarter -- innovations to market. We see this in every day devices like cell phones and automobiles, where the real value comes not from the device itself, but from software that makes it different, or better than your old phone or your last car. We see this in innovative new services being delivered to customers over the web, or software being used to automate core business processes. The best and brightest companies use software to propel innovation, connecting customers, suppliers, systems, and a host of business modules in a single intelligent, adaptive network. When software is a critical component of a larger system, which can connect to other systems - an automobile to a global positioning system (GPS), for example, or a smart grid - that is when significant fiscal and societal impact can be realized. The convergence of physical devices and information technology opens up the possibility for all types of integrated systems. These systems deliver exponential value to consumers and the public. Some examples of these are… Smart electric grid iPod and iTunes Android and Google Maps Traffic management systems Fleet management systems Healthcare management systems
- #5 Whatever we are doing is not working
- #6 Key Message: Successful organizations understand they must innovate to improve software delivery capability and that c ost, complexity and velocity are increasingly making today’s quality paradigm impractical. Speaking Points: What is driving the need for change? Three primary factors have been developing over the past decade - : (Increasing cost of quality) With the rise in global labor wages , outsourcing and off shoring testing strategies as a way to drive down the cost of software development has reached its practical end of life and is no longer sufficiently changing the dynamics and cost of software quality. (Increasing development complexity) Today’s applications and manufactured products are increasingly complex . They comprise an unprecedented level of connectivity and dependency between systems, processes and infrastructure. Whether deployed in traditional software development or cloud environments, businesses are able to create products, systems and services that are increasingly instrumented, interconnected, and intelligent. While software is fueling this innovation and growth, new challenges for testing these composite, heterogeneous applications, products and services while keeping pace with development teams has increased. 3. (Balancing quality and speed) Historically, businesses have had to balance their ability to deliver quality against speed/time to market . Over the past several years, software development teams have been finding new and innovative ways to drive down cost while increasing their flexibility and software development productivity through agile development and the use of automated tooling. Test teams can no longer keep up with development’s increased agility and the velocity at which they deliver working software code to be tested.
- #8 Huge Test Lab Costs : Use of hardware-based virtualization or cloud based resources provides partial savings (20-30%) Installation and configuration of software is still very labor intensive Certain systems cannot leverage hw virtualization, e.g. costly third party services, mainframe applications, proprietary systems Longer Cycle Time : Investment in UI test automation has proven to reduce cycle time for regression testing Testing new functions still require to have an environment available to develop test scripts The time wasted waiting for a test environment is severely reducing the ability to do proper acceptance testing Higher Risk : Addressed through better collaboration between development and testing, better test planning, e.g. using Rational Quality Manager Too many “trivial” defects are still found late in the process by Quality Assurance teams
- #10 Key Message: IBM Rational Test Virtualization Solution can help improve software quality management and testing to drive down the cost of software development, cut risk to the business and reduce cycle time without compromising software quality. Speaking Points: (Drive Down Cost) Cost of software development is driven by the effort, hardware and software to configure and deploy complex test environments. Virtualizing complex test environments which may be deployed in traditional software development or cloud environments can aid in driving down cost. (Reduce Risk) Big bang integration issues discovered late in the development cycle increases risk to the project. Executing on-going integration testing much earlier in the cycle helps development teams identify and resolve defects sooner. (Improve Cycle Time) Increasing demands for the availability of complex test environments is negatively impacting development team velocity. Virtualizing services allows teams to reduce wait times and quickly deliver the necessary testing environments. Each of these measures of success can help customers: Avoid project delays and costs associated with traditional test labs (drive down cost & improve cycle time) Test third-party services, complex heterogeneous environments and applications through virtualization which enables test clouds (drive down cost & improve cycle time) Identify and respond to defects earlier by testing virtualized application and system components until they become available (reduce risk & improve cycle time) Share test environments across the team enabling parallel development (drive down cost & improve cycle time) Minimize test as the bottleneck testing unavailable services by virtualizing them, enabling more iterative, agile development (reduce risk & improve cycle time)
- #13 See announce : http://www-01.ibm.com/common/ssi/cgi-bin/ssialias?infotype=an&subtype=ca&appname=gpateam&supplier=897&letternum=ENUS212-177 What is IBM Rational Test Virtualization Solution? The IBM Rational Test Virtualization Solution focus on a key problem most organizations face: the time, and resources required to set up and manage test environments. Traditionally, people have been running around installing hardware, setting up application servers, database servers, installing application software, configuring all of that. Not only this is very capital intensive problem, but as environments have become more and more complex, this is also a very error prone process that typically involves a lot of scrap and rework. IBM Rational Test Virtualization Solution enables organizations to address that problem by Virtualizing complete stacks of software, hardware, and services, enabling developers and testers to stand up test environments in a matter of minutes vs weeks, to do that whenever they want, and in effect, start their testing much earlier than what has been traditionally possible. The net result: IBM Rational Test Virtualization Solution really can help organizations transform the way they deal with software quality by: 1. better managing their costs: Reduce hardware, software and labor costs associated maintaining complex test environments, 2. improve test cycle time: By reducing wasted time spent waiting on the availability of and setting up test environments), and 3. better manage risk in delivering software: By doing testing earlier, organization can avoid late stage integration issues IBM Rational Test Virtualization Solution consists of : - Rational Test Workbench - Rational Test Virtualization Server - Rational Performance Test Server
- #19 [22:12]We’re all familiar with running the unit tests the developers have made using Ant as part of the build process. If these tests fail, the build is considered to be broken and the build script generally stops. What we have been working on with many of our clients is how can we revolutionize this technique so we can get tests written by testers into this process rather than just relying on the unit tests written by developers into their code. These tests have a lot more business value than a developer unit test for the reasons we discussed before. This is technique #1: continuous and incremental integration testing. So, if we are going to be testing continuously, it’s no good executing these test cases manually, we’re going to need automation at the UI layer. This is a fundamental change in role for some of our testers, they will move from doing tests to feeding new tests into the test automation engine. Stephen Covey wrote that “a producer can invest one hour of effort and produce one unit of results, assuming no loss in efficiency. A manager, on the other hand, can invest one hour of effort and produce ten or fifty or a hundred units through effective delegation”. When we automate a test, we delegate execution of that test to the computer, freeing us to get on with adding more value elsewhere, perhaps concentrating on usability tests or building more automation. This is the only scalable way to be more agile, and yet not jeopardise quality. The trouble with this approach though is the requirement for the UI to exist. This, as we have already discussed, can delay testing. Luckily, our thinking with an automation mindset, coupled with modern approaches to building scalable application architectures can provide the answer: we can move the automation backwards to the lower layers, catching problems sooner and with greater ease. In the past this was not possible, as applications were monolithic, but the modern componentized approach enables testers to take on this role as the interfaces are exposed, and standard. At the same time, we need UI automation to become lighter weight. It’s no longer acceptable to have to write code to automate the testing of a UI. Many modern application tools are getting away from the developers writing code, instead they configure, so why do we still expect testers to code in order to automate? I know many testers are a big fan of open source tools, one reason being because they are considered free and thus easy to obtain but that is ignoring the biggest source of costs in testing: the time used by the people to do the testing. This includes the time taken to build and modify automated tests. Much scorn is directed at record and playback mechanisms for creating automated testing scripts that cannot be easily maintained, pushing an application change into a category where making the change, despite it being the best thing for the end user is now deprioritized because it would break the automated tests. This is clearly wrong. We want creating an automated UI test to be as easy as running a manual test, but for the results to be so easy to change later that we don’t blink. We cannot rely on gifted individuals choosing the one right approach to automation out of 50 wrong ones in order to make our agility possible. It has to be impossible to do automation badly. This combined approach allows us to isolate defects at an earlier stage in the development process, and report them in a more timely fashion to the developers as they occur. Today we see IDEs doing continuous compilation to highlight syntax errors and other benefits, all without someone raising a defect. Tomorrow we will see the results from continuous and incremental integration testing appear in the IDE in the same way, alerting developers in real-time to regression issues, allowing them to choose whether they need to quickly fix some code, fix a test case, or raise a work item for a larger piece of work to be prioritized. An interesting discussion for another time is whether “micro-defects”, defects that are known only to the developer writing the code, ought to be tracked for statistical interest.
- #21 We always want to be testing. Stub interfacing components out that are unavailable at the time and then re-introduce them when they are available. Continuous integration testing at a “system” level. When new components are introduced the automated test suite scan be run as regression tests. Thereby controlling the risk of extra functionality being deployed into the test environment by having mitigated the risk through earlier testing against stubs. This is an incremental and iterative approach to integration testing.
- #28 Key Message: Green Hat’s technology is real, available today and proven in the marketplace Speaking Points: Four examples of how Green Hat’s unique capabilities are driving real ROI: Major telecom carrier Business challenge: Multiple channels used to register and service customers. SOA environment featured B2B integration with channel partners. Early success meant production issues increased with additional projects Solution: Combined automation of integration platform with intelligent automation of QA process. Green Hat provided common assurance process across SOA lifecycle. Consumers could instantly utilize requirements to virtualize applications or services enabling testing even when services unavailable Results: Improved time to market: 30% increase in productivity“ Reduced complexity/risk: 40% reduction in errors. A leading global financial services firm Business challenge: Customer bought a next generation payments system. Impact of integrating was significant given many disparate, legacy formats Solution: Virtualized third party systems, otherwise unavailable for testing Results: Reduced cost of labor: 10 days of manual testing down to just 10 minutes; Reduced risk: Saved >$7 million so far, “Project would have been impossible without the tool” Major US insurer Business challenge: Needed flexibility to change its quote engine and customer service delivery more frequently without massive financial burden associated with manual testing. “Rate filing” cost $500,000 each time (external Professional Services and internal resources) Solution: An agile middleware solution was developed to match the legacy systems’ functionality. This development included new interfaces that enabled customer facing employees to generate additional revenue from each policy. Once developed, test scripts were stored in GH Tester software which enabled the team to quickly rerun and report results. Daily validation was necessary to ensure the daily deployments did not affect the quote engine calculations. In addition to validation, the team to regression test before and after each change. Results: Reduced cost of labor: User testing reduced by 95% to 2 hours, QA testing reduced by 90%, Total testing time per ‘Rate Filing’ reduced by 3,500 hours at an estimated cost saving of $76,000. Improved time to market: Simulation of quote engine transactions reduced by 94% to 2 hours, ‘Rate Filing’ validation reduced by 94% to 320 hours Global manufacturer acquires competitor – Business challenge: A cquired competitor and needed to migrate them off rented infrastructure onto company’s standardized middleware platform. Regression testing was essential Solution: GH Tester performed all required functions quickly and easily. Virtualization of unavailable systems whilst they migrated was critical Results: Increased time to market: Fully integrated in six months, two months early. Reduced risk: Saved significant rental costs and dependencies on 3 rd party owned system Identify/Qualify Lantana opportunities in your territory Complex applications with integration challenges (multiple technology, legacy, packaged applications) MQ Series, TIBCO, Software AG, System Z, SAP
- #29 Author Note: Mandatory Rational closing slide (includes appropriate legal disclaimer). Graphic is available in English only.