O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.

2010-11 World Quality Report

1.436 visualizações

Publicada em

How well is your organization driving software quality and testing compared to other companies today?
-Which areas in Application Lifecycle Management (ALM) pose the greatest challenge to an organization’s success?
-How well do companies follow best practice standards?
-How quickly are companies adopting Agile methods?
-How are virtualized environments changing the way companies test?
-What skills do testers need in increasingly distributed teams?

The 2010-2011 World Quality Report provides insights into these questions and more. This report is based on survey findings from hundreds of testers, business analysts and developers and reveals emerging trends in quality and a forecast of how these trends will shape ALM and associated software in the future.


Publicada em: Negócios
  • Seja o primeiro a comentar

  • Seja a primeira pessoa a gostar disto

2010-11 World Quality Report

  1. 1. 2010-11 world quality report
  2. 2. contents Preface ForewordbytheAuthors 2 3 ExecutiveSummary 4 Introduction 5 TestingToolsTechnology 6 TestingOrganizationTesterProfile 11 OutsourcingTrends 12 FromAgileDevelopmenttoAgileDelivery 14 DeployingApplicationsontheCloud 18 IndustryTrends 20 EconomicImpact32 Summary34 AbouttheStudy35 AboutUs36
  3. 3. Preface•Raf Howery Jonathan RendeVP, Global Channels Partners Executive, Capgemini General Manager VP, Application Business, HP Software SolutionsWe are pleased to bring you the 2010-2011 edition of the HP’s Application Lifecycle Management business has longWorld Quality Report. This report is designed to help been a thought and market leader in the Qualitycompanies understand the trends in application quality and Management sector. Our continued investment in qualitytesting across different geographies as well as industry management, requirements management, service testing andsectors. The report also presents best practices to help agile process support is based upon a deep understanding ofquality assurance organizations learn how to do things better, industry shifts and trends. We are very proud to team withquicker, and cheaper. Capgemini and contribute to the new World Quality Report.We believe that it is important to continually develop We believe that the role of quality is increasing in status andmarket-driven solutions that meet our clients’ quality business relevance. It is no longer just a gate in the softwareassurance demands. In this report, we monitor business, development process. It is now, arguably, the top ingredienttechnology and economic trends in the software quality and for business success for an IT organization, that begins fartesting space. This year, we added data on the direction of IT upstream in demand and requirements management andinvestments and application quality in a number of specific extends far downstream into production. There is no doubtindustries: Consumer Products and Retail, Distribution and that quality is a top CIO priority. Quality organizations nowLogistics, Financial Services, Public Sector, High Tech and get involved earlier, have greater decision making ability andTelecommunications. We hope you find this report to be have far more business responsibility when representing thevaluable and that you gain insights into an increasingly end user community.important aspect of the Application Lifecycle Managementprocess. In our next edition, we intend to increase the Profound changes are driving how applications are built andstatistical sample and create an associated tool that allows delivered. Mega trends such as process and organizationalcompanies to benchmark themselves against their peers in shifts driven by the agile movement and cloud computingthe same market segment and other segments who may be are increasing in visibility and implementation. While themore mature, thus improving the learning process. worldwide business climate is showing some signs of improvement, I see that more organizations are leveragingWe want to thank HP for its continued collaboration and for the uncertain economy to transform their IT applicationhelping to make this study possible. Capgemini Group and portfolios in the best way possible.HP share a common culture, so it is natural to combine thedepth and breadth of our business intellectual property with I want to again thank Capgemini for the invitation tothe strength of HP in the testing market. Introducing the participate in the 2nd edition of this report. We are proud toWorld Quality Report is just one of many collaboration co-sponsor this project and see how the above changes willpoints between Capgemini Group and HP in our ongoing continue to drive positive change in the application industry.efforts to provide guidance to our clients. Most importantly, I’m excited about how organizations can now find better ways to deliver their applications.2
  4. 4. world quality report 2010-11ForewordbytheAuthors•Murat Aksu Charlie LiGlobal Head of HP Software Alliance, Capgemini VP, Global Testing Services, North America Leader, CapgeminiWelcome to the 2010-2011 edition of the World Quality Report2010 has brought many changes. The economies around the In this year’s report, we revisit quality trends and approachesworld are slowly on their way to recovery, IT spending is on and take a closer look at related industry-specific issues andthe rise, and companies are starting to reinvest in modernizing challenges. We also examine new trends in applicationtheir applications while software quality is once again moving delivery – such as using agile methodologies and testinginto the spotlight. cloud-based applications – and explore the effects that these new approaches are having on QA practices and techniques.What the recessionary years have taught us is that often theonly way to thrive in a difficult economic climate is to find We are committed to continuing this tradition for years toways to cut operating costs, while increasing speed and come – giving our readers a closer look at software qualityefficiency. To do that, companies need IT systems and tools, the state of quality, changes in QA organizations,organizations that can adapt quickly to changing market outsourcing and the recent economic effects in the testingconditions, perform without interruption and provide space. We would like to thank all IT professionals who tooksecurity and data integrity that’s expected in today’s the time to complete the survey and contribute to thisbusiness world. report. We would also like to recognize contributors from HP and Capgemini Group for their immense support andWith a combined 40 years of experience in Quality dedication to this research.Assurance (QA), Capgemini and Sogeti are often asked tohelp companies develop and implement quality processes We hope you find this report engaging and beneficial to yourthat would help ensure their IT applications’ reliability and company’s continued success.stability. Since 2008, Capgemini Group and HP – recognizedleaders in application quality – began conducting a formalglobal survey and publishing a yearly World Quality Reportto help clients stay on top of the latest trends in applicationquality, methodologies, tools and processes. 3
  5. 5. ExecutiveSummary•The 2010-2011 World Quality Report is a result of ongoing who also have a good understanding of developmentcollaboration between Capgemini Group and HP Software practices and methodologies. Close to 72% of all surveyedSolutions. It presents findings from the 2010 global survey of IT professionals say that testers with developmentCXOs, IT directors, quality assurance managers and engineers, backgrounds perform their tasks better than those withoutand examines the state of application quality and testing the technical expertise. This shows a trend for greaterpractices across different industries and geographies. convergence between the business, development and QA skill sets, and an increasing need for testers to take on aEach year, this report looks at current trends in software more active role in both the business requirement andquality and examines emerging tendencies that may affect development processes.the future of testing. Survey respondents are asked to •This is especially true for agile organizations. Agile isevaluate the level of investment in Application Lifecycle requiring the teams to work with a greater degree ofManagement (ALM) solutions in their organizations, rate the collaboration. QA can ensure that quality becomes part ofreturn on investment for these solutions, answer questions the application lifecycle by providing the right skill set –about their testing practices, and assess the general levels of technical knowledge of the development process and strongapplication complexity. This year, the survey also included business process expertise.specific questions about the adoption of agile delivery and •Agile delivery methods continue gaining popularity. Overtesting methods, and application virtualization. 60% of surveyed companies leverage agile, although most are still experimenting, utilizing agile only in a portion ofKey Findings new IT projects. This approach allows organizations toThe research revealed a number of key findings: adopt agile gradually. With this approach, they can identify•Although in 2009 most companies had to reduce their IT development and testing practices that work well for them spending, the vast majority of organizations (71%) are still – rather than switching a large number of projects to agile investing in new application development projects. IT without considering the implications on their organizational investments are shifting from a daily, operational focus structure, available skill sets and existing practices. towards building new applications that bring competitive •Organizations that have adopted agile see a significant advantage for the business. improvement in time-to-market and application quality,•Developing new IT systems with limited resources means although there are still challenges – specifically in that both developers and testers experience an increased facilitating communication and collaboration between teams. workload. Half of the survey respondents indicate that •Communication is also noted as the number one barrier resource cuts and heavier project loads have forced their preventing agile organizations from outsourcing application project teams to take on more work. Increased pressure delivery. However, the trend for outsourcing is not affected mandates the need for greater efficiency, more consistent by an organization’s development methods. Nearly 50% of QA methodology and better reuse of automated testing assets. organizations outsource application development and•Having a defined application lifecycle methodology is no testing whether they use agile or utilize more traditional longer viewed as a barrier for agility. More than 70% of all methods. Additionally, over half of organizations are surveyed organizations admit to following a specific ALM planning to increase the use of offshore resources in the methodology, and 82% of respondents indicate that testing coming year. The biggest growth is seen in Europe, where processes are being consistently followed in half or more of quality management outsourcing is outpacing other regions. their IT projects proving that agile methodology and testing •Along with improved agility in software delivery, IT processes are not mutually exclusive. organizations are aiming to gain agility in their applications•QA organizations increasingly require well-rounded and infrastructure. Virtualization enables companies to resource skill sets. While resources with strong testing skills improve capacity planning and save significant resources on and business knowledge are always important, an hardware and datacenter maintenance. Nearly three quarters increasing number of teams prefer to bring on board testers of all respondents are starting to leverage deployment of some4
  6. 6. world quality report 2010-11 Introduction• of their applications on the cloud, with only 8% experiencing To keep pace with current application quality trends and problems that caused them to roll back to physical infrastructure. technologies, Capgemini Group is surveying thousands of IT•Cost reduction is the number one reason for companies to professionals from around the world on an annual basis to move their applications to the cloud, followed by increased assess the current state of application quality, how companies agility. However, many companies are still unclear how to perceive their application quality and how quality is affected leverage cloud services for application testing. Cloud across different industries. infrastructure could offer many advantages in quickly building up test environments, hosting testing tools on the This year’s World Quality Report examines the effects of the cloud, or generating load for performance testing. We recent economic downturn on companies’ IT investments, believe that in the coming years, more companies will identifies emerging trends in using test automation tools and begin to realize these benefits, and we will see an even testing methodologies, looks at the changing profile of a QA greater adoption of cloud services. tester and analyzes current trends in QA outsourcing. The report also explores new technologies that have a direct impact on quality – agile application delivery and cloud computing. The 2010-2011 report also includes a detailed profile of the state of IT as it relates to quality in six specificIT organizations are industries: •Consumer Products and Retailaiming to gain agility in •Distribution and Logistics •Financial Servicestheir applications and •Public Sector •High Techinfrastructure. •Telecommunications We hope that the data points provided by the report – together with our experts’ opinions – will help you gain a better understanding of the current state of application quality and the trends that affect it. 5
  7. 7. Application complexity is increasing, Defect tracking and test management represent morebut quality is also on the rise traditional QA tasks, and automation tools for these phases have been available for many years. Solutions for the earlierToday’s IT applications support increasingly complex stages of the ALM are still relatively new on the market andbusiness processes. Not surprisingly, applications themselves represent an innovative approach to application delivery –are becoming more intricate and inter-connected. In our involving QA in all stages of the lifecycle. A growing numbersurvey, an overwhelming majority (88%) of survey of companies recognize that the cost of fixing problems laterespondents agree that software systems have increased in in the development phase is much higher than the cost ofcomplexity. In the past, organizations have been able to deal preventing problems by incorporating quality intowith increased workloads by adding resources to their requirements and use cases. Yet, they are still slow to adoptquality management teams, but with reduced IT spending, quality solutions for the entire lifecycle – largely due to theircompanies are looking at alternate ways of keeping pace with inability to accurately quantify Return on Investment (ROI)growing complexity – such as streamlining the quality on the solutions that they already own.process, increasing the degree of test automation andoutsourcing testing activities.However, despite poor economic conditions and limited QA Q: What components of your organization’s ALM processresources, IT organizations still seem to see a steady increase leverage automation technology?in software quality over the last two years. Over 80% of Figure 1: Companies are slower to adopt automationrespondents say that quality is actually improving. technologies for earlier stages in the application lifecycle.We attribute this to the fact that a growing number of }organizations are treating application quality as a formalprocess – incorporating quality methodology into theirapplication lifecycle. Nearly three quarters of respondents All of the aboveadhere to a defined QA methodology that they follow in themajority of their IT projects. We see this as a sign that Business IP-domain mgmtapplication quality will continue to improve and keep up Business flow modelingwith the increased complexity of business applications. Requirements Use casesBusinesses use more test automation, but struggle Test casesto realize full ROI on their solutions Defect mgmtThe use of test automation technologies is on the rise.Companies are increasing their investments in qualitysolutions and are becoming more skilled at using them.Our survey found that nearly half of all businesses reuse50% or more of their testing assets, ultimately leading toincreased testing efficiency, higher levels of test caseautomation and shorter testing cycles.However, a more detailed analysis reveals that testautomation technologies are more widely leveraged in thelater stages of the application lifecycle, such as test casemanagement and defect tracking, and not as commonly usedin the earlier stages, such as requirements gathering, usecases, and business flow modeling. (See Figure 1.)6
  8. 8. world quality report 2010-11Companies are increasingtheir investments in qualitysolutions and are becomingmore skilled at using them 7
  9. 9. Nearly half of all survey respondents are unclear as to theextent to which purchased ALM tools’ licenses are beingleveraged and whether they are generating the expected ROI.(See Figure 2.) Across all regions, only 4% of IT professionalsagree that all of their ALM investments are fully paying off,and only slightly more than a third of respondents said thathalf or more of their solutions have been fully implementedand are helping to improve application quality. QA organizations 100%need to focus on increasing the adoption rates of their Up to 80%purchased technologies. This will help demonstrate positive Up to 50%ROI and measurably improve their ability to secure funding Up to 25%for additional investments in automation technologies, Can’t quantifyespecially for the much-needed solutions in the earlier stagesof the ALM. Establishing formalized measurement programswill enable QA organizations to track relevant metrics drivenby business objectives. This will provide data and facts in abusiness context to demonstrate the value of investments inapplication quality.Not enough resources are invested in tool adoptionOne of the main reasons companies fail to fully leverage theirtest automation solutions has been the lack of planning andinadequate funding needed to deliver key services aroundthe deployment of automation tools, training and mentoringof resources. (See Figure 3.) } Q: What percentage of your purchased ALM tools’ licenses are being used and generating the expected ROI? Figure 2: Only a small number of companies believe that all of their ALM tools’ investments are generating expected ROI. Almost half cannot quantify the return on investment. Q: What are the biggest obstacles that prevent you from fully leveraging 100% of your ALM tool investments? Figure 3: Limited investment in adoption and lack of internal processes are leading the list of obstacles that prevent companies from fully leveraging their investments in ALM tools. } Not enough resources are invested in the adoption of the technologies No management support Lack of internal process Lack of integration among different technologies Technologies too difficult to fully adopt and/or lack of expertise Wrong choice of technology 8
  10. 10. world quality report 2010-11In order to cut costs at the time of technology acquisition, Companies believe in boosting investment inpurchasing departments often veto service-line items that automation technologieswould help with the planning and successful implementationof enterprise-level technologies. In some cases, the Survey results indicate that new and increased investmentorganization may purchase services for a successful initial needs to be considered for automation throughout ALM,implementation, but underfund ongoing maintenance and particularly for requirements visualization, testmentoring, thereby undercutting the future success of the environments, data management and business modeling.investment. In other cases, turnover in resources and project Investment in these areas is expected to generate higherchampions can cause automation efforts to slow down or quality applications, reduce rework, and decrease the cost ofcompletely stall, with QA teams reverting back to creating testing and defect management later in the applicationmanual test cases using word processing applications. In lifecycle. (See Figure 4.)extreme circumstances, QA teams diligently create thousandsof automated scripts without a clear strategic plan, only to 1. Requirements visualization and businessfind that their efforts go to waste due to drastic changes in flow modelingthe platforms and the functionality of applications under Requirements gathering and management have traditionallytest. It is important to note that simply learning how to use been a problematic area for the quality process. Most of thethe automation tool is not enough. To successfully requirements documents that are generated by the businessimplement a test automation project, it is essential to have a analyst community have been fairly complex anddeep understanding of automation strategies, frameworks voluminous. The fact that most requirements documents doand tools. In addition, having an experienced champion not accurately correlate to the actual, testable businessmentoring the project will increase the organization’s chances processes limits the ability of QA teams to extensively utilizeof meeting their quality objectives. In practice, it takes at them. If QA is unable to understand the requirements, theyleast three to five years of direct, hands-on experience to cannot effectively translate them into corresponding testdevelop automation skills that resemble a champion. scripts. The complexity of interpreting and tracing requirements to test scripts has undermined theIn order to fully leverage their investment in test automation, establishment of repeatable testing processes. This hasenterprises need to look at the big picture when acquiring resulted in ad-hoc testing rather than the requirements-basedand deploying tools. A well-defined implementation, validation that is the industry best practice.integration and adoption strategy with qualified professionalservices and training for internal staff can help push atechnology investment to exceed expected ROI and move theorganization closer towards their commitment to quality. QAteams need to learn to look at IT investments in the sameway business investments are considered and implemented. Requirements management } Test management and execution Q: Where would you like to see additional technological Environments and data management investments made in ALM automation? Business flow modeling Figure 4: Companies see requirements management, test Defect management management and environments and data management as None, we have all that we need priority areas for investment. 9
  11. 11. There are new technology vendors, such as Blueprint However, when asked to choose from a list of industry-Systems, that are attempting to bridge the gap between the standard methodologies, nearly three quarters of respondentsbusiness analyst and QA teams by transforming requirements say that they do not follow Test Management Approachfrom plain text into logical business flows that are supported (TMap®), Test Maturity Model Integrated (TMMI) or otherby visualization, prototyping and simulation. In some cases, common standards. Instead, their organizations develop andthe requirements visualization has been tightly linked to QA document their own best practices that are followed in theprocesses and can automatically generate use cases and test majority of development and testing projects. While having ascripts to reduce upfront testing cost and time. Time and common methodology is a positive trend, it is often open toeffort should be spent by the business in evaluating where interpretation. Different groups in the organization mayvisualizations will add the most value. adopt their own “versions” of the standard practice, and as a result, the company as a whole is not fully realizing the2. Environments and data management benefits of standardization, economies of scale, commonTest environments and data management are becoming a metrics, unified reporting and asset reusability. This isgrowing concern for QA teams. As application complexity especially true for large enterprises that give their divisionsgrows, it is becoming increasingly difficult for QA teams to and subsidiaries relative autonomy in choosing their testingcreate testing environments that mimic the real-life tools and practices. Also, many of these homegrown methodsinfrastructure and environment of production applications. are usually development-focused and do not specificallyIn some cases, government regulations and privacy concerns address testing. Implementing a standard testingrestrict the use of production data in testing environments. methodology such as TMap® or TMMI requires a company-Technologies such as test environment virtualization and wide commitment. Only when every IT group within thedata masking are increasingly becoming a necessity for QA company adopts the standard practices can the companyteams to manage these situations. achieve the quality improvement benefits offered by the methodology. Having a homegrown policy is stillIT organizations are more consistently using significantly better than not following any formal QAcommon methodology, but their testing practices methods, but it may not afford companies the full benefits and efficiencies offered by a more standard, widely acceptedare still mostly proprietary and not based on methodology.industry standardsThe growth in automation technologies is at least partiallyfueled by the increase in adoption of consistent testingmethodologies. As organizations mature, they realize that ad-hoc testing does not provide the advantage of reusing testingassets, standardizing on common metrics or having arepeatable quality process. This year’s survey indicates that asmany as 72% of IT organizations follow a defined ALMmethodology both in their development process and in theirtesting activities. The number varies slightly between regions,with the lowest number of ALM followers in Europe (65%)and the highest in Asia (78%).10
  12. 12. Business is looking for a new tester profile This clearly suggests that future testers must become better rounded in both business domain and technical skills. QA isHistorically, testers have been former developers who have becoming a more professional career where organizations areswitched careers to QA or business subject-matter experts expecting a smaller, but more skilled and highly developedwearing multiple hats. Today, the profile of a tester is workforce steeped in business knowledge. As applicationchanging rapidly. Due to the growing integration of IT and complexity increases, time-to-market challenges will requirecompany-specific business goals, companies prefer testers testers to have domain, application and technologywho have both strong technical skills and relevant domain knowledge in order to quickly dissect problems and provideand business knowledge. In addition to being knowledgeable fast and accurate information to development teams.about automation technologies, testers need to be able tounderstand business requirements and create test validation Along with tester profiles, the nature of a test organization isfor specific business needs. Organizations are also looking also rapidly evolving. Twenty years ago, most testers werefor testers who can provide actionable analysis to developers part of the overall development group and often reported tofor fast resolution of defects. developers. This limited their ability to be independent evaluators as the testing teams were under the influence ofMore than two thirds of respondents (72%) say that their developers. In the late 1990s and early 2000s as testingtesters have several years of business domain experience and started to become an accepted professional career, we startedan additional 18% indicate that their testers have full domain to see independent test organizations co-exist withexpertise. Business knowledge is the second most valuable development groups. In some instances, testing organizationsasset identified for a QA tester following QA skills. In-depth even started to report to VPs of Operations in order to bridgeknowledge of development processes and programming the gap between pre-production and productionlanguages came fifth in the survey behind database and management. With the advent of agile delivery, we expect toscripting skills. (See Figure 5.) see an increased trend that moves away from clearly defined development and testing organizations to a project team concept that combines the best talents from development, Q: When hiring testers, which of the following skills testing and business teams, working together towards a are most important to you? common business goal. Figure 5: Companies value business domain knowledge in testers above development skills. } Development skills (C++, Java,etc) Scripting skills (Perl, Python, etc) Database skills Mainframe skills Networking skills Security skills Business knowledge QA skills 11
  13. 13. Outsourcing continues to play a key role in quality Over a third of European companies (35%) prefer their QAmanagement and, by all indications, this trend will increase vendors to be co-located with their company employees,over the next few years. More companies are choosing to 26% prefer to outsource to India and 22% to near-shorefocus on their core business and outsource functions like locations. (See Figure 7.) Most European countries – such asQA to third-party vendors. the United Kingdom (UK) – have favorable laws that allow them to use affordable long-term local or near-shoreEurope is leading the way in outsourcing testing projects contractors. Many of the organizations that we researchedwith 65% of European respondents confirming their use of have QA organizations completely staffed by long-termoffshore testing. In Asia however, an overwhelming 85% of contractors (with 20 continuous years or more in somerespondents say that they do not currently use outsourcing cases) and niche local consultants. This would not beand are instead relying on internal resources for their testing possible in the United States (US), where independentactivities. This low level of interest in outsourcing is most contractors are traditionally more expensive than outsourcinglikely attributed to the fact that labor costs in Asia are still locations. Furthermore, large enterprise accounts in the USsignificantly lower than in Europe or in North America. have internal policies that limit the amount of time a(See Figure 6.) resource can be employed as a contractor before the contract must be converted into a full-time employee.Q: Does your organization use outsourced resources for testing?Figure 6: Europe leads the way in QA outsourcing trend, while Asia shows thelowest percentage – possibly because of lower labor costs in the region. Q: What would be your ideal location to outsource your testing activities? } Figure 7: Companies prefer to have their QA providers close to their employees, Yes followed by outsourcing to India and near-shore locations. No } 1% Other locations Near shore within my country or continent ROW Asia Europe India North America Prefer contractors co-located with my employeesSeveral key trends are emerging in outsourcing:1. QA outsourcing is on the rise, but companies prefer domestic or near-shore vendorsOf the respondents who outsource, over half say that they Europeare increasing the use of offshore QA resources, with another North America29% planning to keep their outsourcing at current levels.Only 19% of respondents are planning to reduce the numberof outsourced testing projects. Not surprisingly, the largestnumber of responses for reducing outsourcing investmentcomes from Asia and the smallest from Europe. In fact, 41%of European respondents say that their organizations areincreasing the use of outsourced resources for testing,compared to 30% in North America.12
  14. 14. world quality report 2010-11Naturally, when QA service providers are co-located with the 3. Automation could reduce the number ofcompany’s employees or are close in language and culture, outsourced resourcesthe flow of information and communication is much easier. Companies in Europe and North America that useThe exception to this trend is outsourcing to India, but we outsourcing without proper governance often find that theirbelieve it can be explained by India’s leadership and outsourced QA teams grow uncontrollably large as a result ofexperience in providing QA services, cost advantages and having to add more people to solve quality problems. Thisdeveloped infrastructure that supports communication and type of unchecked growth, coupled with rising costs ofcollaboration. offshore testing services, threatens to negate any cost-saving benefits achieved by outsourcing. In the survey, most2. Outsourced resources are becoming more skilled respondents say that their organizations can reduce the and business-oriented number of outsourced testing resources by increasing theirSimilar to the trend that we saw in internal QA hiring, levels of automation. With more QA automation and acompanies are beginning to place a higher premium on higher-skilled labor force, companies would feel comfortablebusiness and domain knowledge when selecting their reducing the size of their outsourced QA teams and projects.outsourced service providers. One of the biggest challenges isthat vendors do not provide enough skilled resources with 4. New value-added services demanded fromthe business and domain knowledge of the client. Only 14% outsourcing partnersof all survey respondents say that their outsourced resources There is a divide between organizations that are looking topossess relevant domain and business knowledge. continue business-as-usual by simply augmenting their QA teams with offshore resources and organizations that areIn the earlier phases of outsourcing, companies were seeking looking for more value-added services from their outsourcingcheaper resources to augment their capabilities at lower cost. vendors. The more mature organizations are now looking toWith rates at overseas outsourcing centers increasing, gain more specialized services, such as industry-specificcompanies in North America and Europe are re-evaluating quality solutions with pre-built requirements, test cases orprofiles. In the near future, companies located in the Western automation from outsourcing vendors. On average, 45% ofhemisphere will require outsourcing resources to provide respondents say that they would increase outsourced testingdomain intellectual property (IP) knowledge, industry- resources if they could have more value-add from vendors. Aspecific experience and highly technical testing skill sets that further 21% say that, while they would not increaseare backed by robust best practices. This shift in demand has outsourced resources, they would still be interested inresulted in a rise of many vertical-specific service providers vendors’ solutions and IP. Similarly, 36% of respondents saidand has forced other service providers to further organize that they would use outsourced testing services if theytheir solutions into verticals. offered expertise and methodology in testing-specific applications such as SAP. This clearly indicates that in orderOver a third of European to continue the successful relationship with their clients, outsourcing vendors need to offer more specialized, value- added services, and not just be an extension of the client’scompanies prefer their QA workforce.vendors to be co-located withtheir company employees. 13
  15. 15. By Ken Brennock, Manager, Agile and Automation Practice, Sogeti, and Brad Hipps, Senior Manager, Applications Solution Marketing, HPAgile development methodology took the IT world by storm Q(A): Do you leverage agile development/delivery methodswhen it first emerged nearly 10 years ago. It was a dream- in your company’s ALM?come-true for developers who no longer had to spend weeks Figure 8: Companies in all industries leverage agile methodsgathering project documentation, and instead could focus on across projects and technologies.what they liked best, writing code. However, most }importantly, it held a big promise for business and ITexecutives who were tired of taking on huge projects only tosee them fail, run over budget, miss deadlines and deliverlittle or no business results. Companies big and small began Yesturning from waterfall and other traditional development Nomethods to agile, hoping to improve businessresponsiveness, make their applications more adaptable tochanging market conditions and enhance the quality of theirIT systems.Nevertheless, the road to agile wasn’t always easy. Somecompanies tried to switch all their major developmentprojects to agile only to find out that their IT and QAorganizations did not have the required skill set to supportthe new development methods. Others, concluding that ROWfaster development means that formal quality management Europe Asiaprocesses are no longer needed, attempted to assign all QA North Americafunctions to Research Development (RD) engineers, (A)negatively impacting the quality of the finished product. Stillothers tried to force agile on large, globally distributed Q(B): Where in your organization is agile developmentteams, without taking into consideration the high degree of being applied? (select all that apply.)communication and collaboration required to work in an New developmentagile environment. Large projects Web technologiesDespite many challenges, agile is here to stay, and it is Client server technologiesgaining popularity. Our survey and other industry studies Maintenance workindicate that over 60% of all companies have either already Small projectsadopted agile methods or they are planning to do so in the Mainframe/legacy technologiesnear future. Companies of all sizes, across all verticals and For non-IT projectsgeographies, are adopting agile for both new and existingprojects to give a competitive edge to their business.(See Figure 8.)There is also a correlation between the types of IT projectsthat companies continue to invest in and the growingpopularity of agile delivery methods. Most companies arefocusing their IT investments on projects that generate (B) Source: A commissioned study conducted by Forrester Consulting on behalf of HP, April 2010.immediate ROI, and agile is ideally suited to deliver Base: 113 IT decision makers with insight into budgeted modernization activities currently using agile development approach.functional applications fast and with predictable quality.14
  16. 16. world quality report 2010-11Agile can helpsignificantly improveapplication quality,productivity andtime-to-market, but it isnot a cure-all remedy. 15
  17. 17. the way we see itBest practices adopted by successful agile companies include: 3. Find the right skill set Agile brings development and testing functions closer1. Set realistic expectations together, but it does not automatically turn developers intoAgile can help significantly improve application quality, good testers, or make testers more familiar with theproductivity and time-to-market, but it is not a cure-all development process. Organizations need to take a closerremedy. Perhaps the most significant benefit of agile is that it look at the skill set required by the QA team to effectivelyprovides a much more realistic measure of progress. Instead support agile methods.of gauging the effort, developers are measuring true progress,which brings them closer to the business and makes them Agile testers need to be more versatile than traditionalmore aligned with the stakeholders’ expectations. testers. On the one hand, they must be more technical, more familiar with development practices, and comfortable withSurvey respondents say that time-to-market is the greatest using non-traditional test automation tools to validatebenefit that their organizations realized by adopting agile. Graphical User Interface (GUI)-less applications. On theAgile is helping IT teams deliver applications on time and other hand, they need to be close to the business towithin budget while minimizing rework due to its iterative understand the requirements, work with end-usersand predictable nature, greater productivity and faster throughout the project, react quickly to change and tiereaction to change. application quality directly to business value. Because agile teams are typically small (six to twelve people) or a2. Do not ignore quality collection of smaller teams (“scrum of scrums”), an agileMany companies claim that they are moving away from tester can be compared to someone working in a smallwaterfall because they are discovering issues too late in the company where everyone wears multiple hats. With the rightapplication development process when it is too costly to do skill set and a strong understanding of all aspects of theanything about them. Unfortunately when they move to business, development and testing, QA can truly make agileagile, they find themselves falling into the same routine – an effective and successful method for the company.develop in fast sprints, but still test at the end of an iteration,or even after several iterations. 4. Emphasize communication Some agile “purists” may say that companies cannot haveQuality should be part of every step of any agile agile when they have distributed teams. In contrast, thedevelopment process. Each sprint should be composed of survey results confirm that the reality of today’s business issome amount of development and testing. This is the only distributed teams, and many of them are successfullyway to identify issues and project risks earlier in the lifecycle. adopting agile methods. Nearly half (46%) of all respondentsWith the right approach, agile organizations can achieve leverage offshore resources in their agile delivery. Thesesignificant improvements in quality. In fact, surveyed results clearly indicate that it is possible to be distributedcompanies perceive quality improvements as the second and be agile. The key is to stay connected. The two biggestbiggest benefit of agile – behind only time-to-market challenges when working on agile projects with outsourcedadvancement. (See Figure 9.) partners are poor communication and lack of mutually accepted collaboration technologies. (See Figure 10.) It is ourQ: In which areas has your organization seen an improvement experience that in offshore situations, companies mustas a result of moving to an agile delivery method? emphasize the management of cultural and language differences since they often directly contribute toFigure 9: Agile yields significant improvements for IT organizations. communication problems. } Through proper work planning, team structure, central asset repositories and open communication, distributed teams can achieve agile success. Cost savings Quality of application Time-to-market Resource utilization16
  18. 18. world quality report 2010-11Q: What is the biggest challenge working with offshore resources 6. Organizations may need to find their own, uniquefor agile delivery? form of agileFigure 10: The biggest challenge when working with outsourced partners Agile can be more complicated than traditional developmentis lack of communication and lack of collaboration technologies. methods, and the complexity is not only about writing code and testing at a faster pace. Agile changes the very way IT } interacts with the business. Adopting agile means finding the balance between speed and quality, responsiveness and predictability, embracing change and economies of scale, Agile delivery cannot be successful leverging offshore resources minimal documentation and accurate reporting. Lack of automation Lack of collaboration technologies Many organizations find success in taking the best of Lack of communication/ traditional methodologies and applying them to their agile communication problems projects, creating a unique hybrid model that works for We do not have any challenges with the offsore resources them. Perhaps some testers are more used to gathering for agile delivery requirements than user stories, or specific project teams need more granular reporting to satisfy regulatory compliance. Companies can find an agile method that’s right for them. The main principles of agile – building in small iterations, hands-on communication with the stakeholders, rigorous testing and receptiveness to change – are going to continue to work, even if teams add elements from other methods. The end result will be higher quality applications and greater business value.5. Achieve speed through automation For the testing organization, agile represents both anAgile thrives on efficiency. Agile delivery teams need to make opportunity and a challenge. The majority of surveyedtheir processes as efficient as possible, and this means organizations (67%) are not currently using agile-specificincreasing the leverage of automation frameworks and test testing methods to support agile application delivery, whichoptimization techniques such as risk-based testing and suggests that the role of the agile tester is still not wellorthogonal arrays. Developing and managing requirements, defined. For years, testers have been trying to ensure thatgenerating test data, storing project assets, building and application quality be considered earlier in the lifecycle.running tests – all tasks that are run routinely – cannot stay With agile, QA can help make it happen by providing themanual without slowing down the process and creating right skill set, including technical knowledge of theunnecessary burden on the teams. Technologies and development process and strong business process expertise.techniques that are considered “nice to haves” in thetraditional application lifecycle have become necessary Agile is requiring teams to work with a greater degree ofenablers of agile success. collaboration. It collapses the silos making the entire team feel ownership of the product. Testers no longer have theTraditionally, test automation in an agile environment was back-seat role where they are forced to wait until thedifficult due to lack of application UI. Most available application is delivered to them. They are actively engagedautomation tools required a GUI to start building test from beginning to end. Agile is a different way of working.components and scripts. However, the emergence of the new It requires a new level of competency and commitment fromgeneration of test automation solutions allows testers to the QA organization, because it is not agile development, butbegin automating much earlier in the process, without the rather agile delivery that ultimately makes projects successful.need for the UI to be completed. This significantly improvesthe speed, quality and consistency of agile testing andcontributes directly to higher application quality. 17
  19. 19. By Dennis Corning, Senior Manager, Product Marketing - BTO Virtualization, HPHistorically, IT has planned for application capacity by If a company is planning to run a big promotion orallocating a dedicated physical server for each application or introduce a new, highly anticipated product, they must planproject. The general rule was to run each server at about for increased traffic to their web site and higher order50% capacity to allow for planned and unplanned spikes in volumes coming through their IT systems. Traditionally, ITactivity without degrading performance. In such would have had to purchase additional server capacity andenvironments, datacenters were expanding rapidly, and IT deploy it in their datacenter, requiring added staff and extramanagers did not have the ability to easily reduce the maintenance expenses. Using the cloud architecture, ITapplication footprint and use existing capacity in a more managers are now able to quickly replicate their applicationefficient manner. In reality, most servers are only utilized at environment on the cloud and have the capacity they needabout a 15% rate, and IT doesn’t have an accurate way to on-demand. The same holds true for applications that arepredict actual capacity needs. only used several days out of the month – such as accounting or financial systems. Application performanceTo minimize capital expenditures and gain better control of during those critical times is essential, but for the rest of thetheir IT resources, companies are beginning to take advantage month it may remain virtually idle, so capacity can be turnedof the delivery model known as cloud computing. In cloud off or re-allocated elsewhere.computing, applications and information are provisioned on-demand as a shared resource. Cloud architecture can be set Companies are still cautious about moving all of theirup as a “public” cloud – with services dynamically delivered mission-critical applications to the cloud. In our survey, onlyfrom a third-party provider hosted outside of the firewall; 4% of respondents said that their entire application“private” cloud – where a cloud-like architecture can be set landscape has been moved to the cloud, with the biggestup over the company’s private network (inside the firewall); group of participants (35%) indicating that up to a quarter ofor a hybrid model utilizing a combination of internal and their applications are now cloud-based. (See Figure 12.) Weexternal providers. are seeing an increased number of companies virtualizing their applications as the first step towards moving to a publicPublic and hybrid cloud customers typically pay for capacity or private cloud. We view hardware virtualization as anand applications on-demand, enabling them to streamline enabling technology of cloud computing. Virtualization istheir IT architecture and reduce operational expenses such as essentially commoditizing hardware. It enables IT todatacenter maintenance and power consumption. Our survey disconnect its applications and environment from thefound that IT professionals who deploy their applications on hardware and move applications quickly regardless of whatthe cloud view cost reduction as the number one benefit of platform they reside on. Virtualization eliminates configurationsswitching from the physical infrastructure. This is followed by at the hardware level by providing a layer of abstraction –increased agility – giving IT the flexibility to move the paving the way for companies toward cloud computing. }workloads around and add capacityas needed. (See Figure 11.) Q: What percentage of your company’s applications are virtualized by your internal IT department? Figure 12: The majority of companies deploy It is trendy a quarter of their applications on the cloud. Time-to-market } Increased agility None Cost reduction 100%Q: If you use cloud, what isyour company’s top reason? Up to 75% Up to 50%Figure 11: Nearly half of Up to 25%IT professionals deployapplications on the cloudto save costs.18
  20. 20. world quality report 2010-11Despite the Service Level Agreements (SLAs) offered bycloud vendors, moving applications to the cloud stillrepresents a major shift in architecture. Applications cannotbe deployed without thorough testing and validation. In fact,a quarter of all survey respondents indicate that they Q: If you host applications on the cloud, what are the top issues that you encounter within the first few months?encountered application performance issues within the firstfew months of moving to the cloud infrastructure, followed Figure 13: Just like applications deployed on a physical infrastructure, cloud applications need to be thoroughly tested to avoid production problems.by 20% who experienced security and vulnerabilityproblems. (See Figure 13.) }We believe that within the next 12 months, a growingnumber of organizations will begin to take advantage of thecost savings and agility that cloud computing has to offer. We have issues butFrom a testing perspective, our survey found that over half we do not know the reason We have not seen any issuesof the respondents’ companies have already begun to Infrastructure performance problemsleverage cloud architecture for their testing efforts, such as Application functionality issueshosting their testing environments on the cloud or generating Security and vulnerability problemsload traffic for performance testing from the cloud. Application performance issuesThe key to the successful adoption of cloud architecture is infinding the right balance between the predictability andcontrol of traditional datacenter environments and the agilityand speed of cloud-based deployments. With properprocesses and testing practices in place, organizations canminimize the risk and rationalize their IT infrastructureusing new cloud technologies. 19
  21. 21. Consumer Products and Retail: Using the Downturn to Drive Improvement By Brian Girouard, Leader, Global Consumer Products Retail, Capgemini Inthisyear’sreport,weselectedsixspecific industriestoprofile:ConsumerProductsandRetail, DistributionandLogistics,FinancialServices,Public Sector,HighTechandTelecommunications.Each industrysectioncontainsahigh-leveloverviewof dominantITtendencies,aswellasspecificfindings fromourITprofessionals’surveyonQAtrends,IT investments,outsourcingandadoptionofnew technologiesandmethods. Consumer Products and Retail (CPR) is one of the broadest industries, combining food and beverage manufacturers, makers of health and beauty products, and several other sub- sectors involved in making and distributing common goods used by consumers. CPR companies rely heavily on enterprise resource planning (ERP) and customer relationship management (CRM) systems, as well as supply chain management applications. Some companies have global instances of these systems to help manage their product development, supply chain, sales and marketing activities. Others have regional or divisional systems. Traditionally, the role of IT was seen as a supporter of the business functions, but not as an equal partner responsible for the success of the overall mission. This view is changing rapidly. With increased competitive pressure, growing business and supply chain complexity and the ever-increasing amount of information about consumer behavior and industry trends, the business is beginning to see IT as a collaborative partner. As a result, companies are placing a greater premium on aligning business and IT priorities. In particular, the area of predicting and monitoring consumer demand and balancing it with the supply of available products demonstrates the strongest synergy between business and IT. Demand for consumer products can often be volatile, and the recent downturn in the economy has shifted consumer buying patterns. To remain competitive, companies need to be able to capture, understand and respond to demand data from consumers in a timely manner. They need sophisticated, comprehensive and integrated IT systems that can accurately measure and predict demand trends, as well as tightly manage the product supply chain. A slight misbalance between supply and demand results in either too much product in the value chain or not enough to satisfy the customer’s appetite, sometimes resulting in empty shelves (lost sales).20
  22. 22. world quality report 2010-11When asked about the greatest challenges facing their CPR companies will need to make changes to their qualityorganizations in the application lifecycle, CPR respondents processes in the near future. Their agility and competitivenesshighlighted the following three areas: getting business depend on it. In addition, as many of these companiesconsensus on the right investment (27%), understanding produce products for human consumption that are subject tobusiness requirements (27%) and getting development right government regulations, they need to have systems and(20%). Not surprisingly, when asked about the relationship processes in place to ensure that products are being handledbetween business and QA, the majority of the IT correctly and can be recalled if necessary. Only integratedprofessionals from CPR companies (63%) say the business and reliable IT systems can truly support the entire productbelieves the only role of QA is executing test scripts. This is lifecycle – from manufacturing to sale or recall.the highest percentage among all industries, indicating asignificant disconnect between the business and QA. The recent recession has led CPR companies to focus on improving business and operational efficiency. OrganizationsThe lack of relevant industry domain expertise among testers are working on reducing their overall inventory andin the CPR sector also points to the misalignment among localizing their offerings to be more relevant for consumers.different organizations as well as a siloed approach to CPR companies are beginning to invest their IT dollars intoapplication quality. On average across all sectors, 10% of all getting closer to consumers and better managing their supplyrespondents say their testers have “very little” expertise, 72% chains throughout the business. The survey shows that CPRhave a “few years” and 18% consider themselves “true is among only three industries (the others were Media anddomain experts.” By comparison, CPR respondents say that Entertainment, and Telecommunications) that indicated an67% of their testers have little business knowledge, and none increase in IT spending – despite the challenging economiccan be considered industry domain experts. (See Figure 14.) conditions. Respondents from the CPR sector agreed that their companies are continuing to invest in new IT projects –The key to improving application quality is to re-examine the and not just those generating immediate ROI.relationship between business and QA and make quality atrue part of the application lifecycle – from gatheringrequirements to monitoring the application in production.This approach will also help improve the relationship withoutsourcing vendors. Almost one half of CPR respondentsindicate that their companies are leveraging outsourcedresources for testing and are planning to increase their use ofoffshore vendor services in the next two years. By streamliningcommunication between business and IT and making qualitya part of every stage of the application lifecycle, companieswill be able to get better results from their outsourcingengagements and achieve greater efficiency in theirquality processes. } Consumer Products and Retail Distribution/Logistics Energy, Utility and Chemicals Very littleQ: How much relevant industry domainknowledge do testers in your organization have? Few years relevant Financial Services industry domain knowledgeFigure 14: CPR testers have less business Experts in our industry domainknowledge compared to other industries. Healthcare/Life Sciences Manufacturing Media and Entertainment Public Sector/Government High Tech Telecommunications 21
  23. 23. Consistent with the other sectors profiled in this report, Distribution and Logistics: The Road to IT Transformation By Dennis Wereldsma, Worldwide Transportation Sector Leader, Capgemini Distribution and Logistics is a diverse segment that includes a variety of logistics and transportation service providers: freight forwarders, postal service providers, the maritime sector, rail and even airlines and airports. Naturally, each sub-sector has a very different approach to IT with a different set of priorities. However, in analyzing the companies in the Distribution and Logistics sector, we discovered that some of the most interesting trends in IT are emerging among the logistics service providers, so we chose to focus on the freight forwarding sub-segment for a more detailed analysis. Freight forwarding companies rely heavily on IT systems for their core transportation functions, such as scheduling, planning and dispatching. However until recently, there has not been a standard ERP-like packaged solution on the market to support this very specialized sector. As a result, each freight forwarding provider resorted to building its own custom set of IT applications. Depending on the size of the provider and the complexity of its operations, these dedicated systems often take several years to build and require substantial effort to maintain and keep up-to-date. The real issue with custom systems is that they can be extremely difficult to integrate when companies’ IT systems merge. Logistics companies typically grow by acquisition, with larger providers absorbing smaller players and their operations. After several acquisitions the company’s IT landscape becomes cluttered with redundant systems – some obsolete and others still supporting disjoined, specialized functions. These systems are too costly to maintain and too difficult to integrate with newer software packages, but almost impossible to retire. Our research shows that logistics operators are at a pivotal point in their IT transformation journey. Many companies are trying to define the roadmap for rationalizing their application landscape to achieve greater agility, security, reliability and compliance, but the change may not be as fast as in some of the other sectors – such as Financial Services or Telecommunications. Logistics is a low-margin industry that has traditionally invested most of its resources in keeping its vehicles running and goods delivered, not advanced IT technology. Only about 2% of logistics companies’ annual revenue is spent on IT, and over 60% of all surveyed logistics providers still admit that the alignment between business and IT needs improvement (Source: The State of Logistics Outsourcing, 2009 Third- Party Logistics, Capgemini).22