This document discusses using cloud services for testing instead of maintaining local infrastructure and tools. It notes that testing in the cloud reduces costs by eliminating the need for expensive local infrastructure and tools. It also provides benefits like easy setup, centralized environment management, performance defaults, transparency and traceability of tests. However, it acknowledges that using the cloud requires technical knowledge for management and more standard cloud services may be needed.
This is one situation a cloud service may have come in handy. European air traffic was halted for days because of a volcano in Iceland. No one could get in or out of northwestern Europe. And what do you do when you’re stuck on your holiday location or wanting to fly home from work? You call the airline! But these airlines were smart, when you called them you got a recording telling you to check their website. So thousands of people did, and thousands more that were waiting for them. And what happened. The site got overloaded, their servers couldn’t hold it. And didn’t respond any more… A cloud could’ve helped those airlines. I case of such an emergency they could have added some servers and have a full functioning site. This is in a production environment. But the same problems appear in development and test environments.
Well, for software testing a number of technical tools are needed and these include test and acceptance environments and test tooling. However both test and acceptance environments and test tools are expensive to buy and, more importantly, to maintain. In addition, test and acceptance environments impose a large demand on the servicing of the infrastructure.
Why are testing environments so expensive to maintain? First you have program number 1. It needs the environment most in the beginning and a little bit after the second half of the year. But there are use laps in the demand vs. the availability. Program number 2 starts of high and demands even more at the end of the year. And program 4 has the highest demand in the start of the years. In most test environments I’ve seen they planned for this [second red line from above]. That lets you miss out on this peak [4 column from the right] and have a hole in the middle. There are environments that can facilitate almost all of the testing’s demand on the infrastructure. But you don’t need that availability the whole year. So you pay for maximum need all year round, but you don’t have the need for it! Having development, test and acceptance environments in the cloud can help with that.
Clouds can give you the sheer number of environments to perform all of your comprehensive testing. With cloud computing, standardization and virtualization it’s possible to create multiple (testing) environments in the cloud. Therefore reducing the need for expensive environments that have to be used at the times when the testing team executes its magic. Not only can it be used for the infrastructure, but clouds can also help with test tooling. Test tooling can be made available through clouds using SaaS [Software as a Service]. When you need the software they can be used whenever it is needed. And thus creating a greater flexibility for using these tools. So, we agreed the cloud has potential for testing. But this was all theory and that’s why we at Sogeti wanted to try this in practice. So we started a Proof of Concept together with IBM and their Development and Test Cloud to try out the test tooling and infrastructure in the cloud. We wanted to design and implement clouds to see how it could help our testing. We did this by using a real testing project for the pilot. One of our clients [in Holland] gave us the option to use one of their projects to try out this cloud. We created a cloud test environment for this client and set-up the testing tools within this cloud.
We created a test environment made up out of test management tools, test execution tools and the actual environment of the client for this project. In this environment we can now execute our test scripts <click> from within the test cloud and run it like a normal testing project. click> There is a challenge here. The IBM tool for automated test execution runs on Linux in the cloud, but we need to create the scripts first. To do this we created them in a Windows environment and transferred them to the cloud. We can now execute them from there. Now what were our findings. First the good news.
This cloud solution from IBM is fairly easy to setup. We used the standarized service and this could be done very fast. After this you need to install on this service, but once that’s done it’s easy going from there. The performance of the current cloud environment is sufficient for a normal sized project. The performance is scalable when using more or less resources of the cloud, but we don’t have figures about that yet. It’s possible to do central management over all the cloud environments. There are already a reasonable number of default settings set in the clouds (this includes Rational Tooling). And an important one, in terms of compliancy, is that the location of the services, whether is storage, servers or applications was traceable for us. It’s very transparent where your services are running. Of course you hope that IBM tells the truth about it, but what we saw was compliant. It’s even possible to segregate the data from the servers, but that was not needed for this pilot
Of course there where some challenges: I already mentioned the use of Rational Functional Tester first in a Windows environment and after that in a Linux environment. Enhance the interface to make it less technical. Because now technical knowledge is a must to manage the cloud More (standard) servers. Only a few non-IBM serverces are available. Like PayPal, SOASTA and Redhat. We needed more services so we used studs & drivers to create the environment. With the addition of other standarized services will be needed to give an added value. Move on to a larger project without ‘falling’ back in normal managing of the infrastructure.
When we can overcome these challenges we can use the cloud for other clients and give them advice on using it.
