I have written article for TEsting Experience magizine. It\'s about who and why should write automated tests and why writing is different than maintenance

1. 14
June 2011
The Magazine for Professional Testers
printed in Germany
print version 8,00 €
free digital version
www.testingexperience.com
Improving the Test Process
ISSN 1866-5705
© iStockphoto.com/ jgroup

2. © iStockphoto.com/carlosphotos
Automation responsibility
by Zbyszek Moćkun
We are in the project meeting and discuss how to speed up the On the other of the hand, we have maintenance, which contains
time of software releasing. Suddenly someone from the end of activities like: run tests, report results, debug failed tests and up-
the room shouted: “Let’s automate our regression tests”. The idea date automated scriptsif there were changes in the application.
looks very interesting, but during further discussion about auto- There is no way to do this outside the project, so maintenance can
mation we found several questions that need to be answered. Be- be considered as a type of sub-project (internal project) which
fore we start to work on automation we definitely have to answer should be conducted within the budget and resources of the pro-
one of these questions: Who should be responsible for automa- ject. It will be very difficult to make quick changes to scripts (con-
tion and why? nected with new features/change requests to the application)
as an external project. So the changes should be done as soon as
The first step is the same as for any project - at the beginning one they were identified by one of the project team members.
should define a goal. We need to know what we want to achieve.
Do we want to automate only smoke tests (few tests that verify if Let’s summarize: Writing automated scripts is an external pro-
everything is ok), check all important features (about 20% of cov- ject, and maintenance is internal.
erage), or do we need to automate almost all of the functionality
of our application (about 80%). Please note that we can automate It’s time to go back to the second part of the question - who should
even 100%, but the costs of automation for the last 20% will be write the automated tests scripts, and who should be responsible
very high and in most of cases it’s not worth to do this (manual for the maintenance? Should it be done by developers or testers?
tests are cheaper). Below I will concentrate mostly on the most Does anyone know developers that like testing? I don’t think. It’s
popular and the best approachinvolving 20 to 80 percent of au- only the one reason but really important (if people do not do what
tomated tests. they like, they just change a work or their performance slowly
decrease). Another reason against developers is that they would
We must not forget about two resources that are always limited: have to learn about the test case management tools, where we
time and budget. If we sum up those (refer to the goal from the keep test cases, learn about the environment and about the tool
paragraph above) we realize something very important – auto- that we will be using for test automation. Learning new tools is
mation is a project. It’s unique, and we move within a triangle a time consuming process and we have to be sure that our invest
(scope, budget and time). returned in long term. Developers should rather concentrate on
unit tests, which is their level. So, why should automated tests be
Another question is - What kind of project is it? Is it external or written by testers?
internal in accordance with the application (main project) that
we want to test? Before I answer this question, let’s go back to au- Testers are specialized in application testing; they are trained
tomation. The task of automating can be divided into two phases: and should have a sixth sense. They know the testing technics
phase 1 - create new test scripts, and phase 2- maintenance (run and tools, so there is no need for additional training. In addition,
and keep up to date). The question is - should we treat both of when they write the automated tests, they gather knowledge,
them in the same way? not only about the automation tool but also about the tested
application. Thanks to this they can help us if there will be more
Of course not - Writing new test scripts can be outsourced to an work in the project (e.g., complicated features, need of manual
external project. The author (person who writes automated test regression, a lot of maintenance work in automated tests) or as
scripts) does not need to be involved in the application project a backup to testers to minimize the project risk. Learning auto-
since he rewrites regression tests. If manual test scripts are well mation tools is not accomplished in just few days, sometimes it
made, the author does not even need to know anything about the can take weeks. It’s crucial not to lose that knowledge. Developers
application. He knows the steps and the expected results, which go to other projects, where automation skills may not be desired,
should be enough to created automated test scripts. but testers’ knowledge will be needed again and again for writing
112 The Magazine for Professional Testers www.testingexperience.com

3. tests but also for maintenance. DEVELOPERS DEVELOPERS
INTERNAL
EXTERNAL
If we summarize the above, we see that there are four different
ways to write and maintain automated tests:
1. Developer Internal - This is a very interesting way, which I
tried to introduce in one of my projects. The idea was that
each developer delivers newly implemented features to- MA TESTERS TESTERS
INTERNAL
EXTERNAL
gether with the automated scripts. Unfortunately, it failed IN CR
for several reasons. We had to invest a lot of time in automa- TA EA
N AN TE
tion tool training (the cost was high as we had to train all
developers in the team). Another issue that we faced is the CE
decision on who would write the test scenarios that were to
be automated. Developers, testers? In the case of developers, Figure: Automation responsibility square
we would need to train them in how to write test scenarios
and how to use the test case management tool. If we chose
testers, we need to find a way for them to deliver scripts to Maintenance is connected with current application development
developers (this idea requires them to create test scenarios where new features have an impact on existing scripts that are
at the beginning of a sprint, before requirements go into de- used for regression tests. As testers (internal) know the test suites
velopment). Another issue (I think, the most decisiveone for and application best they are the ones that can recognize points
the failure) is whether automated tests are required for all that have to be changed and can introduce the changes quickly
features. There are always other important things to do, like without the need to send them to an external team. It’s very irri-
features that should have been finished yesterday, so writing tating to run test suites that we know have failed, because we are
automated tests takes second placeafter a while and auto-
waiting for changes to come from an external team. We should
mation dies. It is for the same reason that this approach can’t
note that failed tests which are waiting for update do not test
be used for maintenance.
anything, and we should cover this area by manual tests. Please
2. Tester Internal - The idea is that the same team executes note that it is very hard (and needs a lot of time) to find new er-
functional, regression and other types of tests and spends rors among several already known. This is why fixing or updating
any free time on automation. From our experience, however,
test scripts immediately is very important.
we know that there is no free time because there is always
something to do. In this approach, a lot depends on the test-
er’s personality. If he is “lazy”, he will automate the most fre- Employing on two teams (an external team which only writes
quently executed tests only. So if we just want to have really new scripts and an internal team to test the application and
poor automated smoke test suites, this approach is for you. maintain automated scripts – it can be even the same person
For bigger projects it will never finished with success. who has divided task in time) results in a process for creating
new ones, which is free of interference. We can achieve continu-
As described above, this approach is best suited for mainte-
nance of automated tests. Testers run test suites, report and ous gains of automated tests and coverage. If we sum this up, it
debug failed test scenarios. As automated tests should be ru- gives us shorter regression time and lower budget for testers in
nas often as possible (continuous integration approach) it is the internal project.
very important to make changes quickly. Testers in an inter-
nal team can do this very quickly because there is no need to An approach like this gives you clear division of responsibility,
send anything to an external team, which always takes time. ensures continuous growth of coverage, and achieves the aim of
In addition,the external team does not lose time on updat- a fixed budget without any risk to the application development
ing existing tests and can instead create new ones for new
project.
features.
3. Developer External – The idea is that we create an external
team composed of developers with a separate budget, with a
> biography
defined aim and resources. The use of developers takes over
Zbyszek Moćkun
all the minuses mentioned above for the ‘developer internal’
has more than 6 years of ex-
project. The biggest factor is that developers are not for writ-
perience in QA engineering for
ing test scripts, but for writing the application.
big as well as for small com-
4. Tester External - This idea combines all benefits from inter- panies. Currently he works
nal tester internal and external developer external. Testers as Head of QA Practice for
do what they are specialized in, it’s very easy to manage the Cognifide - digital technology
budget, and we have a backup for testers. I have managed agency.
several automation projects using this approach, and all of The author has been involved
them finished with success. Of course, it is always nice to in several projects of different
have a developer (it can be rotated role) assigned to an au- size and methodology where
tomation team to resolve complicated programming tasks. he could work on or introduce
Everyone knows that 80% of automated scripts can be done automation. His experience
by copy & paste using already written code, so there is no was used to optimize the time spent on automation and
need for special development skills (I believe that each tester getting maximum benefits. This is very important, espe-
should have basic development skills). cially in Agile methodologies, which is the area in which
he specializes.
Why do we need two different approaches for creating and for
maintaining automated test scripts?
www.testingexperience.com The Magazine for Professional Testers 113