A product organization at Ericsson wanted to improve their continuous integration practice. This presentation discusses the actions, challenges, differing perceptions and ways to proceed in the case. The presentation was given at Agile 2015 conference.

1. Stakeholder Perceptions of
the Adoption of Continuous
Integration – A Case Study
Eero Laukkanen (Aalto University),
Maria Paasivaara (Aalto University),
Teemu Arvonen (Ericsson)

2. What is Continuous Integration?
“Continuous Integration is a software development practice where
members of a team integrate their work frequently, usually each
person integrates at least daily - leading to multiple integrations per
day. Each integration is verified by an automated build (including
test) to detect integration errors as quickly as possible.”
- Martin Fowler
8.8.2015
2

3. Why This Study?
Continuous integration practices in the industry vary a lot and are
not all really continuous, but more infrequent.
“Automated Software Integration Flows in Industry: A Multiple-case Study” by Ståhl and Bosch, 2014.
Adopting continuous integration has social challenges, such as
adopting the mindset and setting clear goals.
“Challenges When Adopting Continuous Integration: A Case Study” by Debbiche et al., 2014.
Idea: Could we study individuals to understand these challenges
better?
8.8.2015
3

4. Questions
1. What actions were done to adopt continuous integration?
2. What challenges were faced during the adoption?
3. How was the adoption perceived by different stakeholders?
4. How would different stakeholders proceed with the adoption?
8.8.2015
4

5. Ericsson Organization
• Platform + Services
• Early market lifecycle
• 11 teams, >100 people
• Releases every 8 weeks
8.8.2015
5
Large-Scale & Distributed Complex Platform
Team
Development site
Operations site
Business system
component
Network system
component
Site CSite B
Site A Site D

6. Research Approach - Interviews
8.8.2015
6
Site A Site B Site C Total
Team
member
12 4 3 19
Product
owner
2 1 1 4
Other 1 2 1 4
Total 15 7 5 27
• What does CI mean to you?
• Why do you practice CI?
• Experiences of adoption
• Challenges of adoption
• What should be done next?

7. What actions were done to adopt
continuous integration?
8.8.2015
7
2011 2012 2013 2014 2015
Platform acquired
Started agile
Started building CI
on Site A
CI roadshows
Interviews
CI experts planted
Watchdogs

8. What challenges were faced during the
adoption?
Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
8.8.2015
8
“I don’t think we have
the capacity in the
both in the
development
organization to do this
at the same time as we
do everything else we
have committed.”

9. What challenges were faced during the
adoption?
Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
8.8.2015
9
“[A component] has
not been designed to
be tested or version
controlled.”

10. What challenges were faced during the
adoption?
Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
8.8.2015
10
“No, [the tests] are not
working. It's, maybe 30
or something like that,
that fails every time
and no-one cares
because we know that
they fail.”

11. What challenges were faced during the
adoption?
Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
8.8.2015
11
“It's many hours now,
the nightly build. It's
running several hours
and... I'm not sure how
many more test cases
we can actually add to
that before it starts
taking too long.”

12. What challenges were faced during the
adoption?
Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
8.8.2015
12
“There is a problem
that, some of our, our
test environments,
like, I say staging for
example, it doesn't
have full connectivity
at the moment.”

13. What challenges were faced during the
adoption?
Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
8.8.2015
13
“It's quite easy to write
[test cases], but I
experienced that the
test framework was
very buggy. So it was
difficult to do the test
cases.”

14. What challenges were faced during the
adoption?
Lack of Time
Difficult Components
Unstable Tests
Slow Tests
Insufficient Testing Environments
Disagreement on Tools
Global Distribution of the Organization
8.8.2015
14
“So, we use video
conferences, it's very
hard to say what [the
participants from Site
A] say, they turn to
each other, talk to each
other within that little
group, and eventually,
we're, hey, we're here,
could we say
something.”

15. How was the adoption perceived by
different stakeholders?
Most were positive on Site A
Some were critical on Site B
How CI was adopted
Enforcement of automated test cases
No clear vision or strategy
Some were neutral on all sites
Not using product CI
8.8.2015
15
“We have found quite
many faults from the
system, and also if
some modification
changes something
unintentionally, then
we notice that quite
well with the cases we
have at the moment.”
Site CSite B
Site A Site D

16. How was the adoption perceived by
different stakeholders?
Most were positive on Site A
Some were critical on Site B
How CI was adopted
Enforcement of automated test cases
No clear vision or strategy
Some were neutral on all sites
Not using product CI
8.8.2015
16
“I think it could have
been done in a much
more efficient way, if it
had been driven in a
more open-minded
manner and if it had
been, a
communication rather
than an idea which
was imposed.”
Site CSite B
Site A Site D

17. How was the adoption perceived by
different stakeholders?
Most were positive on Site A
Some were critical on Site B
How CI was adopted
Enforcement of automated test cases
No clear vision or strategy
Some were neutral on all sites
Not using product CI
8.8.2015
17
“I don't really see it. I
just see meeting notes,
I see focus area
discussion in our
employee meetings
that we should focus
on it, but I don't really
see the results.”Site CSite B
Site A Site D

18. How would different stakeholders proceed
with the adoption?
Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
8.8.2015
18
“We should clean the
vague signaling
between the systems,
such as databases. We
should standardize the
signaling between the
systems.”

19. How would different stakeholders proceed
with the adoption?
Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
8.8.2015
19
“I think the next step is
to have of course this
stage environment,
that would be the first
thing.“

20. How would different stakeholders proceed
with the adoption?
Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
8.8.2015
20
“Looking at CI for its
own sake, [people at
Site A] need to open
up and be much more
perceptive of the
overall system and the
requirements the
system have on ways
of working.”

21. How would different stakeholders proceed
with the adoption?
Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
8.8.2015
21
“When it comes to the
verification I think we
should really, really
think through which
nodes, which parts of
the system we want to
make automated test
cases for.”

22. How would different stakeholders proceed
with the adoption?
Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
8.8.2015
22
“We should probably
also have dedicated
resources, that are
experts with the test
framework.”

23. How would different stakeholders proceed
with the adoption?
Change the Architecture
Proper Testing Environments
Better Communication
Testing Strategy
Decide Who Creates Test Cases
Need Support for Creating Test Cases
8.8.2015
23
“The competence
needs to be spread
more. There has been
some trainings, which
are good, but if you
don't start the work
immediately then you
forget it.”

24. Implications for Practice
Architecture has a significant effect on CI adoption
Provide time to overcome the initial learning phase
Invest in distributed communication and ensure that every site has
the competences needed
8.8.2015
24

25. Thank you!
Questions?