This session provides an in-depth look at best practices for component-based software development practices in configuration and build management. Learn how to manage the challenges often faced by companies with many products, diverse teams and daily releases.
Powerful Google developer tools for immediate impact! (2023-24 C)
Configuration and Build Management of Product Line Development with Perforce
1. Configuration and build
management of Product
line development
Steve Kim (Sungchul Kim)
Principal Engineer
Samsung SDS
( sungchul3.kim@samsung.com )
2. AGENDA
Product-line development
• Definition
• Considerations
Usage models
• Depot structure
• Branch Strategy
• Baseline Strategy
• Integration with defect tracking tools & process
• Build and Release
Demo
4. Introduction
Enhance the efficiency of SW development when multiple
products are to be developed simultaneously
• Higher productivity
• Higher quality
• Faster time to market
• Lower labor needs
Many methods and practices are introduced
• S/W Reuse
• Component-based development
• Product line engineering ( Product family engineering )
5. Introduction
Variation management is a key element to distinguish
the other development process
• Reusable S/W Architecture
• Separated teams and responsibilities
• The governance enforcing S/W reused
The usage model of configuration and build management on
product line development will be introduced
• Code structure
• Branch strategy
• Label strategy
• Change management
• Build and Release management
6. Definition
Definition of Product-line development
• A set of related products are produced through the combination
of reused core assets together with product specific custom
assets
[ Adapted from “General configuration management and asset evolution model for software product line”, Softwareproductlines.com ]
Artifacts under configuration management
Core Assets
Core Asset
Core Asset
Software Architecture Core Asset
Composed
Teams for Core Assets
mapping
Team A TeamB TeamC Production
Custom Assets
Custom
Custom
Custom
Asset Product Instances
Asset
Asset
Teams for Custom Assets
7. Definition
[ A Configuration Management Model for Software Product Line, Liguo Yu and Srini Ramaswamy, 2006
]
Terms Definition
Component The basic unit for configuration management
A collection of components. An asset may contain one or
Asset
more components
Contains a set of domain specific but application independent
Core Asset components that can be adapted and reused in various
related products.
Custom asset Contains a set of application specific components
A collection of core assets and custom assets.
Product
Products share the same or similar core assets.
After a new product is produced, it may also need to be
Product
configuration managed. The product under configuration
instance
management is called product instance.
8. Definition
Variation management for Software Product-line
• Variation in time and space
• Divided into nine smaller issues and suggest the solution for each issue
[ Nine Sub-problems of Variation management, Charles W. Krueger
]
Variation
Type Sequential Time Parallel Time Domain Space
Granularity
Basic Configuration Management
Files Version Branch Variation Point
Management Management Management
Branched
Baseline Customization
Components Management
Baseline
Management
Management
Branched Customization
Composition
Products Management
Composition Composition
Management Management
Component Composition Software Mass Customization
10. Considerations
Considerations when you implement the usage model
• The code structure of repository to manage both a common assets
and variant assets
• Branch strategy for enforcing the development process
• Baseline strategy for tagging each asset and a whole product
• Integration with defect tracking tools
• Daily Build and Release process
11. Depot Structure
Depot Structure can be considered by the followings
• SW architecture
• The structure of the development team ( Single vs Multiple teams )
• The access control policy
Software Architecture Depot Structure
Workspace
Depot
Branch/Directory
Branch/Directory
Manage Component A
Directory Composed
the change
Depot by
on
Branch/Directory
Branch/Directory
Component B
Directory
• Each depot holds one asset
• One depot holds all assets
• One depot holds multiple assets by the characteristics of assets
12. Depot Structure
Each asset is coded and then released by a team. All code of an
asset are managed by a project.
• A management unit to control all activities related to an asset
• Using naming rules, you can distinguish a directory from a branch
( “PRJ_”, “[ ] “ )
• Grouping related projects with directory can be used
Depot holds several projects in which the code
for assets are managed
Directory can be used to group the related projects
- Year, Organization unit ( HQ, Oversea labs )
13. Depot Structure
Additional data is also needed to maintain a Project.
• Project data including Name, Description, Depot path
• Branches and the hierarchy of branches which belong to a project
• The administrators of each project. They are in charge of
assigning new developers and making a baseline
for releasing an asset
• The policy which controls integrating flow and locks the branches
• Sometimes, the protection table of each project can also be
managed
These can be stored in a depot or other storage such as a
database
14. Code-line (Branch) Strategy
Sophisticated Code-line strategy is required
• The size of team is getting bigger
• The quality requirement of assets is getting higher
The quality requirement of assets
• Core Assets is higher than Custom assets
[Project name] § The changes gathered from the lower branches
are built with the other assets
Stable Mainline § Test activities are performed
§ Release baseline is tagged
Integrate
§ The changes gathered from the lower branches
Maturity of Sub-Code-line are reviewed and integrated
the code Rebased § If the members of a team are small, it can be
Integrate optional
Developer’s Code-lines § The codes are submitted with linking jobs
Rebased
§ The changelists are integrated into the upper
Unstable branch
15. Branch Strategy
Sparse branching is a good solution to avoid integrating
the files which are not changed in the developer’s branch
2 3 Rebase the
changes into
Integrate Dev-One
Dev-One to
Mainline
- Refer to KB #1166
1
Only Module 1 is
branched
[ View spec using Sparse branching ] - Refer to KB #890
//Core_Assets/PRJ_Core_Asset/Mainline/… //wk/Core_Asset1/..."
+//Core_Assets/PRJ_Core_Asset/Dev_One/ComponentA/Module1/... //wk/Core_Asset1/Module1/...
16. Branch Strategy
Policies to govern the activities of a project are controlled
• Submitting : ( P4 Trigger )
- Linked with at least one job ( Activity based change management )
• Integrating : ( P4 Broker )
- The status of jobs linked with changelists is verified
- Rebase the integrated codes from the target branch before integrating
- The integrating flow is controlled ( refer to the below tables )
O : Allowed, X : Disallowed, : Configured depends on each project
1. Inner project
2. Between the projects
From/To DEV Sub-INT INT REL From/To DEV Sub-INT INT
DEV O X X DEV X X X
Sub-INT O X O X Sub-INT X X X
INT X O - O INT X
REL X X X -
17. Label Strategy
Two types of Baseline will be used
• Baseline : Attached to each asset when the code of the asset
is released
The use of naming rules to indicate the maturity level of
code is very useful
è _INT, _REL
• Composite Baseline : A set of Baselines to reproduce all files
which compose a product
- Keeps the labels of assets composing a product
- Easily synchronizes all files of a product with a client workspace
for developers who don’t know the combination of baselinesand
assets
- Can also include composite baselines recursively
18. Label Strategy
Include
Composite Include Composite
Baseline of Asset A1
Baseline of Product A Baseline of Asset Group A
Baseline of Asset A2
Baseline of Asset C1
Baseline of Asset C2
Baseline of Asset P1
Number Project Name Stream Name Depot Path Label Name
1 Core Asset C1 INT //Core Asset_C/Project_C1/INT C1_0420_Release
2 Core Asset C2 INT //Core Asset_C/Project_C2/INT C2_0419_Release
3 Custom Asset P1 Sub-INT //CustomAsset_P/Project_P1/Sub-INT P1_0420_Release
4 Core Asset A1 INT //Core Asset_A/Project_A1/INT A1_0419_Release
5 Core Asset A2 INT //Core Asset_A/Project_A2/INT A2_0419_Release
19. Integrating with the defect tracking tools
Change based modification should be enforced
• All work items such as Change Requests and Defect can be
synchronized through Jobs in Perforce
• Synchronizing can be considered as one-way or two-way
- One-way : Defect tracking tool can only change the job status
- Two-way : Both Perforce and the Defect tracking tool can change
the job status
• The policy to verify the job status and whether it can be promoted
- P4 broker can be a candidate to implement those policies
21. Access control
On each asset (project), It’s own access control mechanism
is also required
• Set access control on each code-line by groups
• Freeze some code-lines during integrating and building. But,
exceptional users can be allowed
Project • Default group and member is set,
Create Group and Default Group
(Asset)
when a project is created
Assign developers
Additional Groups
Additional Groups • Create a group, assign/release
Additional Groups
Project developers on groups
administrator
Manage access control No Access Level User/Group Name Host Path
Table of a project
1 write group Default group
//Depot/[Core_Prj]/...
2 read group Additional Group
//Depot/[Core_Prj]/Mainline/CompA/…
22. Build and Release strategy
More complex than conventional development
• The difficulty is the combination of the right versions of the right assets
- Every asset is changed continually
- Each team has different rules when managing stable code
• Core assets should be released frequently and these also should
be testified on all products
- Building and testing on all products requires time and effort
• Miscommunication among the development teams
- Major changes such as interface changes lack notification
- The time when applying the change of asset can be disordered
23. Build and Release strategy
The followings should be clearly defined :
• All products should be built with the same rules how to combine
the assets
• Daily builds on all products should be run, and the results of
daily builds also are shared with all developers by indicating
which assets were failed
• Periodically, the causes of the build error are analyzed and
improved
24. Build and Release strategy
Daily Builds and release process can be depicted as the below
Release Core
Daily Build Release Products
Assets
Get the released
Success
Core Assets
Daily Build Build Runs Tests
( All models ) Result
Developer’s
Builds Build & Test
Fail
Release the Core
Integration assets with
Builds Fix the error Baseline Release
Run builds
Build management Core Assets Product’s
System Builders Builders
Run the CI Run the daily Indentify and fix the
builds builds daily builds
25. Build and Release strategy
The following features can be supported by build automation tool
• Synchronize
- the latest label of each asset
- the latest composite baseline of a product
- the specific label/changelist of each asset
• Make a baseline
- each asset
- a composite baseline of a product
• Integrate codes into the upper branch and make a baseline
after the build was completed successfully
- Builds è Unit Test è Integrate Codes è Make a baseline
- Automatically generates the release notes of each asset
with P4 jobs which gathered from the defect tracking tools