Configuration management for database overview1. Configuration Management for
Databases Overview
Robert Berliński, support Mercedes Gavilan
Document version 0.8, February 2, 2012
Table of contents:
Preface ...........................................................................................................................................1
Introduction ...................................................................................................................................2
Challenge specific to a database ....................................................................................................2
Other, common challenges ............................................................................................................2
The strategy ...................................................................................................................................3
The solution ...................................................................................................................................4
Adopting the solution ....................................................................................................................6
Summary .......................................................................................................................................6
Preface
The document describes author's strategy and their successful implementation of a
configuration management for a database project. First it describes the nature of a configuration
management dedicated to a database and the corresponding challenges. Then it explains the
strategy and selected insights to the solutions, procedures and tools. The document is meant for
professionals who have similar experiences and it doesn’t assume any specialized technical
backgrounds. It might help project managers or config managers to compare and choose the
best solutions for their organizations.
Copyright © www.scmsupport.com 2012. All Rights reserved. Page 1/6
2. Introduction
An IT configuration management process is a complex system of many procedures involving
the work of many professionals caring on different responsibilities as well as variety of
resources that provide platforms for documenting, developing, testing and deploying the
changes.
Let's consider a process of building a Java application as a reference. It takes the latest version
of many files to build an EAR file. Actually it includes all constituting files whatever the files
have changed or remains untouched between version A and version B. So at the end an
application is a mix of recently modified and unmodified files.
In contrast database's changes are delivered as upgrades, that include only the differences
necessary to take a database from version A to version B. Unfortunately organizing
components that define a difference is not that simple due to the technical characteristic of a
database. Configuration management procedures need to consider not only code changes as it is
with Java but all relations between model changes and data. All relevant changes need to be
managed together and they must be delivered in the correct order. It means that a configuration
management process should preserve the order of changes and deliver them to a database in the
same order.
On the top of that, the world changes dynamically and it is not unusual for a Business to
request many changes in short and overlapping timeframes. Again the database nature, the
relations between code/model/data require special attention. It happens that the expectations of
business are limited by the technology or by the available resources.
Challenge specific to a database
There are many reasons for changing a database: changing the functionality in order to satisfy
new legal requirements, developing new business functions to compete with other businesses or
fixing problems – just to name a few.
In many scenarios a request for changes transforms not only source code objects (PL/SQL
procedure, functions, packages etc.) but also the corresponding data model and/or data. E.g.
adding a new column for an existing table generates model changes, code changes and a
procedure that will fill-in the new attribute of existing records with a default value. In this case,
it is not only important to remember about performing all the changes, but to execute them in
the correct order. E.g. model changes should precede other relevant changes to codes/data. The
process of configuration management for a database project should follow all the technical
dependencies. Breaking any of them might cause an issue with a database. And resolving the
issue might be as time-consuming as working on the change on the first place.
Therefore it is more critical for the configuration management of a database project to have
solid procedures and discipline. All involved professionals should understand and follow the
procedures as it is more important for the overall success. The human-factor can make the work
of project managers and config managers more challenging or it can transform to build a space
for professional synergy that will improve the configuration management process.
Other, common challenges
Besides the challenges specific for a database project there are typical code relevant issues as in
any other IT project, e.g. with a Java application project. For instance it is necessary to:
Copyright © www.scmsupport.com 2012. All Rights reserved. Page 2/6
3. keep track of code changes; document who, when make what change and what is the
business reason for the change; sometimes there is additional documentation to keep track
of the tests for each change,
deliver the changes in organized and controllable way for deploying; an owner of changes
(usually the Business) should have a mechanism to control when and what change delivers
to Test/Pilot/Production,
on the other hand the changes need to be applied in a way, that will not absorb to much of
the administrator’s time; usually it means more work on the config manager site to satisfy
the expectations,
organize the development in a way, that will be flexible to allow team of developers to
work simultaneously on one complex change/version as well as allow developers to work
on concurrent versions.
The strategy
The chosen strategy is based on a simple assumption: since database projects and other projects
shares the same common nature it should be a valuable approach to take a good, well-known
solution that works e.g. for Java projects and then extend it in order to address the more
complex nature of database projects. Author's previous experiences with Java applications
served as the base line for the solution including a version repository system.
The illustration 1 shows two teams (A and B) working on two concurrent projects/versions.
There are dedicated environments for development (dev1 and dev2). Developers commit the
code changes to the central version repository, where every project has its own dedicated
branch for storing changes. Then config manager takes the changes from the version repository
and delivers updates to Test/Pilot/Production.
Illustration 1 - the generic strategy.
The key elements of the strategy:
Trunk and branches is the way of organizing the code within a version repository. There is
also a structure of subfolders within each branch of the repository that corresponds with the
database structure and map all its components.
Changes made to a database are committed to the version repository with comments that
reference the business reasons for changes, e.g. the request for change number, the fix
number, etc.
Copyright © www.scmsupport.com 2012. All Rights reserved. Page 3/6
4. Config manager pulls-out the committed changes and builds a delivery patch that includes
the changes. Each patch is stored in a dedicated sub-folder within the same branch in the
version repository. Patches are being built based on a number of changes and requests from
business. Config manager notifies business about every new patch. Patches can be prepared
on daily basis and might include one or more changes, that makes them perfect to serve the
frequently changing regions like test and fixing critical problems. They also can serve for
Continuous Integration.
Config manager collects all patches preserving their order into an upgrade just before
Pilot/Production delivery. Each upgrade follows the same procedure as a patch and
includes ordered patches that have been previously released and tested. An upgrade is
more useful for one-time deployment of all changes, e.g. for Production.
Business keeps a list of all the available patches/updates and business can request from
administrator to deploy a patch/upgrade to a specific region but should preserve the order of
patches.
It is important to notice, that the process of researching and then extending the solution is rather
an evolution than a revolution. As working on it, the well-known solution (that supports the
common nature of all IT projects) remains in place. And the new extensions are being gradually
introduced to address more and more database specific issues. It guarantees a stable
transformation of an existing configuration management process.
The solution
The Subversion (SVN) repository was chosen for its availability and quality. Even if the out of
box SVN doesn’t provide a dedicated support for the database issues, additional procedures and
tools help to overcome the gap.
First, it is important to notice the structure of SVN branches as shown on the illustration below.
Illustration 2 - the specific directory structure in version repository.
There is a separate branch for each delivery named after the version. Each branch includes the
primary 3 subfolders:
Patches folder that holds all prepared patches and upgrades.
Scripts folder that includes documentations, scripts and items other than sources.
Copyright © www.scmsupport.com 2012. All Rights reserved. Page 4/6
5. Sources folder that includes database objects: functions, packages, procedures, triggers,
types and views.
The context of the folders is managed according to specific procedures depending on the role:
for developers, e.g. commit changes with comments, merge some changes between
branches,
for config manager, e.g. start a new branch, merge changes to trunk, build patches and
updates, close a branch.
Depending on the calendar of tests, the number of changes and their importance, Business
schedules delivery of all the currently available changes to a test region. Before each delivery
developers must commit their stable changes and then config manager pulls-out and collects all
the changes when building a patch.
Actually config manager uses an automated tool. The tool checks for new changes
committed in a branch of a version repository and then collects the changes in a special patch
that again is saved in the same branch of the repository. Each patch is an executable script
that applies the changes to a database. Automation is the key aspect in the process of
building a patch. As long as all developers follow the procedures for committing and
commenting changes, the tool takes care of important aspects, e.g.:
Recognizes new entries committed since the previous patch or it can pick-up changes
specific to a given revision number.
Delivers script changes preserving order dependencies.
Delivers the latest version of the source code changes without duplicates. It helps with
e.g. packages that changed many times.
Delivers complex changes that touch model/data/code following the order required by
database for successful deployment.
Takes care of types dependencies.
Takes care of deleted objects.
Illustration 3 - delivering patches and upgrades.
Since the tool covers over 95% of the typical scenarios for changes, it is a tremendous progress
versus manual work in terms of time and eliminating places for human errors. Only some rare
cases might require a manual intervention by config manager. And since every patch is an
executable script it saves the time for administrator who applies changes to a database. The
Copyright © www.scmsupport.com 2012. All Rights reserved. Page 5/6
6. Administrator's work is to execute a patch and verify the output for potential errors. Usually it
takes a few minutes and it doesn’t require any advance knowledge of the database - the work
can be performed by an average administrator.
Adopting the solution
Learning from experience and observations made possible to introduce more extensions with
each delivery. The research process allowed to improve the procedures and the culture of work
in a period of a few years. On the other hand the accumulated experience and solution can be
applied to another organization in a period of one to a few delivery processes. And the primary
reason for the time is the transformation of professional habits and collecting experience.
Summary
The authors of the solution work together on database changes and researches since 2008. The
result helps to overcome most of the identified issues. The strategy generated positive solution
that addresses the technical bonds of a database and meets the business expectations.
The solution delivers important benefits:
Provides control over the code changes and makes possible to answer the questions
who, when and for what reason changed what part of a code.
Allows teamwork and concurrent projects, giving project managers more flexibility in
scheduling changes considering limited resources.
Automates most of the processes of delivering changes, it saves config manager's and
administrator's time as well as eliminates many areas for human mistakes.
Can serve as the base for automated tests and for continues integration.
Now the authors work on improving the process and on continuous integration.
If you would like to learn more, please check our websites www.scmsupport.com. The website
describes in more details the procedures and the technicalities. Thank you for your attention.
Copyright © www.scmsupport.com 2012. All Rights reserved. Page 6/6