Read the article on How Quality Engineering Helps You Clean Up Your Act which is written by Roger Nessier, VP, Client Management, Estuate, Inc.

1. Quality Engineering Helps You Clean Up Your Act
By Roger Nessier, VP, Client Management, Estuate, Inc. - January 14, 2013
Trends such as cloud based software delivery and emphasis on uptime create even greater demands
on software quality. Independent software vendors (ISVs) and software-enabled businesses must
forego their usual pre-release quality control approaches and instill quality principles into every aspect
of product development, from whiteboard to long term maintenance. Users no longer accept the
tradeoff of whiz-bang technology for buggy, hard to use software. A recent example of this was the
release disaster of Apple Maps, from a company which normally produces high quality innovative
software. Software providers, software enabled businesses, must focus on high quality, secure, and
reliable deliverables. The question is how will they address this?
Software development organizations must respond to this changing market dynamic by placing a
greater emphasis on quality engineering (QE). QE imbues quality principles into every aspect of the
software product development lifecycle (PDLC), as well as technology and architecture decisions.
The impact of taking a QE approach to software development can be huge. Catching process or design
flaws early on reduces the impact and cost of remediating those issues as you get closer to the
release. The most mature QE centric development organizations clearly understand the impact of
quality on their PDLC process and software design. They use this information to ensure that final
deliverables are of the highest quality.
For some organizations this can be a big cultural shift where architects and developers were the power
players. In some organizations, QA is a place where those who aspire to development roles attempt to
prove themselves, or those who can’t cut it as developers are relegated to lesser roles. In a QE centric
organization the Quality Engineer is an equal to the architect or developer. In communicating this
change, quality should be regarded as the mutual goal for both the developer and quality engineer.
Teamwork is the hallmark of a QE organization. The quality engineer should work to help development
engineers preemptively identify issues and address them early in the development cycle. The quality
engineer needs to be a highly capable engineer that combines the problem solving skills of a
developer with the eye for quality of a QA engineer.
The first step in adopting a QE centric approach to development is to understand your Quality
Engineering Maturity Level (QEML). QEML is a framework we have developed to help you better
understand where your development organization is in QE adoption and what level of QE adoption
your organization aspires to achieve. Once you understand your QEML, you set incremental QE goals
over times that are supported by business benefits.
What is your Quality Engineering Maturity Level?
Feedback mechanism from stakeholders to refine design/process on an ongoing basis. Strong reliance
on metrics, surveys and interviews to identify quality trends. Use of trend data to take action to
optimize technology, design and process decisions to meet the changing needs of stakeholders.
Leverage predictive mechanisms to better understand weak

2. links.
Recognition that quality extends beyond catching defects early to include QE as part of design and
development processes. Every aspect of development is analyzed and scrutinized to ensure it
optimally aligned towards achieving quality goals. Technology, architecture and design decisions are
made with an eye on quality. Stakeholders are tightly integrated into design and process decisions to
ensure their needs are met.
QE oversight to processes ranging from requirements gathering to long term support. Empower QE to
“stop the line” should defect trends show a need to reassess design, process or technology decisions.
Extensive use of metrics to understand quality trends. Culture of quality is instituted.
QA and development activities are quite coordinated through the PDLC. No large backlog of bugs. Test
cases derived from use cases and user stories early in development cycle. Developers responsible for
unit testing. Limited use of metrics to better understand “problem areas”.
“Throw it over the wall” pre-release QA, little emphasis on unit testing, test cases developed late in
development cycle, releases regularly go out with several S1/S2 defects. Heavy reliance on point

3. releases to improve product quality.
Development organizations should assess their QEML from two perspectives: Process and technology.
It is possible to rank differently on the QEML scale from a technology and process standpoint. For
example, you may have excellent development processes with strong stakeholder involvement, but
are limited to how well you are able to serve your stakeholders because your applications are built on
an inflexible architecture. Conversely, your application may be built on a modern web framework, but,
your development processes are immature.
Once an organization understands their technology and process QEML, then it sets goals over time for
increasing their QEML. QEML goals should be supported by business cases that have strong linkages to
financial goals. For example, if no “severity 3” defects in releases doesn’t have a business justification
because the application isn’t used that often and users are tolerant of severity 3 defects, then that
should not be a process QE goal.
Organizations need to develop separate technology and process QE plans for each product if those
products are developed using different technology and development processes. It is usually
recommended to make process QE improvements before making technology improvements, since your
development organization will ultimately benefit from those process improvements when making
technology changes. It not recommended to try to conduct both technology and process changes
concurrently, or to try to move too rapidly since the shock of these changes could impair your
organization’s ability to “keep the lights on”.
In the example below, a development organization has decided to revamp the development process
for Product A first (Step 1). Once the process changes are completed, the value of the changes will be
assessed, tied to business benefit, before deciding to proceed with “Step 2”, improving the technology
platform.

4. Development organizations stand to derive the greatest benefit from integrated, holistic process to
QE. If a “big bang approach” to Process QE is too daunting, or stakeholders need convincing that there
is a business benefit to Process QE, incremental or point adoption can also be leveraged. The following
are process QE adoptions you may wish to consider.
Process QE
 Test Regime: Use Cases/Test cases should be developed at inception and applied early in the
development cycle. Make sure that use cases/test cases are comprehensively documented.
With the aid of use cases, comprehensive unit tests should be created and leveraged to ensure
developers understand the business issue being addressed and to catch bugs or
misunderstandings early in the development cycle. When possible, peer reviews should be
leveraged to identify issues. Automated testing reduces testing overhead and may help
uncover more issues. A heavy reliance on testing early and frequently eliminates issues before
they become serious and expensive to fix.
 Requirements Traceability: Trace completed requirements back to business needs to
ensure the need is met. Iterative and complex developments sometimes lead to the original
product mission getting changed or lost. If requirements evolve, then it is important to update
business requirements to reflect this evolution. The completed software product and its
subsequent releases must always be in sync with the business requirements document.
 Use Defect Analysis to Identify Root Causes: Finding clusters of defects may point to a
particular root cause such as a design flaw. Developers should investigate issue clusters and
not just focus on fixing observable defects. Defect analysis should be conducted to track the
number and source of defects throughout the development cycle to identify trends and take
early action if an endemic root cause is uncovered. Advanced techniques such as Taguchi

5. methodology identify applications areas that are prone to defects and helps focus testing
activities to those areas. This results in fewer test cases to achieve full application coverage.
 Adopt Agile Processes: Adopting an Agile development approach can help you “test drive” a
QE approach. Agile by it’s nature uses smaller teams and requires close collaboration across
all disciplines. Each Agile team almost by definition results in a QE approach although it’s
restricted to that team and what’s being accomplished in a given iteration. Applying any Agile
approach (be it SCRUM, XP, FDD) to development helps build upon early successes and
validates design assumptions. Instead of trying to release multi-faceted complex functionality,
start simply and leverage market and stakeholder feedback to continually refine and enhance
functionality. By building it simply the first time, you eliminate unneeded complex functionality
that can detract from the user experience and introduce more issues. Only add in extra
capabilities after you feel you have a strong justification to do so.
 Increase Focus on PSR Testing: Comprehensive performance, scalability, reliability (PSR)
testing uncovers “weak links” in the product. PSR should cover all possible uses of the
product, using different configurations, hardware/software environments, and data sets. PSR
testing should help drive coding standards to ensure that as new functionality is added, PSR
does not degrade.
 Document Processes to Enhance Repeatability and Consistency: Great software
products are not created unless development processes are well documented and followed.
Whether you follow a classic waterfall or an agile methodology (or something in between), it is
important to have total process clarity from the requirements stage to end of life for a
software product. Good process understanding is especially important if you have distributed
development teams. Always look for opportunities to improve the process, and be sure to
update process documents with those improvements. Leverage metrics to understand how
well the team is tracking to expectations around productivity, quality and on time delivery.
Report on those metrics using a real time dashboard that provides total transparency on the
state of a release.
Deciding on QEML process goals depends on a variety of factors: Are you taking an integrated on
incremental approach? What are the cultural, educational, and business challenges you are facing that
could impede adoption? What are the perceived benefits (business case) for QE process adoption?
What is your expected ROI? How will you measure success?
Technology QE
Development organizations are often faced with supporting and/or extending several generations of
applications from Cobol to Java or .NET. Disparate technologies, design and skill requirements all

6. present challenges to a QE centric development organization. One of the biggest questions many
organizations face today is: How can I afford to rewrite legacy applications to leverage the latest
thinking in architecture/technology, while continuing to meet the needs of the marketplace? Any
technology QE strategy must be balanced against market and business realities. Like process QE,
many organizations take an incremental approach to technology QE, this can sometimes feel like
changing the engine in your car while driving down the freeway. Like process QE, technology QE must
be substantiated by business benefits. For example, a development organization doesn’t adopt SOA
because it seems like the technology savvy thing to do, or because developers like to have SOA
expertise on their resume. SOA adoption should be justified by a strong business case. In some cases,
SOA may not make sense for your organization. The following are technology QE adoptions you may
wish to consider.
 Reduce Design Complexity: Eliminating complexity and encouraging reusability can help
simplify large system design, thus reducing quality challenges. Designs incorporating SOA ,
object orientation, and partitioning, are more extensible and able to adapt as the product
evolves. Avoiding a hard design commitment is preferable if an abstraction serves the same
need. Re-factoring code early in the development process will reduce complexity and help
uncover flaws. Designing for ease of integration to other products/systems by encapsulating
variation reduces 3rd party software integration/testing challenges. Keep external interfaces
as stable as possible when changing internal application functionality.
 Improve Usability/Interactivity: Poor usability/interactivity should be considered a design
defect. A rushed, poorly researched approach to UI design will result in users logging issues.
These issues cannot be ignored just because the product was “built to spec” and is supposed
to have a user interface that is unintuitive. Be cognizant of how the product is used,
leveraging use cases and story boards to validate the final design. Take into consideration the
different needs of casual users and power users when analyzing usability/ interactivity. Make
sure UI designs are intuitive, self-explanatory, and self-documenting.
 Design for Maintainability: If fixing parts of the application causes an undue amount of
collateral damage to other parts of the application, there may be issues with the design.
Engineers need to look at memory usage, designs and data access to understand if there are
fundamental flaws causing the issues. As well, if porting the application to different
platforms/environments, causes an undue amount of functionality to break, there may be
issues with the design portability that needs to be addressed. Poor maintainability is
sometimes caused by developers who feel they can walk away from a software project after it
is released. Developers should realize they their team (or a subset of the team) are
responsible for bug fixes and improvements in future releases.
 Design for Extensibility: If adding functionality causes an undue amount of existing
functionality to break, there may be issues with the design. Revisit the product architecture as
a product evolves to ensue it meets the needs of product. The adding of new functionality can
be constrained by an architecture design that does not anticipate the product evolution. Poor
extensibility is sometimes caused by a “project view” towards software development, vs. a
long term multi-release view.
 Take Security Seriously: Proactively identify security holes and look for vulnerabilities
before users stumble on them. Conducting a thorough security audit as part of the release
process not only uncovers security flaws, but also helps refine coding standards so designs are
more secure. Make sure your security goals are aligned with the type of application you are
developing/supporting, and how exposed the application is to hackers.

7. Deciding on QEML technology goals depends on a variety of factors: Are you rewriting all your
applications or taking an incremental approach? What are the training and business challenges to
making technology changes? What are the perceived benefits (business case)? Do your customers
really care if your application is written in Java or C++? What is your expected ROI? How will you
measure success?
The QEML planning process should be sequential and iterative. Checkpoints and metrics are used to
ensure value is being generated. The chart below describes the steps your organization should use to
set QEML technology and process goals.
Finally, instituting a “culture of quality” is critical across the entire company, not just the R&D
organization. Develop training programs and make sure stakeholders in the rest of the organization
understand the QE mandate of the development organization. For example, sales or management
should not over commit development, resulting in “death march” releases and the resultant quality
degradation. Professional Services teams should create customizations or configuration changes
leveraging the same QE principles that development uses. Creating a culture of quality that permeates

8. throughout the entire organization will return dividends in the form of higher customer satisfaction,
greater ability to meet market needs, greater customer retention, and ultimately, a healthier
company.
Author
Roger Nessier
Roger provides product development services to software companies and companies that use software
in their business. He has delivered dozens of projects to a range of clients around the world, using
global teams, from gathering initial requirements, to design/build, to long term support. He is
currently VP of Client Management at Estuate, and has held services management and product
development executive positions at i2 Technologies, Symphony Services, and Steelwedge Software.
Roger lives in the Bay area. In his spare time, Roger enjoys running or cycling the hills near his home,
or taking a yoga class with his wife.