See how IT Risks Impacts your Business. CAST help you to check on software performance, stability, maintainability, and security vulnerabilities in which CAST excels and successfully differentiates from code analyzers.CAST’s Application Intelligence Platform and Rapid Portfolio Analysis solutions can help you avoid these types of “software glitches” or "software risks" by allowing you to gain greater visibility through automated code review that identifies the root causes of risks before they become production problems, while expediting time-to-market with shorter release time lines and improved business agility.

1. Managing Software Risk with CAST
Building Resilient Software to Support Business

2. CAST Confidential 1
Webinar goal and content
Goal: Understand how CAST can help avoid software glitches
Content
 Review of state of software risk in business technology industry
 Analysis of reasons that software fails
 Explanation of CAST technology for software analysis
 Examples of potentially-lethal software CAST has uncovered
 How to implement CAST as a quality gate to lower software risk

3. CAST Confidential
IT risk has become a serious concern
2
How IT Risk Impacts Business
Percent of respondents identifying each business element
Source: 2012 IBM Global Reputational Risk and IT Study
n = 427
What Drives Reputation Risk

4. CAST Confidential
System outages have never been easy to control
3
Sources: The Register – 2008 Risk & Resilience Study, IDC Software Quality Study 2011
n = 200
Number of defects requiring patches
in 12 months after production rollout
21% of project managers
report over 50 defects in the
first 12 months after rollout

5. CAST Confidential
Incidence of software “glitches” is clearly on the rise
4
 Software is the primary culprit in
system outages
 Software glitches in live business
systems happen frequently
 Most of the time we don’t find
out, but recently there’s more in
the news
Trading platforms & exchanges Airlines
Sources: Wall Street Journal, Bloomberg, The Register – 2008 Risk & Resilience Study

6. CAST Confidential
Incidence of software “glitches” is clearly on the rise
5
 Responsible for 10%
of North America
trading by volume
 $440 million loss in
45 minutes

7. CAST Confidential
Air traffic control system Ticketing self-service website
6
Past forensics related to similar outages
 Variable not sized properly,
limited to 50 days of operation
 IT procedure to reboot system
every 30 days reset timer almost
3 weeks before it ran out
 Until that procedure was
changed
 A user accidentally types a URL
into the wrong field
 Thousands of personal, records
leaked all over the internet
 Website service suspended for
months until new version
released

8. CAST Confidential 7
Are we just getting used to software failure?

9. CAST Confidential 8
Why does this happen?
 System complexity keeps increasing
 Too many applications to track
 Hitting limits of doing more with less
 Turnover and short-term-ism
 Sourcing complexity & offshore
 Speed of software production
 Inadequate approach to QA
No institutionalized product oversight at the structural level

10. CAST Confidential 9
Analyst perspectives on the problem, and solution
“There is a balance between ‘just get it done’ and ‘do it
the right way.’A few additional quality measures help
you find that balance.”
“Addressing technical debt is really a risk decision for IT
executives. I can invest in fixing some of the technical
quality problems now, or risk that they result in outages,
breaches or other problems that can cost far more.”
The architectural assessment of design consequences (on
software performance, stability, adaptability, maintainability,
and security vulnerabilities) is an area in which CAST
excels and successfully differentiates from static analyzers.”

11. CAST Confidential
Defects in poor systems turn into software failures
 Software delivered
contains 5 potential
defects per FP
 Many defects are
dormant in the code
 Technical debt
continues to mount
Source: Capers Jones. Data collected from 1984 through 2011;About 675
companies (150 clients in Fortune 500 set); About 35 government/military
groups; About 13,500 total projects; New data = about 50-75 projects per
month; Data collected from 24 countries; Observations during more than 15
lawsuits.
1. Design defects 17.00%
2. Code defects 15.00%
3. Structural defects 13.00%
4. Data defects 11.00%
5. Requirements creep defects 10.00%
6. Requirements defects 9.00%
7. Web site defects 8.00%
8. Security defects 7.00%
9. Bad fix defects 4.00%
10. Test case defects 2.00%
11. Document defects 2.00%
12. Architecture Defects 2.00%
TOTAL DEFECTS 100.00%
Severity 1 = total stoppage; Severity 2 = major
disruption
Defect Origin
% Severity 1 or 2
Defects
10

12. CAST Confidential 11
Industry starting to pay attention to code quality
But code quality & hygiene is only a small part of the solution
Component-level
Violations
Architecturally
Complex Violations
Dev
Test
83%
10%
Operations
2%
13%
% of violations crossing a phase boundary
8X worse
6X worse
60,700
83,000
168,000
2009
2010
2011
Searches for
code quality
Violations that
cause defects
Sources: Li, et al. (2011). Characteristics of multiple component defects and
architectural hotspots: A large system case study. Empirical Software Engineering

13. CAST Confidential 12
Measurement based on standards
Consortium for IT Software Quality
Characteristic Architectural & System Level Flaws Coding & Component Level Flaws
RELIABILITY
Multi-layer design compliance
Software manages data integrity and consistency
Exception handling through transactions
Class architecture compliance
Protecting state in multi-threaded environments
Safe use of inheritance and polymorphism
Patterns that lead to unexpected behaviors
Resource bounds management, Complex code
Managing allocated resources, Timeouts, Built-in remote addresses
PERFORMANCE
EFFICIENCY
Appropriate interactions with expensive and/or remote
resources
Data access performance and data management
Memory, network and disk space management
Centralized handling of client requests
Use of middle tier components versus stored
procedures and database functions
Compliance with Object-Oriented best practices
Compliance with SQL best practices
Expensive computations in loops
Static connections versus connection pools
Compliance with garbage collection best practices
SECURITY
Input validation
SQL injection
Cross-site scripting
Failure to use vetted libraries or frameworks
Secure architecture design compliance
Error and exception handling Use of hard-coded credentials
Buffer overflows Broken or risky cryptographic
algorithms
Missing initialization Improper validation of array index
Improper locking References to released resources
Uncontrolled format string
MAINTAIN-
ABILITY
Strict hierarchy of calling between architectural layers
Excessive horizontal layers
Tightly coupled modules Unstructured and Duplicated code
Cyclomatic complexity Controlled level of dynamic coding
Encapsulated data access Over-parameterization of methods
Hard coding of literals Commented out instructions
Excessive component size Compliance with OO best practices
www.it-cisq.org

14. CAST Confidential 13
Technical debt is related to software risk
 Most technical debt measures do not categorize the debt
 There’s a lot of debt out there, many questions about “when to
pay it off?” and “which to debt focus on?”
 It turns out only about 30% of technical debt has any immediate
risk component
Source: CRASH Report for 2011-2012, CAST Research Labs
Distribution of Technical Debt
n = 756 applications
(365 million lines of code)

15. CAST Confidential 14
CAST approach to software risk management (1/2)
IDENTIFY
 Risk reduction starts with identification of risks to understand the scale and
scope of risks across an organization
 Identification using automated tools for consistency and objectivity
 Output of “Identify” stage should include portfolio view & high profile risks
STABILIZE
 Prioritized list provides an action plan
 Focus on immediate, short-term risks to critical business systems
– Security risks
– Production defects
 Reassess to validate that short term risks have been addressed
IDENTIFY STABILIZE HARDEN OPTIMIZE
Risk Perspective Immediate-Risk Long-Term Risk
Assessment Level Portfolio Critical Systems Application Application

16. CAST Confidential 15
CAST approach to software risk management (2/2)
HARDEN
 Move beyond short term, immediate risks to address the “long tail”
 Focus on performance, robustness, security
 Improving brittle systems to become responsive, adaptable
OPTIMIZE
 Shift to long-term thinking
 Shift from process thinking to product thinking
 Focus on improving maintainability and transferability of systems
 Address organizational or process issues for long-term improvements
 Technical debt management and reporting strategy
IDENTIFY STABILIZE HARDEN OPTIMIZE
Risk Perspective Immediate-Risk Long-Term Risk
Assessment Level Portfolio Critical Systems Application Application

17. CAST Confidential
Analysis strategy for typical IT application portfolio
16
Effort(ManDays/Year)
Importance to
Business
Highest Lowest
Critical Apps
Entire Application Portfolio
CAST AIP
 Deep Structural
Analysis
 Risk Detection
 Lean Application
Development
 Function Points &
Productivity
 Vendor Management
 Continuous
Improvement
CAST Highlight
 Fast Cloud-based
Delivery
 No source code
aggregation
 Key Metrics on Entire
Portfolio
 Size, Complexity and
Risk analytics
 Annual/Quarterly
Benchmark

18. CAST Confidential
Portfolio risk review with Highlight
17
Risk vs. Application Criticality
This chart examines business criticality against the risk level of the applications. 40 applications
are situated in the high risk zone. These 40 applications require detailed assessment and
planning for ongoing improvement.

19. CAST Confidential
ArchitectureCompliance
Enterprise IT applications require depth of analysis
18
 Intra-technology architecture
 Intra-layer dependencies
 Module complexity & cohesion
 Design & structure
 Inter-program invocation
 Security Vulnerabilities
Module Level
 Integration quality
 Architectural compliance
 Risk propagation
simulation
 Application security
 Resiliency checks
 Transaction integrity
 Function point & EFP
measurement
 Effort estimation
 Data access control
 SDK versioning
 Calibration across
technologies
System Level
Data FlowTransaction Risk
 Code style & layout
 Expression complexity
 Code documentation
 Class or program design
 Basic coding standards
Program Level
Propagation Risk
Java
EJB
PL/SQL
Oracle
SQL
Server
DB2
T/SQL
Hibernate
Spring
Struts
.NET
C#
VB
COBOL
C++
COBOL
Sybase IMS
Messaging
Java
Web
Services
1
2
3
JSP ASP.NETAPIs

20. CAST Confidential
CAST going well beyond static analysis
Static Analysis
Behavioral
Simulation
Dependencies
Code Pattern
Scanning
Data Flow
Architecture
Checker
Rule Engine
Transaction
Finder
Function
Points
Aggregation &
Consolidation
Understanding of language syntax and grammar using source code parsing
Analysis of some run-time behaviors to understand dynamic behaviors of applications
Understanding of cross-layer and cross-technology links between application components
Finding patterns and anti-patterns in application control flow
Tracking the use of the content of variables such as user inputs along static and dynamic call stacks
Identification of invalid calls and references between application architectural layers
Analysis of knowledge base against quality rules, metrics and constraints to identify violations (non-
compliant objects or situations)
Identification and configuration of cross-layer and cross-technology transactions from UI down to
data entities
Estimation of Function Points functional sizing, relying on data entities and Application-wide
transactions
Aggregation and calibration of results along the quality model and consolidation across applications
Intelligent
Configuration
Capability to build object sets based on object properties, links, etc. to support layers, modules, and
scope definition
Content
Updater Adjustment of analysis results to better match application advanced behaviors
19

21. CAST Confidential
Simulating runtime behavior to resolve links in code
20
Behavioral
Simulation
Emulating some run-time behaviors to understand dynamic behaviors of applications
Consider “Select Title from Authors where Author = ” as a SQL statement
Use (select) link between Java method “f()” and SQL table “Author”
quasi-runtime behavior

22. CAST Confidential
Multi-tier analysis for dependencies (1/2)
Capability to handle cross-layer and cross-technology links between Application components
Create links between Java Class and Sql Table
Hibernate mapping.dtd
Table oracle address
Address.java
Dependencies
21

23. CAST Confidential
Multi-tier analysis for dependencies (2/2)
22
Create links between JSP page and Action mapping
Create links between Action mapping and Java class
Struts-config.xml
Payment.jsp
ActionPaymentMethod.java
Capability to handle cross-layer and cross-technology links between Application components
Dependencies

24. CAST Confidential 23
AIP counts of framework diagnostics
 Frameworks are the link between components in a well-
architected system
 There are also rules to using such constructs effectively
Framework Rule Counts
Struts 1.x 21
Struts 2.x 9
Spring 3
Hibernate/JPA 23
EJB 8
JSF 1
Servlet 2
Tiles 1

25. CAST Confidential
Data flow – cross distributed architecture
24
Capability to track along static and dynamic call stacks the use of the content of
variables such as user inputs
(1)
(2)
(3)
(4)
SQL injection vulnerability – CWE-89
Data Flow

26. CAST Confidential
Configuring rules specific to enterprise architecture
25
Capability to identify invalid calls and references between Application architectural layers
Architecture
Checker

27. CAST Confidential
Security breach due to architecture misuse
 For example: banking application, for monitoring reasons, all
database calls must go through specific stored procedures
 Investigations showed:
– Many transactions developed offshore did not comply with secure
architecture framework
– Without automation, this could not be monitored
• 100 UI elements (250 kloc)
• 2000 mid-tier programs (1 mloc)
• 250 tables, 350 kloc of PL/SQL
 Use of Architecture Checker
– to define the desired architecture
– To generate and enforce the
appropriated quality rules
26

28. CAST Confidential
“UPDATE” trigger causing big problems at a global services provider
 In reservation system Java application must access legacy main-
frame to finalize transaction. In production, a performance issue
occurred when a volume of transactions occurred at one time.
 Investigation showed:
– Abnormal activity on the database due to an "on update" trigger that was fired too
frequently.
– The Hibernate ‘show SQL property’ revealed that the trigger was firing even if the data
had not changed. Error was due to a specific parameter in Hibernate: select-before-
update on the entity that was set to false. When set to false, Hibernate updated the
table systematically.
MY_ENTITY
A
B
C
D
MyUpdateTrigger
Always
fired
27

29. CAST Confidential
Real, measurable performance improvement numbers after fixing open/close inside loops.
We get around 90% performance improvement.
28
90% performance improvement in large mainframe batch process

30. CAST Confidential 29
Application shows a potentially dangerous lack of data control
Reduce risk – better use of safe components

31. CAST Confidential 30
Violation with the largest impact on the rest of the application,
regarding Robustness, Performance, or Security
LogicLayerDataLayerGUILayer
Propagated Risk Index (PRI) explained

32. CAST Confidential 31
 Allows to rapidly identify the most significant critical violations related
to a Health Factor
 PRI is based on
– Violation Index (VI) which assesses the quality issues a defective object
for a specific Health Factor
– Risk Propagation Factor (RPF) which assesses the number of call paths
of a defective object
Violation ViewContext (software /
Health Factor)
Propagated Risk Index – Prioritize findings

33. CAST Confidential 32
Transaction Risk Index (TRI)
 Identify the riskiest transactions for pen testing, remediation
 Sum of Violation Indices (VIs) of the objects along a specific
transaction: Robustness, Performance or Security.
Transaction View
Transaction Details View

34. CAST Confidential
Transaction Weight Risk Index explained
33
GUILayerLogicLayerDataLayer
Transaction with largest number of Robustness, Performance or Security violations

35. CAST Confidential
Stabilizing a multi-tier IT application
Missing error handling block across all layers
User Interface - Flex
Business Logic – C# .NET
Data Access – SQL Server (T-SQL)
34

36. CAST Confidential
Securing a multi-tier IT applications
Multiple violations across the same transaction
make warfighter / broad end-user facing applications more vulnerable
 Input validation - 4 form fields without validator in
user interface
 Architecture design - action class talking to data
access object bypassing business layer
 Database access security - multiple artifacts
accessing and modifying data on the LOAN table
potentially containing confidential data
1
1
2
2
3
3
35

37. CAST Confidential 36
Making risk management actionable
 Identify and stabilize are the tactical steps
 To harden and optimize is a move towards proactive risk
management
 Requires inserting some actionable processes into the
application lifecycle
IDENTIFY STABILIZE HARDEN OPTIMIZE
Risk Perspective Immediate-Risk Long-Term Risk
Assessment Level Portfolio Critical Systems Application Application

38. CAST Confidential
Measuring risk is important, but not enough
 At some point, inserting proactive prevention into application lifecycle
37

39. CAST Confidential 38
Cost vs. risk tradeoffs
 If you have Technical Debt – so what?
Technical Debt
SoftwareRisk
L H
H
L

40. CAST Confidential
IT risk management is an area of investment
39
IT executives expect to spend
more on IT risk
IT, and IT risk, is a C-level
concern
Who has responsibility for
reputational risk due to IT?
If you’re working on code quality,
your efforts should be tied to managing
software risk

41. CAST Confidential
Market leader in Software Analysis & Measurement
40
Ambitious
Mission
Rock Solid
Foundation
Market Leader
Introduce fact-based transparency into application development and
sourcing to transform it into a management discipline
 Broad market presence in Europe, North America and India
 Strongly endorsed by software industry gurus and long term investors
 Over $100 million of investment in R&D, driven by top talent in
computer science and software engineering
 Pioneer and recognized market leader since 1999
 CAST Research Labs, the world’s largest R&D facility dedicated to the
science of software analysis & measurement (SAM)
“CAST metrics have become the de facto standard for measuring the quality
and productivity of application services.” – Helen Huntley, Research VP, Gartner

42. CAST Confidential
Driving software measurement in the ADM industry
41
Key Influencers Recognize CAST
250 Global Leaders Rely on CAST
Institutions Engage CASTSIs Resell CASTSIs Use/Resell CAST
Top technology
First in business IT
Biggest benchmark DB

43. CAST Confidential
CAST dashboards, reports & benchmarks
42
CAST Highlight
Portfolio Analysis
 Size
 Complexity
 Risk
 Technical debt estimation
Zero Deployment
 No centralized source
code collection
 Portal results
 Full analysis report
CAST Application Intelligence Platform
Risk Drivers
 Robustness
 Performance
 Security
Cost Drivers
 Transferability
 Changeability
Alerts, trending, root cause analysis
Discovery Portal
Automated
App Blueprint
Discover, modernize
and change
applications
Function Point Manager
• Automated
FP counts
• Technical
Sizing
• Effort
Estimation
Function Point Changes Due to a Sequence of
Change Requests
0
5
10
15
20
25
30
35
40
0 50 100 150 200
Cumulative Effort (Staff Hours)
#FunctionPoints
1 52 3 4
Benchmarking Services
Compare to industry
business process
and technology

44. CAST Confidential 43
Year end assessment offer from CAST
 Immediate, actionable insight into a
business critical application regarding:
– Resilience and stability risk
– Performance risk
– Portfolio risk assessment
 How it works:
– An assessment will typically take 3 weeks,
the longest part of that is collecting all the
source files
– Can be delivered by CAST or a certified AI
Services partner
– Typically $10k to $50k for an assessment,
depending on the size and complexity of the
application
Contact Pete Pizzutillo
for more information

45. CAST Confidential
Contact Information
Pete Pizzutillo
p.pizzutillo@castsoftware.com
www.castsoftware.com
blog.castsoftware.com
linkedin.com/company/cast
@OnQuality
slideshare.net/castsoftware