1. ICTN 6883 System Integrity for Information Technology
Enterprise Risk Management of Vulnerability Assessment
versus Real-World Threat Assessment: Does a Dichotomy Exist?
Koger, Milton Scott CISSP, CRISC, C|EH. Information System Security Officer,
CLASS.noaa.gov, M. S. Technology System, Information Security (Graduate Fall 2015). B. G. S.
Data Management, University of New Orleans, New Orleans Louisiana, A.S. Computer Science,
Delgado Community College, New Orleans Louisiana
Rayborn, John Joseph CISSP, Premier Field Engineer at Microsoft. M.S. Technology System,
Information Security (Graduate Fall 2015). B.S. in Business Administration, Colorado State
University – Pueblo, Pueblo, Colorado. <raybornj13@students.ecu.edu>
Samson, Pierre Benoit,CISSP, CQIA, System Software Programmer at BB&T. M.S. Technology System,
Information Security (Graduate May 2015, candidate), B.S. Mechanical Engineering, Ecole de
Technologies Superieur, Montreal, Canada <Pierre.B.Samson@gmail.com>
2. Abstract— When an organization’s risk management team completes a financial evaluation
of the cost of their Information Technology (IT) assets, they typically undervalue it; which
leads to the actual risk exposure costs to be underestimated as well. This can be caused by a
multitude of factors, including the accuracy of the assets’ data availability, what valuation
system is used, the prioritization of efforts in determining vulnerabilities and exposure, the
tools utilized, and the audit effectiveness.
The intent of this paper is to provide an overview of how a continuous and automated asset
monitoring and classification system could assist an organization in quantifying the risks
and costs of each IT asset when incidents occur. This system would help prioritize risk
management activities by obtaining metrics-based asset threats and vulnerabilities. The
development and utilization of a structured process is needed in order to maintain an up-
to-date state of the actual vulnerabilities currently identified in the “real world” and to
correlate this data with potential vulnerability risks the assets are exposed to. This would
be accomplished in part through the use of the Common Vulnerability Scoring System
(CVSS) and correlations with statistical information available from sources such as Veris’
VRDB, Microsoft, Symantec, etc… Organizations have to utilize tools that identify the
threats, attacks, vulnerabilities, and countermeasures in a manner that prioritizes the
efforts and resource allocations to remediate or mitigate these risks based on solid metrics
to minimize costs and maximize results. This has to be a continuous process.
Ensuring accurate and complete financial costs evaluation of IT assets by the
organization’s risk management team is a key factor to ensure that the risk mitigation
3. prioritization efforts correlate to the assets value within their area of responsibility.
Organizations need to focus on the identification and then the reduction of real world risk
exposure of their IT assets rather than focusing only on achieving compliance based on
auditors’ reports and findings. The work performed by auditors does not come with a
warranty, therefore there is no recourse for an organization to recoup financial losses from
passing IT audits…and then failing when real world incidents occurs.
Keywords—RiskManagement; Vulnerability, CVSS; Risk Exposur, IT Assets, Threat Analysi,IT Audit, AVRDB,
CERT, CMDB COBIT, CPE, CVE, CVSS, DISA, FISMA, HIPAA
I. INTRODUCTION
Enterprises are facing a new challenge that sometimes seems unrelated to their core business [1].
With the advent of a transition toward a “service economy” the relations between service
providers and clients necessitate that the former gather more information about the later. New
laws, regulations and business imperatives require that this information remains mostly
confidential. In order to achieve compliance, the enterprises must develop a Risk Management
plan, usually based on a standard framework from where they produce standards and procedures
to manage the potential risks that they are exposed to.
These enterprises need to know the value of the assets that are at risk if they want to be able to
quantify and then prioritize the efforts and resources that will be used to mitigate the risks
associated with collecting clients’ confidential data. This is generally referred to as “Information
Security” and Information Security these days is predominantly related to Information
Technology.
4. We will therefore try to understand what tools and techniques enterprises use to value their IT
assets that are considered prone to cyber-attacks and how they evaluate the severity of these
threats. If the valuation is erroneous or the threats underestimated, the adverse consequences for
the enterprise will be worse than anticipated. “In the author’s experience, these values are often
left to out-of-the-box defaults, or are assigned inaccurately. This oversight of appropriate asset
ranking consistently can lead to inappropriate responses, alarmist consequence claims and
inefficient use of incident management resources..” [2] We think that the current methods used
and the resultant policies, controls and procedures are largely inefficient. Government agencies
have started to seriously work toward regulating different industry sectors, “SEC guidance also
requires that businesses implement risk-assessment processes, as well as more effectively assess
vendor risks and due diligence.” [3]
Many enterprises also rely solely on audit reports to address perceived flaws in their risk
management systems; therefore they lack the dynamisms and flexibility when it comes to assess
threats, old and new. This, in turn, might dramatically change the content of the assets at risk,
and the resources and costs valuation to protect these assets: “Within the last years the business
continuity management domain has gained in importance as it attaches value to determining the
effects of threats on business processes with the first priority on a company’s survival” [4]
II. IT ASSETS VALUATION MANAGEMENT
The most serious obstacle that an enterprise can face is finding a reliable methodology to
associate IT assets to services and to value everything required to allow these assets to perform
as designed. This not only means the hardware cost, but should include the labor involved in all
phases of the assets lifecycle, from planning to disposal [5]. It also means that one has to take
into account the “infrastructure” assets used to monitor and manage them; this has to include the
5. archiving infrastructure, the networking infrastructure and the monitoring/auditing infrastructure
for example. Some Information Security Risk Management frameworks (ISRM), like ISO/IEC
27005 [6] provide guidelines, but they are generally too vague, “Although this standard
introduces requirements, processes, and some criteria for identification and valuation of assets,
the specific methodology for asset analysis is not provided, which is up to the organization to
define their approach” [1]
The process is well defined as far as defining the criticality of services and the related software
or programs associated with the service. It used to be simpler to find the IT assets related to these
software since most systems used to be monolithic, one server/computer=one program. In the
relatively recent shift toward distributed architectures and even more recently in the cloud
paradigm, tracing the assets that a service and all its components are running on or utilizing is
quite an endeavor. There is a solid and reliable inventory of software to manage and monitor
assets from a technical perspective [4], but they generally lack functionality in discovering assets
relationship for a specific service and are poorly designed in associating any kind of financial
value to these assets… we’re not even thinking about collating that type of data. Some research
is done to try to find ways to achieve this goal; one approach is to simulate an attack with
automation and analyze the extent (assets) of successful penetration and build a risk model from
that: “Our novel formal model enables the mapping of real world scenarios with professional
tools like Simulink in order to perform risk-aware business process simulations. The simulation
results have been very promising, showing the negative impacts of threats and the positive effects
of safeguarding measures on the execution of business processes.” [4].
One of the models to automate the assets value assessment is to integrate the different
applications used to produce a service infrastructure. By doing so, you not only get an insight on
6. the costs but also the relationships and the resources used to build this infrastructure. Most
research points to this kind of integration:
1- Planning process applications provides the budgeted items and the High Level Design
(HLD) of the service.
2- The Project Management process and application provides the timelines, resources
allocations, Low Level Design (LLD) and all the cost discrepancies budgeted.
3- The Procurement process and applications provides the Vendor relationships, licenses,
and itemized assets costs.
4- The IT Inventory and Discovery application provides the up-to-date information about
the services components.
5- The Service Management applications provide relationship information between the
different services.
7. We will add to this model later in the paper. These graphics show the components required to
automate and assess value of assets:
Network Automation Application
Asset value replacement Db
Project management Application
Procurement Application
Server Automation Application
IT Inventory (CMDB)
Automation of Assets Valuation
Service Planning and Modeling application
Figure 1 Asset Valuation Automation Components
These components exist both commercially and in some form of Open Source or other. As with
most software applications, a single vendor solution is usually inadequate and enterprises have to
usually pick the best for each function because these components are critical in their own rights.
The major hurdle is to find an integrator that will correlate the pertinent information from all
these components and calculate the value of services. Since the value of the service is derived
from the summation of its assets’ values, built costs, etc., analyzing the cost of a partial failure is
a lot easier.
8. There would be no valid reasons to build integration like this if it was not for Risk Management
of Information Systems. Evaluating risks is the part that allows an enterprise to know which
assets are vulnerable, to which degree they are vulnerable and to interpolate the level of damage
these assets would face if the attacks were successful. An enterprise would then try to find out
how much would be the costs of such attacks would be. As we’ve seen, assets are not the same
as services. Usually multiple assets are required for a service to function, and that’s what the
enterprises want to know. They want to be able to quantify the costs associated with the risks.
So now we’ve seen that this integration between different applications and processes in an
enterprise could be used to automate the assets valuations and therefore find the service costs.
III. PRIORITIZING EFFORTS AND RESOURCES
There are a series of processes defined by the different Information Technology Risk
Management (ITRM), NIST SP 800- xx, ISO/IES 27000, COBIT [7], etc. They all require C
level management to do an initial evaluation of what criticality to assign to each service the
enterprise provides, then it usually goes down the management chain to be refined, while
keeping management priority lists intact.
While this reflects the true corporate business goals of stability, it lacks in finding the real
underlying applications’ drivers, which enable these services to function. For example, for a
bank, the branch operation service might have been classified as “Business Critical”, but the
Regional Carriers that serve all rural areas has been classified “Important”. If one of these
providers goes dark for any reason, the Business Critical Branch Operation might be 75% down
and the resources planned for the “Important” services might not be adequate to meet the
criticality of recovering the Branch Operation service. This happens because the enterprise did
9. not take into account the relationship between the two services, or misjudged their respective
interdependency.
Enterprises need to have a way to evaluate the real risks based on real threats. In the Information
Technology realm, there is a series of data sources that can help in identifying what these threats
are. For basic configuration management, the NIST 80- xx list of controls is very extensive and a
series of server/network management systems are able to be customized to scan systems and
generate reports on the state of systems, devices and even software. These are also generally
stable, meaning that the specifications don’t change often; therefore enough time can be
allocated to make modifications and satisfy compliance requirements. Where it gets more
difficult is when new vulnerabilities are found on assets that were thought to be safe. These
vulnerabilities are found regularly and usually need to be addressed very rapidly.
The vulnerabilities found often come from the Common Vulnerability and Exposure database
maintained by Mitre Corporation. It is described as “CVE is a dictionary of publicly known
information security vulnerabilities and exposures”.
IV. INTEGRATION
Every Risk Management team has at their disposal an important source of vulnerability
classification and information in the National Vulnerability Database (NVD).This database is
maintained by NIST. The NVD contains the information provided by CVE, US-CERT Alerts,
US-CERT Vulnerability Notes, OVAL queries, and CPE Names that is updated on a regular
basis. The Common Vulnerability Scoring System (CVSS) calculator is also available at the
NIST website and can be used to assess the criticality of each discovered vulnerability. There can
be a potential pitfall in using the CVSS base score because it only provides one perspective into
10. the overall score of the vulnerability. Since the base score leaves out both the temporal and
environmental scores by default, it does not provide valuable selection criteria. However, both
the version 2 and 3 of the CVSS calculators include the ability to input either, or both the
temporal and environmental scores thus giving a much more accurate evaluation of the overall
vulnerability score. “The base CVSS score does not and cannot provide information on risks of
known vulnerabilities to a specific environment, e.g. to a specific organization or industry.” [8].
Incorporating all three aspects into the calculation provides a better understanding of how the
particular vulnerability might affect the organization.
Without accomplishing a thorough review of a vulnerability by including the base, temporal, and
environmental scores, and its potential impact to the organization, the identification of the “Top
X” percent vulnerabilities for remediation, will more than likely be skewed. This can cause the
Risk Management team to put forth efforts toward the remediation of inconsequential
vulnerabilities rather than fixing those that are truly of consequence to the organization. This, in
turn, increases the risk exposure to the vulnerabilities that were reviewed and not positively
identified because of a low base score. This is caused by not accomplishing a thorough review of
all the factors that could have been utilized in determining the true overall score of the
vulnerabilities that really affect the organization’s environment. Another adverse consequence of
the lack of evaluating all criteria would have for effect that a vulnerability which may initially
have a medium to high base score would reduce the rating of the risk associated with the
vulnerability specific to the organization and it may not make the “Top X” percent for
remediation. It is therefore very important to take in consideration all three, The real
consequence for the organization is that it will waste time and resources in the remediation of a
11. vulnerability that may be of little consequence for the information system while leaving other
vulnerabilities open for attacks.
The utilization of all three components of base, temporal, and environmental scores in the
compilation of an overall vulnerability score will increase the confidence of the Risk
Management team in their selection and guaranty that they are targeting the vulnerabilities with
the highest probability of risk exposure within their organization. This confidence is further
demonstrated and validated with the use of all three components, such that, “… the addition of
context information improves the scores’ reflection of the actual severity of a vulnerability from
the organization’s point of view. A better reflection of reality in the scores further improved the
prioritization of vulnerabilities and the selection of more efficient response processes” [9].
V. TOOLS
As stated previously, there are a variety of tools available to assist Risk Management teams in
evaluating and prioritizing vulnerabilities for remediation. We can find some that are open
source and a wide variety of commercial ones. Regardless of the tool(s) utilized to evaluate
vulnerabilities and threats against an organization information system, the knowledge on how to
use the tool(s) as well as an understanding of the business processes in place or that need to be
developed, is paramount to being able to successfully and accurately determine the prioritization
efforts. “Technologies aside, information risk management must be baked into every business
process” [10]. The integration of the components within the asset valuation model is the pivotal
aspect in finding the value of not only the assets of the organization, but the value of the services
that the organization provides. Providing this model to the Risk Management team puts the right
toolset in the right hands for evaluation and prioritization of the vulnerabilities that might affect
the organization.
12. One toolset for the collation and correlation of the assets valuation model is the Asset Valuation
Replacement Database (AVRDB). To be successful, accurate, valid and verified, data must be
fed in order to provide the most dependable information for review and analysis. Both a Network
Automation Application and a Server Automation Application feed detailed assets data on a
regular interval to the IT Inventory Configuration Management Database (CMDB), where this
data is aggregated and provided to the AVRDB. Other components such as the Asset
Procurement Application, Project Management Application, and Service Planning and Modeling
Application also provide input to the AVRDB for establishing service relationship, detailed
costs, resource utilization and service prioritization.
Once the information from all components is provided, the Risk Management team can then
review the reports and have a better understanding to ascertain the values of the assets and
services provided by the organization. This will benefit the Risk Management team when
justifying programs, projects and changes to C-level executives. It is also necessary when
presenting a value proposition for efforts regarding vulnerability prioritization for remediation,
resource allocation and implementation. This will be supported by the asset valuation and the
correlation with critical services.
VI. AUDITS, AUDITORS, AND STANDARDS
Now that a real-world threat analysis can be accomplished by the organization, there a need to
focus on how real-world threat analysis relates to compliance. It is generally understood that to
be compliant you need to satisfy not only audits and auditors’ control verification [11], but also
standards, internals or externals and regulations. “It is imperative for Information Technology
(IT) departments to insure that their applications meet all compliance requirements that govern
their products, services, and other activities. Therefore, it is essential for IT personnel to be
13. aware of the implications of compliance regulations when developing and operating IT systems”
[11]. This is also true of Risk Management teams and their knowledge of how audits are
conducted, what their goals are and who is auditing. This understanding is critical in maintaining
compliance with industry standards (like Sarbanes-Oxley, HIPPA, etc…) and regulations (like
FISMA, DISA, etc…) that affect a particular organization.
The key for the Risk Management team through the use of an automated assessment tool is the
ability to determine the vulnerabilities that should be prioritized for remediation while being
aware of the potential impact to the business organization processes. The goal, in a Risk
Exposure context, is to being effective in the application of remediation and mitigation that are
internally deemed critical or important while maintaining compliance with industry standards
and government regulations as verified by audits, internal or external... This is critical, otherwise
the organization may find itself in a predicament where “…businesses might overreact to
information security threats, resulting in overspending on information security projects that
penalize operational flexibility” [12]. This, of course, would surely impact the secondary goal of
the Risk Management team, which is to allow the business to function optimally, therefore
accomplishing their part of the corporation’s mission, to satisfy the auditors in being able to
prove that the controls in place are effective and to enable end-users to accomplish their tasks
with minimal impact from security implementations.
VII. CONCLUSION
Risk Management teams and organizations as a whole need to make more concerted efforts to
make sure that the enterprise risk management assets vulnerability valuation assessments reflect
what is really happening in the real world. While there is a potential for a dichotomy in the
valuation of these two assessments, it is imperative that the Risk Management team, having
14. executive management support, uses the automation tools, capabilities and their knowledge to
prevent this.
The Risk Management team must be able to review vulnerabilities and determine the real threats
to the assets of the organization. This can be accomplished through the various automations and
resources available publically. The security communities and public agencies like CVSS,
Verizon Threat Intelligence Report, VERIS Community Database, Microsoft Security Bulletins,
etc… are there to help Risk Management practitioners. Through research from resources such as
these, the Risk Management team will be able to utilize this information along with the asset and
services valuation data to determine the potential impact to the organization and what
remediation / mitigation actions are available to minimize this impact. This will provide both
quantitative and qualitative data that will help in the discussion and justification about the
potential effects of remediation and mitigation with executive management if necessary. The
information will also provide a basis in determining whether the vulnerability should be
remediated, mitigated by a workaround, or whether the risk exposure is acceptable to the
organization.
In order to accomplish this, the organization must have a model in place to accomplish both the
identification and valuation of the assets. This is the first step. Secondly, the organization needs
to make an inventory of the business processes and services that these assets provide. Finally, the
assets’ criticality must be evaluated and prioritized based on the criticality of the business
services that they are attached to. The Risk Management team then has to attach a risk ratio to
15. these assets valuation and calculate the expected loss. This will help in prioritizing the
remediation efforts, auditors or not.
Continuous automated inventory and valuation enable this cross-functional relationship between
the assets and the services provided through the business processes of the organization. It is
critical in providing an accurate assessment of the cost of maintaining an information system at a
known acceptable risk level.
Every organization would like to pass all of the industry standards and government regulations
compliance audits. In our research, it became clear that the real value and benefits of automating
these processes is to incorporate near real time, real world threats discovery and remediation into
the traditional “satisfy audit” model. It does not come by meeting and passing the audits on a
scheduled basis. This way, the organization has a precise quantification of the costs and risk
level that it is exposed to.
The baselines and guidelines provided by industry standards and government regulations only
provide a basic and minimal starting point in the protection of assets and services when it comes
to confidentiality, integrity, and availability. The control processes created and accomplished by
the Risk Management team are designed to not only incorporate these baselines, but to minimize
the risk exposure to real-world threats and vulnerabilities. Only when all of these considerations
are taken into account will the Risk Management team be able to conclude that vulnerability
assessment and real-world threat assessment are one and the same.
16. Key Terms
AVRDB – Asset Valuation Replacement Database
CERT – Computer Emergency Response Team
CMDB – Configuration Management Database
COBIT – Common Objectives for Information and Related Technology
CPE – Common Platform Enumeration
CVE – Common Vulnerability and Exposure
CVSS – Common Vulnerability Scoring System
DISA – Defense Information Systems Agency
FISMA – Federal Information Systems Management Act
HIPAA – Health Insurance Portability and Accountability Act
IEC – International Electro technical Commission
ISO – International Organization for Standards
ISRM – Information Security Risk Management
NIST – National Institute for Standards and Technology
NVD – National Vulnerability Database
OVAL – Open Vulnerability and Assessment Language
17. SOX – Sarbanes-Oxley
SP – Special Publication
VIII. REFERENCES
[1] N. Liu, J. Zhang and X. Wu, "Asset Analysis of Risk Assessment for IEC 61850-Based
Power Control Systems—Part I: Methodology," Power Delivery, IEEE Transactions on
(Volume:26 , Issue: 2 ), vol. 26, no. 2, pp. 869-875, 2010.
[2] L. Beaudoin, "Asset Valuation Technique for Network Management and Security," in Data
Mining Workshops, 2006. ICDM Workshops 2006. Sixth IEEE International Conference on,
Hong Kong, 2006.
[3] PWC, "Managing cyber risks in an interconnected world," PWC, New York, 2015.
[4] S. Tjoa, S. Jakoubi, G. Goluch, G. Kitzler, S. Goluch and G. Quirchmayr, "A Formal
Approach Enabling Risk-Aware Business Process Modeling and Simulation," Services
Computing, IEEE Transactions on, vol. 4, no. 2, pp. 153-166, 2011.
[5] R. Sheikhpour, "A best practice approach for integration of ITIL and ISO/IEC 27001
services for information security management," Indian journal of science and technology, p.
2170, 2012.
[6] ISO 2700 Security, "ISO 27K Toolhit," 23 June 2013. [Online]. Available:
18. http://www.iso27001security.com/html/iso27k_toolkit.html.
[7] ISO, "Information Security Standards," 3 September 2012. [Online]. Available:
http://www.iso27001security.com/ISO27k_FAQ.pdf.
[8] A. Ali, P. Zavarsky, D. Lindskog and R. Ruhl, "A software application to analyze the effects
of temporal and environmental metrics on overall CVSS v2 score,," in Internet Security
(WorldCIS), 2011 World Congress on, London, 2011.
[9] C. Fruhwirth and T. Mannisto, "Improving CVSS-based vulnerability prioritization and
response with context information,," in Empirical Software Engineering and Measurement,
2009. ESEM 2009. 3rd International Symposium on, Lake Buena Vista, FL, 2009.
[10] M. Johnson, E. Goetz and S. Pfleeger, "Security through Information Risk Management,"
Security & Privacy, vol. 7, no. 3, 2009.
[11] V. Gudivada and J. Nandigam, "Corporate Compliance and its Implications to IT
Professionals," in Information Technology: New Generations, 2009. ITNG '09. Sixth
International Conference on, Las Vegas, NV, 2009.
[12] V. C. S. Lee and L. Shao, "Estimating Potential IT Security Losses: An Alternative
Quantitative Approach," Security & Privacy, vol. 4, no. 6, pp. 44-52, 2007.
[13] X. Zhaoa, L. Xuea and A. B. Whinston, "Managing Interdependent Information Security
Risks: Cyberinsurance, Managed Security Services, and Risk Pooling Arrangements,"
Journal of Management Information Systems, vol. 30, no. 1, pp. 123-152, 2013.
19. [14] F. Kamoun and M. Nicho, "Human and organizational factors of healthcare data breaches:
the Swiss cheese model of data breach causation and prevention," International Journal of
Healthcare Information Systems and Informatics, vol. 9, no. 1, p. 42, 2014.
[15] Cloud Security Alliance, "Cloud Computing Vulnerability Incidents: A Statistical
Overview," Cloud Security Alliance, 2013.
[16] Verizon, "Example VERIS incident recording tool," 02 02 2014. [Online]. Available:
https://incident.veriscommunity.net/s3/example.
[17] Steven Hernandez, Official (ISC)2 Guide to the CISSP CBK, Third Edition, Bota Raton.FL:
CRC Press, 2013, 2013.
[18] Y. U. Bryan Ford, "Icebergs in the Clouds: The Other Risks of Cloud Computing," in
HotCloud '12, 4th USENIX Workshop on Hot Topics in Cloud Computing, Boston, 2012.
[19] J. Cohen, Intangible Assets : Valuation and Economic Benefit, Hoboken, NJ: John Wiley &
Sons, 2005.
[20] C. Everett, "Information security initiatives: counting the cost," Computer Fraud &
Security, pp. 6-7, 2010.
[21] W.-H. L. Fa-Chang Chenga, "The Impact of Cloud Computing Technology on Legal
Infrastructure within Internet—Focusing on the Protection of Information Privacy," in 2012
International Workshop on Information and Electronics Engineering, 2012.
[22] C. Fisher, "Auditor independence - the hot new issue," Chartered accountants journal of
20. New Zealand, p. 22, 2003.
[23] L. Lee, "Acronym overload: internal auditors should become familiar with the wide range of
IT-related regulations and standards that may impact their work," The Internal auditor, p.
21, 2013.
[24] G. Stewart and D. Lacey, "Death by a thousand facts: Criticising the technocratic approach
to information security awareness," Information Management & Computer Security, Vol. 20
Iss: 1, pp.29 - 38, vol. 20, no. 1, pp. 29 - 38, 2011.
[25] P. Wheatcroft, "Using ISO/IEC 27000 To Benchmark Your Security Profile- Peter
Wheatcroft (Partners in IT)," in itSMF Seminar Manchester - 2nd September 2008,
Manchester, 2008.