Managing The Virtualized Enterprise New Technology, New Challenges

Managing The Virtualized Enterprise
New Technology, New Challenges
The importance of consolidation, correlation, and detection
Enterprise Security Series

White Paper
8815 Centre Park Drive Published: June 15, 2009
Columbia MD 21045
877.333.1433

Abstract
The benefits of employing virtualization in the corporate data center are compelling – lower operating
costs, better resource utilization, increased availability of critical infrastructure to name just a few. It is an
apparent “no brainer” which explains why so many organizations are jumping on the bandwagon. Industry
analysts estimate that between 60 and 80 percent of IT departments are actively working on server
consolidation projects using virtualization. But what are the challenges for operations and security staff
when it comes to management and ensuring the security of the new virtual enterprise? With new
technology, complexity and invariably new management challenges generally follow.
Over the last 18 months, Prism Microsystems, a leading security information and event management
(SIEM) vendor, working closely with a set of early adopter customers and prospects, has been working on
extending the capability of EventTracker to provide deep support for virtualization, enabling our customers
to get the same level of security for the virtualized enterprise as they have for their non-virtualized
enterprise. This White Paper examines the technology and management challenges that result from
virtualization, and how EventTracker addresses them.

The information contained in this document represents the current view of Prism
Microsystems Inc. (Prism) on the issues discussed as of the date of publication.
Because Prism Microsystems must respond to changing market conditions, it
should not be interpreted to be a commitment on the part of Prism. Prism cannot
guarantee the accuracy of any information presented after the date of
publication.
This document is for informational purposes only. Prism MAKES NO
WARRANTIES, EXPRESS OR IMPLIED, AS TO THE INFORMATION IN THIS
DOCUMENT.
Complying with all applicable copyright laws is the responsibility of the user.
Without limiting the rights under copyright, this paper may be freely distributed
without permission from Prism, as long as its content is unaltered, nothing is
added to the content and credit to Prism is provided.
Prism may have patents, patent applications, trademarks, copyrights, or other
intellectual property rights covering subject matter in this document. Except as
expressly provided in any written license agreement from Prism Microsystems,
the furnishing of this document does not give you any license to these patents,
trademarks, copyrights, or other intellectual property.
The example companies, organizations, products, people and events depicted
herein are fictitious. No association with any real company, organization, product,
person or event is intended or should be inferred.
© 2009 Prism Microsystems Inc. All rights reserved.
The names of actual companies and products mentioned herein may be the
trademarks of their respective owners.

Managing The Virtualized Enterprise New Technology, New Challenges

New Complexity, New Challenges
The introduction of virtualization has changed the playing field when it comes to managing the security and
operations of the corporate enterprise.
Until virtualization there had always existed a fairly close relationship between the hardware and software
layers of a computing infrastructure. A server machine was typically a “box”, .i.e. a self-contained machine
consisting of a chassis, CPU’s, an operating system (UNIX, Linux or Windows typically) with some
applications installed and some disk spaced mapped. Network equipment were other “boxes” that managed
the network traffic between servers and desktops. Once provisioned the server and the network equipment
became fairly static and straightforward to manage.
Over the last ten years this relationship at least on the server side has been complicated by the move to
specialized storage devices and rack and blade systems. Despite this growth in complexity, it was still
relatively manageable overall. To provide visibility into the workings of the server you monitored the
Operating System and by doing this you got limited, but adequate, visibility into the underlying hardware
layer, as well as the application layer. The network produced management information that provided
visibility into the information flowing between machines. From a management standpoint you had a set of
trusted users or administrators that were responsible for the machines, a different network team and in bigger
companies, occasionally some storage specialists and a security group. Everyone had distinct and fairly well-
defined duties. It was not perfect, but the complexity could be managed.

Virtualization
With the mainstream arrival of virtualization the close relationship between the physical and the software
layer is now completely severed. Now at best there is a loose coupling of the OS instance with the platform it
runs on and there is an entirely new, virtualized layer that separates the two as well. The close relationship of
OS to physical infrastructure has been replaced by the virtualization layer – the hypervisors and management
tools that manage the setup and deployment of the virtual machines. The host OS still has control over the
application layer, but the hardware is allocated through the VM management layer. The Hypervisors also
support network communication between virtual machines which side-steps the classic network group that
traditionally controlled traffic on the wire. Further complicating the equation is that with virtual networking,
network traffic sometimes never gets onto the wire which renders most network security tools ineffective.

Systems Management
Many organizations are also deploying systems management applications in the form of Dell OpenManage or
HP Insight Manager to manage large scale server farms. These have become important as enterprises move to
“rack and stack”, where virtual servers are often dependent on shared infrastructure to operate. With
potentially many servers dependant on shared infrastructure it becomes important to monitor the hardware
state, as a small hardware failure can have a catastrophic impact on service. These management applications
can help manage at the hardware layer and at the OS or software layer, but typically do not provide the

Prism Microsystems, Inc. 3


richness of a specialty virtualization management product and most experts in the field caution against using
such solutions for the virtual layer.
The addition of this new virtualization layer compounds the complexity of management and monitoring.
There are different and sometimes more critical points of failure, and there are entirely new systems and
applications that need to be monitored. Prior to racks and virtualization, if a machine failed it would take out
a couple (at most) of critical applications. Today if a rack fails it might take out 10 physical servers. A single
physical server could be running 8-10 guest Operating Systems, with each of those running critical
applications and services, so even a single physical server machine failure can be catastrophic. In addition, if
the management application for the virtual infrastructure is successfully attacked or hijacked there is
potential for operational carnage. Server sprawl was messy and inefficient to manage, with lots of points of
failure, but there were few points of failure that could literally take an entire company off-line. In the new
Virtualized enterprise there are more, different and even more critical services to monitor.

Organizational Change
For separation of duties and operational efficiency in many organizations that have adopted virtualization
there is now an admin team that is responsible for management of the virtual layer – the provisioning and
creation of virtual machines. But the clear separation of duties that existed pre-virtualization has blurred – the
virtual team might, for instance, have to worry about networks if they are using virtualization for
communication between guest machines.
Imagine the most simple of examples from this new paradigm – prior to virtualization you turned a machine
on and an OS typically booted up. Done. Now you switch on a machine and the virtualization layer takes
over. It then manages the creation of potentially multiple virtual machines running different Operating
Systems with distinct network configurations. Virtual Machines start and stop, they can move dynamically
from physical machine to physical machine. Even the disk space and often the network are mapped in the
virtual world.
This discussion is not to imply that virtualization is inherently insecure in any way, it is simply changing the
way businesses need to operate and think about their security. There are new and different critical
applications and infrastructure that need to be monitored and brand new threats – and consequently the
approaches to monitoring and prevention must adapt.

SIEM in the Virtual Enterprise
Security Information and Event Management solutions have three real purposes in life. First to help prevent
attacks and security breaches from either internal or external bad actors. With virtualization the attack service
changes. Before virtualization you could attack at the hardware layer or hijack a machine during the boot
process. The other option was to attack at the OS/software layer. Now a hacker can attack the VM layer as
well. Once in the VM layer, the hacker can reconfigure machines and potentially traverse into a guest OS.
Since VMs can all be running on the same physical machine the hacker can then traverse from machine to
machine in the host without the network traffic ever being visible on the wire.
The second purpose of SIEM solutions is to help companies meet compliance by tracking user and
administrator activity and access. With virtualization there is an entirely new set of power users that are
acting in the enterprise – the administrators that manage the virtual layer. They need to be audited as well.



One of the best ways to secure a VM infrastructure is by enforcing strict separation of duties – for example
the persons responsible for the virtual infrastructure (provisioning etc.) and the virtual machine instances
themselves (OS and applications) should not be the same if at all possible, and the network, server and virtual
management teams should have policy-based segregation of duties.
Finally the third purpose is to ensure smooth continuing operations. Having a consolidated view of all the
events happening in the enterprise increases the overall availability of IT service. In an increasingly complex
infrastructure automating these tasks with a SIEM solution is the only way to detect the small signs of
impending problems in advance.
In order to ensure security and smooth operations, enterprise visibility must be maintained and collection of
logs from all distinct layers must be performed. In the next pages we look at several important technologies
that need to be monitored as they have become important layers of the system infrastructure in a virtualized
enterprise, and offer a hacker new attack vectors. In order to keep this manageable we have focused on the
“machines” – the racks, the servers, the storage devices and the software that controls them. We will look at
the types of events generated by Dell OpenManage and both VMware and Microsoft Hyper-V events. The
ability to manage the network, application and OS elements are an assumption, and are already supported
with existing SIEM solutions.



Unified Server Management
Unified Server Management offerings or what HP refers to as “Unified Infrastructure Management” are a
series of management products that are designed to manage the entire IT infrastructure – from the Chassis to
the Network Attached Storage and from the OS level down to the bare-bones hypervisor. These applications
can collect IPMI information that provides rich, low level information on the state of the hardware, and as
they are provided by the server vendors (Dell OpenManage, HP Insight Manager, IBM), they provide a great
deal of information on the state of the SAN devices if a company has standardized on a single vendor for
both storage and systems. These systems also provide a rich set of commands to configure, patch and operate
the hardware, OS and storage of the infrastructure.
With Blades and Racks and shared resource pools of hardware components it is advisable to collect and
monitor logs coming from these applications. In large scale virtualized enterprises these applications are
often used side by side with a Vcenter.

Pre-OS Events
Once, it was safe to assume that when you powered a machine off, it became unreachable. Now with a
combination of UPS, networks and IPMI, even machines that are powered off are still potentially accessible.
The Intelligent Platform Management Interface (IPMI) standard has existed since 1998 with the majority of
the major chip set vendors such as Intel and AMD, and Server Vendors such as Dell and HP, supporting the
Standard. IPMI runs on the Baseboard Management Controller and allows Administrators to remotely
manage a system before an OS is even booted or the power switched on. This powerful combination of
capabilities enables an IT organization to substantially reduce the cost of server maintenance, however it also
opens a potential path for hackers to get in and cause damage. In IPMI 2.0, for example, a person remotely
accessing the interface is able to discover all the commands available to them and perform inventory on the
underlying platform, as well as change hardware settings on the machine. In addition, once the OS has been
booted, the BMC and IPMI can continue to run if provided a power source enabling another entry point into
the device, outside of the operating system.
With this capability, monitoring access through IPMI is a must. Unfortunately a single standard for IPMI trap
generation does not exist and the platform vendors have integrated the IPMI functionality into the Server
Management Systems. Information can be generated from various sources including the BIOS, OS Bootstrap
Loader, Network Interface Card, System Alert ASIC, System Management Micro-controller, System
Management Software and the Alert Proxy Software. A great deal of useful operational data with regards to
the state of the system hardware, memory and disks becomes available. In addition important security and
audit events are generated for IPMI user-logon failures, system reconfiguration or the turning off of logging
in IPMI.



OpenManage Events
Array Disk Events
2106 Smart FPT (predictive failure) exceeded. The disk is likely to fail in the near future.
2107 Smart configuration change. The disk is likely to fail in the near future.
2108 Smart warning. The disk is likely to fail in the near future.
2109 SMART warning temperature. The disk is likely to fail in the near future.
2110 SMART warning degraded. The disk is likely to fail in the near future.
2111 Failure prediction threshold exceeded due to test - No action needed
2094 Predictive Failure reported. The disk is likely to fail in the near future.
2095 SCSI sense data. A SCSI device experienced an error, but may have recovered

Automatic System Recovery
1006 Automatic System Recovery (ASR) action was performed. The Operating System was
hung

Battery Sensor Events
1700 Battery sensor has failed
1701 Battery sensor value unknown
1702 Battery sensor returned to a normal value
1703 Battery sensor detected a warning value
1704 Battery sensor detected a failure value
1705 Battery sensor detected a non-recoverable value
2104 Controller battery is reconditioning
2105 Controller battery recondition is completed
2169 The controller battery needs to be replaced.
2170 The controller battery charge level is normal.
2171 The controller battery temperature is above normal.
2172 The controller battery temperature is normal.
2174 The controller battery has been removed.
2175 The controller battery has been replaced.
2176 The controller battery Learn cycle has started.
2177 The controller battery Learn cycle has completed.
2178 The controller battery Learn cycle has timed out.
2179 The controller battery Learn cycle has been postponed.
2180 The controller battery learn cycle will start in %1 days.
2181 The controller battery Learn cycle will start in %1 hours.
2215 Battery charge process interrupted
2216 The battery learn mode has changed to auto.
2217 The battery learn mode has changed to warn.

BIOS Update Schedule Events
1002 A system BIOS update has been scheduled for the next reboot
1003 A previously scheduled system BIOS update has been canceled



Chassis Intrusion
1250 Chassis intrusion sensor has failed
1251 Chassis intrusion sensor value unknown
1252 Chassis intrusion returned to normal
1253 Chassis intrusion in progress
1254 Chassis intrusion detected
1255 Chassis intrusion sensor detected a non-recoverable value

Chassis Management Controller (CMC) Events
2000 CMC generated a test trap
2002 CMC reported a return-to-normal or informational
2003 CMC reported a warning
2004 CMC reported a critical event
2005 CMC reported a non-recoverable event

Cooling Device Events
1100 Fan sensor has failed
1101 Fan sensor value unknown
1102 Fan sensor returned to a normal value
1103 Fan sensor detected a warning value
1104 Fan sensor detected a failure value
1105 Fan sensor detected a non-recoverable value

Current Sensor Events
1200 Current sensor has failed
1201 Current sensor value unknown
1202 Current sensor returned to a normal value
1203 Current sensor detected a warning value
1204 Current sensor detected a failure value
1205 Current sensor detected a non-recoverable value

Disk Error
2273 A block on the physical disk has been punctured by the controller
2306 Bad block table is 80% full.
2307 Bad block table is full. Unable to log block
2331 A bad disk block has been reassigned.
2340 The BGI completed with uncorrectable errors.
2349 A bad disk block could not be reassigned during a write operation.

Enclosure Events
2138 Enclosure alarm enabled
2139 Enclosure alarm disabled
2151 Asset tag changed
2152 Asset name changed
2153 Service tag changed



2162 Communication with enclosure regained
2173 Unsupported configuration n detected. The SCSI rate of the enclosure management
modules (EMMs) is not the same.
2190 The controller has detected a hot plugged enclosure.
2191 Multiple enclosures are attached to the controller. Unsupported configuration.

Firmware
2120 Enclosure firmware mismatch
2128 BGI cancelled
2131 Firmware version mismatch
2165 The RAID controller firmware and driver validation was not performed. The
configuration file cannot be opened.
configuration file is out of date or corrupted.
2311 The firmware on the EMMs is not the same version.

Hardware Log Sensor
1550 Log monitoring has been disabled
1551 Log status is unknown
1552 Log size is no longer near or at capacity
1553 Log size is near or at capacity
1554 Log size is full
1555 Log sensor has failed

Log backup and clear
0000 Log was cleared
0001 Log backup created

Memory Device
1403 Memory device status warning. Correction rate exceeded acceptable value.
1404 Memory device status warning. A memory device correction rate exceeded an
acceptable value, a memory spare bank was activated, or a multibit ECC error
occurred.

Physical Disk
2049 Physical disk removed
2050 Physical disk offline
2051 Physical disk degraded
2052 Physical disk inserted
2060 Copy of data started on physical disk %1 from physical disk %2.
2062 Physical disk initialization started
2065 Physical disk rebuild started
2074 Physical disk rebuild cancelled
2075 Copy of data completed on physical disk %2 from physical disk %1
2080 Physical disk initialize failed
2083 Physical disk rebuild failed



2087 Copy of data resumed from physical disk %2 to physical disk %1
2089 Physical disk initialize completed
2092 Physical disk rebuild completed
2141 Physical disk dead segments recovered
2146 Bad block replacement error. A portion of a physical disk is damaged.
2147 Bad block sense error. A portion of a physical disk is damaged.
2148 Bad block medium error. A portion of a physical disk is damaged.
2149 Bad block extended sense error. A portion of a physical disk is damaged.
2150 Bad block extended medium error. A portion of a physical disk is damaged.
2158 Physical disk online
2195 Dedicated hot spare assigned. Physical disk %1
2196 Dedicated hot spare unassigned. Physical disk %1
2198 The physical disk is too small to be used for Replace member operation
2211 The physical disk is not supported.
2183 Replace member operation failed on physical disk %1
2184 Replace member operation cancelled on physical disk
2185 Replace member operation stopped for rebuild of hot spare on physical disk
1650 Unknown device plug event type received.
1651 Device added to system
1652 Device removed from system
1653 Device configuration error detected
1500 AC power cord sensor has failed
1501 AC power cord is not being monitored
1502 AC power has been restored
1503 AC power has been lost
1350 Power supply sensor has failed
1351 Power supply sensor value unknown
1352 Power supply returned to normal
1353 Power supply detected a warning
1354 Power supply detected a failure
1355 Power supply sensor detected a non-recoverable value
1600 Processor sensor has failed
1601 Processor sensor value unknown
1602 Processor sensor returned to a normal value
1603 Processor sensor detected a warning value
1604 Processor sensor detected a failure value
1605 Processor sensor detected a non-recoverable value
2048 Device failed
2056 Virtual disk failed
2076 Virtual disk check consistency failed
2077 Virtual disk format failed
2079 Virtual disk initialization failed
2080 Physical disk initialize failed
2081 Virtual disk reconfiguration failed
2082 Virtual disk rebuild failed
2083 Physical disk rebuild failed
2094 Predictive disk failure reported.



2101 Temperature dropped below the minimum warning threshold
2102 Temperature exceeded the maximum failure threshold
2103 Temperature dropped below the minimum failure threshold
2106 Smart FPT (predictive failure) exceeded. The disk is likely to fail in the near future.
2107 Smart configuration change. The disk is likely to fail in the near future.
2108 Smart warning. The disk is likely to fail in the near future.
2109 SMART warning temperature. The disk is likely to fail in the near future.
2110 SMART warning degraded. The disk is likely to fail in the near future.
2112 Enclosure was shut down. The physical disk enclosure is either hotter or cooler than
the maximum or minimum allowable temperature range.
2123 Redundancy lost
2125 Controller cache preserved for missing or offline virtual disk
2129 Virtual disk BGI failed
2131 Firmware version mismatch
2132 Driver version mismatch
2137 Communication timeout
2146 Bad block replacement error
2148 Bad block medium error
2149 Bad block extended sense error
2150 Bad block extended medium error
2163 Rebuild completed with errors
configuration file cannot be opened.
2167 The current kernel version and the non- RAID SCSI driver version are older than the
minimum required levels.
2168 The non- RAID SCSI driver version is older than the minimum required level.
2169 The controller battery needs to be replaced.
2182 An invalid SAS configuration has been detected.
2183 Replace member operation failed on physical disk %1. The physical disk being
replaced has failed.
2191 Multiple enclosures are attached to the controller. This is an unsupported
configuration.
2201 A global hot spare failed.
2250 Redundant Path is broken
2264 A device is missing.
2265 A device is in an unknown state.
2268 Storage Management has lost communication with the controller.
2270 The physical disk clear operation failed.
2272 Patrol Read found an uncorrectable media error.
2282 Hot spare SMART polling failed.
2283 A redundant path is broken.
2289 Multi-bit ECC error.
2292 Communication with the enclosure has been lost.
2293 The EMM has failed.
2295 A device has been removed.
2297 An EMM has been removed.
2299 Bad physical connection



2300 The enclosure is unstable.
2301 The enclosure has a hardware error.
2302 The enclosure is not responding.
2307 Bad block table is full. Unable to log block
2310 A virtual disk is permanently degraded.
2314 The initialization sequence of SAS components failed during system startup. SAS
management and monitoring is not possible.
2316 Diagnostic test failed.
2319 Single-bit ECC error. The DIMM is degrading.
2320 Single-bit ECC error. The DIMM is critically degraded.
2321 Single-bit ECC error. The DIMM is critically degraded. There will be no further
reporting.
2322 The DC power supply is switched off.
2336 Controller event log: %1. Controller generated event log while Storage Management
was not running
2337 The controller is unable to recover cached data from the battery backup unit (BBU).
2340 The BGI completed with uncorrectable errors.
2346 Physical device error occurred.
2347 The rebuild failed due to errors on the source physical disk.
2348 The rebuild failed due to errors on the target physical disk.
2349 A bad disk block could not be reassigned during a write operation.
2350 There was an unrecoverable disk media error during the rebuild.
2356 SAS SMP communications error.
2357 SAS expander error.
2373 Attempted import of unsupported Virtual Disk type

Redundancy Unit
1300 Redundancy sensor has failed
1301 Redundancy sensor value unknown
1302 Redundancy not applicable
1303 Redundancy is offline
1304 Redundancy regained
1305 Redundancy degraded
2098 Global hot spare assigned
2099 Global hot spare unassigned
2122 Redundancy degraded
2124 Redundancy normal
2163 Rebuild completed with errors
2167 The current kernel version and the non- RAID SCSI driver version are older than the
minimum required levels.
2168 The non- RAID SCSI driver version is older than the minimum required level.
2197 Replace member operation has stopped for rebuild.
2200 Replace member operation is not possible as combination of SAS and SATA physical
disks is not supported in the same virtual disk.
1000 Server Administrator starting



1001 Server Administrator startup complete
1050 Temperature sensor has failed
1051 Temperature sensor value unknown
1052 Temperature sensor returned to a normal value
1053 Temperature sensor detected a warning value
1054 Temperature sensor detected a failure value
1055 Temperature sensor detected a non-recoverable value
2100 Temperature exceeded the maximum warning threshold
2101 Temperature dropped below the minimum warning threshold
2102 Temperature exceeded the maximum failure threshold
2103 Temperature dropped below the minimum failure threshold
2154 Maximum temperature probe warning threshold value changed
2155 Minimum temperature probe warning threshold value changed

Virtual Disk Events
2053 Virtual disk created
2054 Virtual disk deleted
2055 Virtual disk configuration changed
2056 Virtual disk failed
2057 Virtual disk degraded
2058 Virtual disk check consistency started
2059 Virtual disk format started
2061 Virtual disk initialization started
2063 Virtual disk reconfiguration started
2064 Virtual disk rebuild started
2067 Virtual disk check consistency cancelled
2070 Virtual disk initialization cancelled
2076 Virtual disk Check Consistency failed
2077 Virtual disk format failed
2079 Virtual disk initialization failed
2081 Virtual disk reconfiguration failed
2082 Virtual disk rebuild failed
2085 Virtual disk check consistency completed
2086 Virtual disk format completed
2088 Virtual disk initialization completed
2090 Virtual disk reconfiguration completed
2091 Virtual disk rebuild completed
2114 A consistency check on a virtual disk has been paused (suspended)
2115 A consistency check on a virtual disk has been resumed
2116 A virtual disk and its mirror have been split
2117 A mirrored virtual disk has been un-mirrored
2118 The write policy change write policy
2125 Controller cache preserved for missing or offline virtual disk
2127 Background initialization (BGI) started
2129 BGI failed
2130 BGI completed
2136 Virtual disk initialization OK / Normal
2159 Virtual disk renamed
2192 The virtual disk Check Consistency has made corrections and completed.



2193 The virtual disk reconfiguration has resumed.
2194 The virtual disk read policy has changed.
2199 The virtual disk cache policy has changed.

Voltage Sensor Events
1150 Voltage sensor has failed
1151 Voltage sensor value unknown
1152 Voltage sensor returned to a normal value
1153 Voltage sensor detected a warning value
1154 Voltage sensor detected a failure value
1155 Voltage sensor detected a non-recoverable value



Virtualization Management
Virtualization technology comes in several different forms. There is virtualization running as a software
application running on a host Operating System such as Microsoft’s Virtual Server 2005 or the virtualization
support included in Windows Server 2008. This approach has perceived disadvantages from a security
perspective as the attack service of the virtualization layer is a general purpose OS. Microsoft also offers
Hyper-V Server 2008 that strips the host OS to Windows Server Core, but still the footprint and the attack
surface is larger than an embedded hypervisor and once into the host OS, the guest OS’s can be
compromised. For the Microsoft virtualization solutions, the logs are all stored in the Applications and
Service Logs in the EventViewer of the host OS. EventTracker is able to collect all these logs through the
standard windows collection methods.
In the case of VMware the 2 hypervisors available are ESX and ESXi ESX is similar to the Hyper-V Server
2008 model, and is a bootable hypervisor. The Operating environment in the ESX case is a stripped down
Linux kernel. It is argued that it is more secure than a general purpose OS installation such as Server 2008 or
even Server Core as it is more stripped and it is Linux. ESXi on the other hand, represents the other popular
type of virtualization technique, and is usually embedded directly on the server hardware and operates more
like firmware than software like ESX or Hyper-V. ESXi is very small, and offers access only through defined
and limited APIs.
In larger installations, ESXi combined with a management application like Vcenter is emerging as the
preferred choice. As the hypervisors and the management application have been pared down, it is expected
that these are inherently more secure as the attack surface has been reduced. From a security perspective this
approach has a completely different management layer outside of the Operating System. Both the Hypervisor
and the management applications fortunately produce logs and these logs should be collected and stored in
the Log Management SIEM solution.
EventTracker is able to collect logs directly from the bare-bones hypervisors such as Vmware ESXi, or the
management application in the case of Vcenter, or from ESX. The following diagram shows the collection
architecture.



Hyper V Events
Hyper-V is made up of distinctive services and each service generates an exhaustive list of events. The
events follow the general Microsoft approach to logging – log it all and log it in great detail. These events,
when normalized, provide a complete picture of what has occurred and when it occurred. Combine these with
login information provided by AD and you have a complete who/what/when picture of both manual and
automated changes in the virtual environment.
Hyper-V Hypervisor
1 Hyper-V successfully started.
5 Hyper-V launch aborted due to auto-launch being disabled in the registry.
6 Hyper-V failed Code Integrity check.
7 Hypervisor traces are corrupted
Hyper-V launch failed. The registry key could not be opened by the Hyper-V boot
17 driver
18 Hyper-V launch failed. Registry value could not be read
19 Hyper-V launch failed; the registry value %2 of key %1 is not a string.
20 Hyper-V launch failed; sleep and hibernate could not be disabled (status %1).
26 Hyper-V launch failed. Hyper-V boot loader's internal logic failed
Hyper-V launch failed; the Hyper-V boot loader was unable to allocate sufficient
27 resources to perform the launch.
Hyper-V launch failed. The Hyper-V boot loader does not support the vendor of at
28 least one of the processors in the system.
Hyper-V launch failed. Processor does not appear to support the features required by
29 Hyper-V
30 Hyper-V launch failed. The system's combination of processors is not supported.
Hyper-V launch failed. The system does not appear to have a sufficient level of ACPI
31 support to launch Hyper-V.
Hyper-V launch failed. At least one of the processors in the system does not appear to
32 provide a virtualization platform supported by Hyper-V.



33 Hyper-V launch failed. the Hyper-V image could not be accessed
34 Hyper-V launch failed. Hyper-V image could not be loaded
35 Hyper-V launch failed. The Hyper-V image could not be read
36 Hyper-V launch failed; the Hyper-V image failed code integrity checks
Hyper-V launch failed. The Hyper-V image does not contain the Hyper-V image
37 description data structures
Hyper-V launch failed. At least one of the processors in the system was unable to
38 launch Hyper-V
40 Hyper-V launch failed The Hyper-V image is not the correct revision
41 Hyper-V launch failed. Either VMX not present or not enabled in BIOS.
42 Hyper-V launch failed. Either SVM not present or not enabled in BIOS.
43 Hyper-V launch failed. Hyper-V supported only on X64 architecture
Hyper-V launch failed. Either No Execute feature (NX) not present or not enabled in
44 BIOS
Hyper-V launch failed. At least one of the processors in the system is incompatible
45 with the others
Hyper-V launch failed. CPU does not support the minimum features required to run
46 Hyper-V
Hyper-V launch failed; Processor does not provide the features necessary to run
47 48 Hyper-V
49 Hyper-V launch failed. Feature mismatch
50 Hyper-V launch failed. Incompatible processor or leaf 4 cache topology
51 Hyper-V launch failed. Virtualization not supported or enabled on processor
52 Hyper-V launch failed. No-execute (NX) or DEP not enabled on processor
8451 Hyper-V failed creating a new partition
16641 Hyper-V successfully created a new partition
16642 Hyper-V successfully deleted a partition

Virtual Machine Management Service
2000 Could not register service connection point
2001 Could not unregister service connection point
10000 SID Mapping Error
10001 Failed to create NT VIRTUAL MACHINE security identifier mappings
10010 10011 The security identifier S-1-5-83 is already mapped to another domain.
10020 10021 Failed to create security identifier mapping
10030 10031 Failed to create security identifier mapping
10104 Failed to revert to VSS snapshot on one or more virtual hard disks of the VM
10107 Corrupt or invalid configuration files
11900 VM configuration section is corrupt
12242 Cannot mount the device read/write because the device is already mounted read-only
12243 Cannot mount the device
13000 User failed to create external configuration store
13001 Failed to create external configuration store at <location>
14030 14031 Failed to update the VM's saved state information
14040 14041 Failed to query domain information.
14050 Failed to register service principal name.
14060 14061 Failed to locate the default configuration store.
14062 14063 Failed to locate the default virtual hard disk directory
14072 Automatic restart has been disabled for VM because the VM stopped responding



repeatedly.
14073 VM stopped responding repeatedly.
14074 VM already running when the Hyper-V VM Management service started.
14080 14081 VM failed to automatically restart
14090 14091 Hyper-V VM Management service is shutting down while some VM's are running.
14092 14093 Service is shutting down.
14094 14095 Service started successfully.
14096 14097 Service failed to start.
14098 Required driver is not installed or is disabled.
14100 14101 Shutting down physical computer. Stopping/saving all VM’s .
14210 14211 Snapshot Operation failed to delete snapshot
14241 Cannot find the specified VM.
14270 VM unable to check user access rights
Failed to delete snapshot because it is specified as the automatic recovery snapshot for
14330 14331 VM
15010 15011 Failed to create new VM
15040 15041 Failed to import VM
15050 15051 Failed to export VM
15070 15071 Service failed to remove snapshot
15080 A new VM was added in a different location and the creation process never completed
15110 15111 Failed to modify service settings.
15120 15121 VM failed to initialize
15140 15141 VM failed to turn off
15150 15151 VM Save Operation failed
15170 15171 VM failed to pause
15180 15181 VM failed to resume
15190 15191 Snapshot Operation failed
15220 15221 VM failed to reset
15240 15241 VM failed to begin delayed startup
15300 Failed to access configuration store
15310 Created configuration store
15320 Failed to create configuration store
VM Bus (VMBus) cannot start because the physical computer's PCI chipset does not
15330 properly support Message Signaled Interrupts.
15340 The VM bus is not running.
15500 15501 VM failed to start worker process
16000 16001 VM Management service encountered an unexpected error
16020 VM encountered an unexpected error. The system cannot find the path specified.
16040 Cannot get information about available space for path
16060 16061 VM paused due to insufficient disk space
16090 16100 Worker Process validation failed
16110 An error occurred while waiting to start VM
16120 VM startup error
16140 VM cannot delete file
16150 Cannot delete directory
16160 Cannot delete snapshot file
16170 Cannot delete snapshot directory
16180 Service cannot update the snapshot list for deleted snapshot
16190 Service cannot update the parent for snapshot



16200 Service cannot update the instance of last applied snapshot
16330 Cannot load the snapshot configuration because it is corrupt
16370 Service cannot create the storage required for the snapshot
16371 Snapshot Operation failed
16430 Service timed out waiting for the worker process to exit
17010 Service assigned to an invalid authorization scope
17030 A VM is assigned to an authorization scope that is not defined in the policy store
17040 The authorization store could not be initialized
17050 Failed to initialize application in the current authorization store
17080 Updated the content of the authorization store successfully.
Content of the authorization store could not be updated from the store persistent
17090 location
17100 Cannot open authorization store
18002 18003 Cannot take snapshot
18030 Import failed. Unable to create identifier while importing VM
18031 Import failed.
18080 18081 VM import failed
18160 Failed to get summary information for VM
18190 Worker process health is critical for VM
18200 Worker process health is now OK for VM
18240 18241 Unable to find virtual hard disk file
VM was reset because the guest operating system requested an operation that is not
18540 supported by Hyper-V
19000 19010 WMI namespace is not registered in the CIM repository.
19020 WMI provider has started.
19030 WMI provider failed to start
19040 WMI provider has shut down.
Failed to get saved state information for VM. It is assumed that the VM is in a saved
19060 19061 state
20100 20101 Failed to register the configuration for the VM
20102 20103 Failed to unregister the configuration for the VM
20104 20105 Failed to verify that the configuration is registered for the VM
20106 20107 Service did not find the VM
20108 20109 Failed to start the VM
20110 20111 Failed to shut down the VM
20112 20113 Service failed to forcibly shut down the VM
20114 20115 Service failed to verify the running state of the VM
20132 20133 Failed to delete the configuration for the VM
14250 14251 Cannot find the specified snapshot
14320 14321 Cannot delete snapshot
15060 15061 Failed to apply snapshot
15130 15131 VM failed to start
The worker process for VM failed to respond within the startup timeout period and
15510 15511 was restarted
16010 16011 Operation failed
16050 16051 VM is about to run out of disk space
16360 16361 Cannot access the folder where snapshots are stored
18040 18041 Unable to rename file or directory
18050 18051 Failed to stop the rename of the file or directory



18060 18061 Import failed
18100 18101 Failed to create export directory.
18110 18111 Failed to copy file during export
18120 18121 An unknown device failed to import
18160 18161 Failed to get summary information for VM
18550 18560 VM was reset because an unrecoverable error occurred on a virtual processor
VM failed to perform operation. The VM is not in a valid state to perform the
19050 19051 operation.

Virtual Hard Drive Management Service
12140 Failed to open attachment
12141 File extension is invalid
15050 The system successfully converted VHD
15051 The system successfully created VHD
15052 The Hyper-V Image Management Service started.
15053 The system is expanding VHD
Device mount failed. The device is already mounted read-only, and an attempt was
15000 15001 made to mount it read/write
15100 Filename is invalid
15101 Failed to open attachment
15102 Invalid file extension
15103 The system is compacting VHD
15104 The system is merging VHD
15105 The system is converting VHD
15106 The system successfully compacted VHD
15107 The system successfully merged VHD
15108 The system mounted VHD
15109 The system successfully expanded VHD
15110 Invalid VHD
Invalid file name. You cannot use the following names (LPTn, COMn, PRN, AUX,
15111 NUL, CON) as they are reserved by Windows.
15200 The Hyper-V Image Management Service stopped.
15201 The Hyper-V Image Management Service failed to start
15202 The system successfully un-mounted VHD
12242 12243 The system is creating VHD

Hyper-V High-Availability Service
21100 Missing or invalid VM ID resource property
21101 Missing or invalid VmStoreRoot resource property
21102 21203 VM failed to register
21103 21104
21502 VM failed to unregister
21105 VM configuration update failed
21106 VM failed to initiate startup
21107 VM failed to initiate shutdown
21108 VM failed to start
21109 21110 VM failed to terminate
21117 Virtual network switch port settings creation failed.



21118 VM update settings failed
21119 VM successfully started
21120 VM successfully registered
21200 System not found
21201 Missing or invalid VM ID resource property
21202 Virtual network switch port already exists

Hyper-V Config
Configuration no longer accessible. The system cannot find the path specified or
4096 configuration is deleted.
4097 Configuration no longer accessible.
4098 Configuration is now accessible.

Hyper-V SynthStore
Failed to mount device. The device is already mounted read-only, and an attempt was
12242 12243 made to mount it read/write.

Hyper-V-Network
14000 Switch created
14002 Switch deleted
14004 Switch port created.
14006 Switch port deleted
14008 Switch port connected
14010 Switch port disconnected
14012 Internal miniport created
14014 Internal miniport deleted
14016 External ethernet port bound
14018 External ethernet port unbound
14020 Switch set up
14022 Switch torn down
14050 Switch create failed
14052 Switch delete failed
14054 Switch port create failed
14056 Switch port delete failed
14058 Switch port connect failed
14060 Switch port disconnect failed
14062 Switch port create failed
14064 Switch port delete failed
14066 Ethernet port bind failed
14068 Ethernet port unbind failed
14070 Switch set up failed
14072 Switch tear down failed
14108 Unable to open handle to switch driver
14110 Network WMI provider service started successfully.
14112 Network WMI provider service failed to start
14116 Timed out trying to acquire network configuration lock
14118 Unable to initialize network configuration



Hyper-V Image Management Service
12140 12141 Failed to open attachment
15000 15001 Invalid virtual hard disk
15051 Invalid file extension
Invalid file extension. You cannot use the following names (LPTn, COMn, PRN,
15052 AUX, NUL, CON) as they are reserved by Windows.
15053 Invalid file name
15100 System is compacting Image
15101 The system successfully compacted Image
15102 The system is merging Image
15103 The system successfully merged Image
15104 The system is expanding Image
15105 The system successfully expanded Image
15106 The system is converting Image
15107 The system successfully converted Image
15108 The system mounted Image
15109 The system successfully un-mounted Image
15110 The system is creating Image
15111 The system successfully created Image
15200 Image Management service started.
15201 Image Management service stopped.
15202 Image Management service failed to start

Hyper-V Worker
3170 3171 Worker failed to initialize the virtual machine during reset
Worker failed to save, but ignored the error to allow the virtual machine to continue
3200 3201 shutdown
3210 3211 Worker failed to save RAM contents during a snapshot operation
3220 3221 Unable to save RAM contents
3230 3231 Unable to restore RAM contents
3240 3241 Unable to save RAM block
3250 3251 Unable to restore RAM block because of an unexpected block data size.
3260 3261 Unable to restore RAM because some RAM blocks are missing.
3270 3271 Unable to restore RAM because some RAM block data is corrupt.
3280 3281 Failed to initiate a snapshot operation
VM was shutdown as a result of a failure to resume execution during a snapshot
3284 3285 operation
VM was paused as a result of a failure to resume execution during a snapshot
3286 3287 operation
3290 3291 Unable to restore RAM and unable to create a restore buffer.
3310 3311 Failed to initialize restore operation
3320 3321 Failed to create memory contents file
3330 3331 Failed to access the snapshot folder.
3350 3351 Failed to create auto virtual hard disk
3360 3361 Unable to stop the virtual processors.



3370 3371 Unable to reset the virtual hard disk path as a result of a failure to create a snapshot
3432 3433 Could not set the processor affinity for the worker process
5110 Failed to start the worker process using the correct security context
11901 Configuration section is corrupt
RC Vista Ultimate SP1 x86 (Device 'Microsoft Synthetic Display Controller'): An
11902 unrecoverable internal error has occurred.
VM' Microsoft Emulated IDE Controller failed to power on with Error 'Incorrect
12010 function.'
RC Vista Ultimate SP1 x86 Microsoft Synthetic Video failed to pause with error
12070 'Catastrophic failure'
12200 12201 Virtual machine Out of Memory Error
Error while opening file during ethernet device startup. The Hyper-V Networking
12440 12441 Management service provider may not be installed
RC Vista Ultimate SP1 x86 device Microsoft Synthetic Display Controller
12540 experienced a protocol error indicative of a deep system problem.
15160 15161 Failed to restore virtual machine state.
17010 Hyper-V Service is assigned to an unsupported authorization scope
VM is assigned to an authorization scope that is currently not defined in the policy
17030 store. The VM will be reassigned to the default authorization scope
17040 The authorization store could not be initialized
17050 Failed to initialize application in the current authorization store
17080 The content of the authorization store has been updated
17090 The content of the authorization store could not be updated
18500 Virtual machine started successfully
18510 VM saved successfully
18520 Snapshot succeeded

VMware Events
VMware generates far fewer raw events than Hyper-V but the events tend to focus on the types of
information that security personnel would need to know and less on general day to day health and
status messages. The following is a list of events emitted by VMware and included in the
EventTracker Knowledge Pack. Items marked "predefined alert" are included in the KP tested against
VMware 3.x.

Virtual Center Events
Alarm created
Alarm removed
Datacenter created
Datacenter removed
Datacenter renamed
High resource usage alarm (predefined alert)
Host added to datacenter
Host removed from datacenter



Virtual Machine Management
Guest OS shutdown
VM resource allocation events
Guest OS state changed
VM resource configuration updated
Virtual machine cloned
Virtual machine created
Virtual machine powered on
Virtual machine registered
Virtual machine reconfigured
Virtual machine removed
Virtual machine renamed
Virtual machine reset
Virtual machine relocated
Virtual machine suspended
Virtual machine switched off
Virtual machine snapshot created
Virtual machine reverted

User Management
Successful user login
Failed user login (predefined alert)
User logout
User permission rule changed
User permission rule added
User permission rule removed
Task failed or canceled by user (predefined alert)

VI Client ( vSphere PowerCLI)
Remote console connected
Remote console disconnected



Conclusion
At its most basic, security management is about first “seeing” everything that is happening, and then
applying processes, tools and solutions that can help you make sense of all the information and make you
more secure. In IT, with each new added technology comes complexity – distributed systems, remote access,
the internet, virtualization all create significant new challenges for security teams. Virtualization is no
different.
Also the real security requirements i.e. what is most critical to monitor, are generally driven by corporate
structure, infrastructure and policy. Businesses have different technology vendors, different organizational
structures, different compliance mandates and rarely, if ever, does one size fit all or even more than one.
With EventTracker, the challenge of visibility is solved. EventTracker provides the most comprehensive
support for virtual environments of any vendor on the market. Having all the data collected dependably in
one place gives an organization the ability to become secure. This data is categorized and available for
advanced real-time analysis where events from all the different technology layers can be monitored. For
example, an enterprise critical application can be assigned to a virtual machine. Using Vmware’s Vmotion,
that virtual machine can be reassigned different hardware based on performance or availability measures.. It
becomes critical to know that if a disk error is being received from OpenManage that that disk is mapped to
that VM, and that VM is running the critical service. With centralized visibility all that becomes possible.
Plus descriptions on all events are available on the EventTracker Knowledgebase, so security personnel don’t
have to worry about understanding hundreds of new events.
From there, with an understanding of the organizational structure and policies, rules can be quickly setup to
alert on violations of policy. For compliance, auditing is easily facilitated and no trusted user is able to effect
change in the enterprise without at least a record being created. Security starts from visibility – not only the
simple ability to see it, but understand it and make sense of it.



About EventTracker
EventTracker is a scalable, enterprise-class Security Information and Event Management (SIEM) solution for
Windows systems, Syslog/Syslog NG (UNIX and many networking devices), SNMP V1/2, legacy systems,
applications and databases. EventTracker enables “defense in depth”, where log data is automatically
collected, correlated and analyzed from the perimeter security devices down to the applications and
databases. To prevent security breaches, Event Log data becomes most useful when interpreted in near real
time and in context. Context is vitally important because often the critical indications of impending problems
and security violations can only be learned by watching patterns of events across multiple systems. Complex
rules can be run on the event stream to detect signs of such a breach. EventTracker also provides real-time
alerting capability in the form of an email, page or SNMP message to proactively alert security personnel to
an impending security breach.

The original log data is also securely stored in a highly compressed event repository for compliance purposes
and later forensic analysis. For compliance, EventTracker provides a powerful reporting interface, scheduled
or on-demand report generation, automated compliance workflows that prove to auditors that reports are
being reviewed and many other features. With pre-built auditor grade reports included for most of the
compliance standards (FISMA, HIPAA, SOX, GLBA, PCI, and more); EventTracker represents a
compliance solution that is second to none. EventTracker also provides advanced forensic capability where
all the stored logs can be quickly searched through a powerful Google-like search interface to perform quick
problem determination.

EventTracker lets users completely meet the logging requirements specified in NIST SP 800-92 Guide To
Computer Security Log Management, and additionally provides Host Based Intrusion Detection , Change
Monitoring and USB activity tracking on Windows systems, all in an off the shelf, affordable, software
solution.

EventTracker provides the following benefits

• A highly scalable, component-based architecture that consolidates all Windows, SNMP V1/V2,
legacy platforms, Syslog received from routers, switches, firewalls, critical UNIX servers (Red
Hat Linux, Solaris, AIX etc), Solaris BSM, workstations and various other SYSLOG generating
devices.

• Automated archival mechanism that stores activities over an extended period to meet auditing
requirements. The complete log is stored in a highly compressed (>90%), secured (Sealed with
SHA-1 checksum) archive that is limited only by the amount of available disk storage.

• Real-time monitoring and parsing of all logs to analyze user activities such as logon failures and
failed attempts to access restricted information.

• Alerting interface that generates custom alert actions via email, pager, console message, etc.



• Event correlation modules to constantly monitor for malicious hacking activity. In conjunction
with alerts, this is used to inform network security officers and security administrators in real
time. This helps minimize the impact of breaches.

• Various types of network activity reports, which can be scheduled or generated as required for
any investigation or meeting audit compliances.

• Host-based Intrusion Detection (HIDS).

• Role-based, secure event and reporting console for data analysis.

• Change Monitoring on Windows machines

• USB Tracking, including restricted use, insert/removal recording, and a complete audit trail of
all files copied to the removable device.

• Built-in compliance workflows to allow inspection and annotation of the generated reports.



About Prism Microsystems
Prism Microsystems, Inc. delivers business-critical solutions to consolidate, correlate and detect changes that
could impact the performance, availability and security of your IT infrastructure. With a proven history of
innovation and leadership, Prism provides easy-to-deploy products and solutions for integrated Security
Management, Change Management and Intrusion Detection. EventTracker, Prism’s market leading enterprise
log management solution, enables commercial enterprises, educational institutions and government
organizations to increase the security of their environments and reduce risk to their enterprise. Customers
span multiple sectors including financial, communications, scientific, healthcare, banking and consulting.
Prism Microsystems was formed in 1999 and is a privately held corporation with corporate headquarters in
the Baltimore-Washington high tech corridor. Research and development facilities are located in both
Maryland and India. These facilities have been independently appraised in accordance with the Software
Engineering Institute’s Appraisal Framework, and were deemed to meet the goals of SEI Level 3 for CMM.
For additional information, please visit http://www.prismmicrosys.com/.


Managing The Virtualized Enterprise New Technology, New Challenges

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (15)

Similar to Managing The Virtualized Enterprise New Technology, New Challenges

Similar to Managing The Virtualized Enterprise New Technology, New Challenges (20)

More from Enterprise Technology Management (ETM)

More from Enterprise Technology Management (ETM) (18)

Recently uploaded

Recently uploaded (20)

Managing The Virtualized Enterprise New Technology, New Challenges