Summit 16: Open-O Mini-Summit - VF Event Streaming Project Proposal, Alok Gupta

1. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
VF (Virtual Functions) Event Streaming (VES) Project
Proposal
June 13th, 2016
For Questions, Please contact:
Alok Gupta
+1 732-420-7007; ag1367@att.com
1

2. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
VES Project Proposal
2
Objective
Enable significant reduction in effort to develop and integrate VNF telemetry-
related data into automated VNF management systems, by promoting
convergence to a common event stream format and collection system.
 Background
 Telemetry data formats and semantics / expected actions by management systems vary widely
 For Fault Events, vendors use SNMP, 3GPP Corba, MTOSI, OSSJ etc, and semantics can differ (e.g. Critical Severity as
“1” or “5”)
 For Measurement events (KPI/KCI), vendors deliver CSV or XML based files, with varying internal data formats.
 This variance complicates NF integration into management systems
 Scope/Deliverables
 A VNF Event Stream (VES) Common Event Data Model
 Consisting of a Header and an Event Specific Block, with additional Name/Value fields for extensibility
 OPNFV support for the VES Common Event Data Model
 A VES Agent that can collect the VNF Event Stream data from the VNF and deliver it to the VES Collector
 A VES Collector that can consume the VNF Event Stream data, and store the data in a database backend
 VES plugins for integration with OpenStack infrastructure services such as Monasca and Vitrage
 Documentation and test support (e.g. reference event data)
 A plan for upstreaming the project code

3. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.3
Problem Statement / Fault Management (FM) Interface Comparison
Categories
SNMP 3GPP Corba MTOSI via JMS
MTOSI via REST over
HTTPS (JSON)
GeneralAttributes
- There are 3 current version of SNMP.
- Devices are not compliantin all 3 versions.
- SNMP V3 handling requires special adapter
due to increased security
- SNMP is connectionless, except for SNMPv3
- No assurance of delivery
- No resynch functionality
- Corba is connection oriented
- assurance of delivery
- resynch functionality
- Corba has sets of actions
- Corba payload provides more
details in less traffic volume
- JMS is Connection oriented
- assurance of delivery
- resynch functionality
- MOTSI has a sets of actions
- MTOSI payload provides more
details in less traffic volume
-service providing
organizedaccess to
stateless resources
-connectionless
-stateless
Availability Agents and processes are widely available ReasonableSupport Mixed Mixed
Reuseability Low High High High
Exapndability Low High High High
Authenticaltion-Security SNMP V3 over SSL High-SecurityIRP High - SSL High – SSL
Platform Dependency Dependent Dependent Independent Independent
SOA Interface and Data event No Yes Yes Yes
Implementation Vendor Dependednt Vendor Independent Vendor Independent Vendor Independent
CLI Command Standard Standard Standard Standard
Payload High Constant constant Constant
MultipleDevice Support No Yes Yes Yes
Multi-VendorSupport No Yes Yes Yes
Sensititivityto new Alarm Data High Low Low Low
point-to-Point/Multipoint
p-t-p p-t-p
deploy muiltplepub-sub to avoid
bottleneck Pub-Sub
latency <1ms <10ms <10ms
Server/Code deployment Difficult Java driven Independent
interfacingto diffenet Apps Difficult Easy Easy
Output SNMP 3GPP XML/JSON XML/JSON
Cost of Wonership
adapter 49 7 2
line of codes 500000 6500 800
EquipmentType Supported 50 20 5

4. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
 Common Event Data Model: Fault, Heartbeat, Performance, Syslogs, Other, Usage, Signaling,
Security, others
 Common Header and Domain Specific Event
 Additional name value pair fields for extensibility
 Profile Driven Event Data Needed at time of VNF Instantiated and Data Profile Support
to change event streaming by VNF and Collector/Listener:
 Collector Address: IP, Port
 Authentication Data: User Name/Password
 Data Profile: Event Type (Fault (Custom (Full, c1, c2..) or Programmable))
VES – Common Event Data Model - Dynamic
Pre-Defined:
No API
Overhead
Defined Real
Time via API
Header
Domain
VNF Attribute
Timing
Fault
Severity
Source
Specific Problem..
Name/Vlaue
Measurement
KPI
KCI
VF Scaling…....
Name/Vlaue
SysLogs
Facility
Data
Proc…...
Name/Vlaue
MobileFlow
RptEndPt
OthEndPt
Flow…...
Name/Vlaue
Other
Name/Vlaue
Usage, Config...
…..
Other: Signaling,
Internal ...TBD
Name/Vlaue
4

5. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
In scope:
• Green/solid interfaces
• Agent/Collector interface
support library and
reference implementation
• OpenStack component
plugins
Agent code is embedded in
VNFCs, Virtualization Hosts,
and Bare Metal Hosts
Agents / VNFCs can also
deliver directly to OpenStack
Plugins (not shown)
VES Architectural View
OpenStack
Application Controller
SDNC
Bare Metal Host
Virtualization Host(s)
VES Collector
VES Agent VNFC VES Agent
Plugins
Monasca
Vitrage
Congress
etc
Collector can
integrate with
other OpenStack
Collectors
Other Management Functions
Message Bus
Direct (REST)
5

6. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
VES and Other OPNFV Projects
 VES is a standardized event framework that can be used by Open Stack, OPNFV
projects
 OPNFV projects that potentially benefit from the VES project
 Fault Management (Doctor)
 Virtualized Infrastructure Deployment Policies (Copper)
 High Availability for OPNFV (Availability)
 Data Collection for Failure Prediction (Prediction)
 Audit (Inspector)
 Fault localization (RCA, Pinpoint )
 Service Function Chaining (sfc)
 Moon Security Management
 OpenStack projects that potentially benefit from the VES project
 Monasca: a monitoring-agent plugin for VES can provide a new/converged source for Monasca events
 Vitrage: similar to Monasca can provide new data for the Graph DB of Vitrage
 Congress: Congress data source drivers can integrate additional data from Monasca, Vitrage, or directly
6

7. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
1.Use cases that illustrate the value, reuse and increased efficiency/cost reduction
2.Software code behind:
 Application Interface Document
 Library Capabilities
 Demo
 Reference VNF
 Event Library Integration
AT&T’s Operational Experience using the Proposed Framework
7

8. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
 Use Case
 Results
 Once development is done for the first VNF, the subsequent VNF can be supported via Configuration changes
and/or Policy Identification and Enforcement, resulting in over 80% cost and time savings
Use Case and Results
Collector
Event
Processor
Correlation
Policy
Enforcement/
Automation
Data Bus
Fault Event /
Clear
Ticketing System
Scenario 3: High CPU Utilization
Published an event for service orchestrator to
create a New VF
Scenario 2: Payload Processor Failure
Publish an event to Restart VM
Scenario 1: False Alarm
Scenario 2: VM Failure
Scenario 3: Scaling Alert Scenario 1:
Failure Event, Notification
PM
(Threshold
Crossing Alert)
Ticket Generated, Updated and Closed
8

9. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
Thank You
© 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
9

10. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
 Hosted in the AT&T Research github site: http://github.com/att
• evel-library
• evel-test-collector
• evel-reporting-reference-vnf
 To be contributed to the VES project under the Apache 2.0 license
ECOMP Vendor Event Listener code on Github
10

11. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
 Purpose
 A test harness to play the role of the ECOMP Vendor Event Listener in environments where access to
ECOMP isn’t available
 Capabilities
 Catches JSON events
 Checks the events are valid against the defined schema
 Checks the authentication credentials provided
 “Pretty prints” the event for desk-checking
 Use-cases
 Testing VNFs during development by vendors
 Familiarization with the API
 Packaging
 GitHub repository with detailed documentation
 Source code tree for development, running in Windows environment etc (Python)
VES Collector
11

12. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
 Purpose
 A simple VNF which provides a Web UI to a local database of event definitions
 Capabilities
 The user can define arbitrarily complex events
 The user can generate single-shot and repeated (scheduled) events
 Use-cases
 Testing ECOMP
 Familiarization with API
 Mocking-up events as part of designing a new VNF
 Packaging
 GitHub repository with detailed documentation
 Source code tree for development (Python, Django, SQLite)
Reference VF
12

13. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
Reference VF
13

14. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
 Latest Python 2.7.x
 https://www.python.org/downloads/
 Check the option “Add Python to path”
 But you may have to edit the Path environment variable to “Apply” the changes – or just reboot!
 Django
 pip install Django
 JSON Schema
 pip install jsonschema
 Initialize the VNF database for first use
 …go-scriptsinitialize-webserver.bat
 Start the app
 Test collector …go-scriptsgo-collector.bat
 Backend …go-scriptsgo-backend.bat
 Webserver …go-scriptsgo-webserver.bat
Ref VF Event Demo
14

15. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
 Point your favorite web-browser at:
 127.0.0.1:8000
 Generate an event
 Fault
 Measurement
 Mobile Flow
 Syslog
 Check that the Test Collector reports its safe receipt…
 …and try to work out why not if it doesn’t!
Ref VF Event Demo – Contd.
15

16. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
 Purpose
 A C library to simplify adding ECOMP event reporting to VNFs
 Capabilities
 Handles all the JSON encoding
 Handles all the HTTP(S) transport
 Provides an asynchronous API to the VNF
 Generates syslog events for delivery failures etc
 Use-cases
 VNF development
 Familiarization with the API if implementing an independent event-reporting mechanism
 Packaging
 Source code tree for development
 gcc toolchain
VF Event Library
16

17. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
 Handles all the JSON encoding
 Trivial JSON encoder “hand-built” into library
 Offloads “non-application” fields (e.g. VM Name and UUID) from the VNF application
code
 Optional fields are omitted where possible
 Handles all the HTTP(S) transport
 Uses cURL library - https://curl.haxx.se/libcurl/
 Provides an asynchronous API to the VNF
 Implements a ring-buffer between the application API and the HTTP client to the JSON API
 Offloads waiting for responses
 Generates syslog events for delivery failures etc
 Last-ditch record of what was going on for post-mortem analysis
VF Event Library Capabilities
17

18. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
 Initialize
 Per client process
 Must be single-threaded at this point
 Generate events
 Create an event (Measurement, Fault or Report)
 Provide mandatory scalar event parameters to factory function
 Add parameters
 Optional scalars
 Mandatory vectors
 Hand-off to the library
 Application code relinquishes all responsibility (e.g. freeing memory) at this point
 Shutdown
 Swift and brutal!
 The ring-buffer is not depleted
 Cleans up all memory & the cURL library
VF Event Library Usage
18

19. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
VF Event Library Initialize
19
/***************************************************************************/
/* Initialize the EVEL interface. */
/***************************************************************************/
if (evel_initialize(api_fqdn,
api_port,
api_path,
api_topic,
api_secure,
"Alice",
"This isn't very secure!",
EVEL_SOURCE_VIRTUAL_MACHINE,
"EVEL demo client",
verbose_mode))
{
fprintf(stderr, "Failed to initialize the EVEL library!!!");
exit(-1);
}
else
{
EVEL_INFO("Initialization completed");
}

20. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
VF Event Library Create Event
20
/***************************************************************************/
/* Create a new Fault event and set optional parameters. */
/***************************************************************************/
fault = evel_new_fault("My alarm condition",
"It broke very badly",
EVEL_PRIORITY_NORMAL,
EVEL_SEVERITY_MAJOR);
if (fault != NULL)
{
evel_fault_type_set(fault, "Bad things happen...");
evel_fault_interface_set(fault, "My Interface Card");
evel_fault_addl_info_add(fault, "name1", "value1");
evel_fault_addl_info_add(fault, "name2", "value2");
/***********************************************************************/
/* Post the event to the Vendor Event Listener. */
/***********************************************************************/
evel_rc = evel_post_event((EVENT_HEADER *)fault);
if (evel_rc != EVEL_SUCCESS)
{
EVEL_ERROR("Post failed %d (%s)", evel_rc, evel_error_string());
}
}
else
{
EVEL_ERROR("New fault failed");
}

21. © 2016 AT&T Intellectual Property. All rights reserved. AT&T and the AT&T logo are trademarks of AT&T Intellectual Property.
VF Event Library Termination
21
/***************************************************************************/
/* Shutdown the library. */
/***************************************************************************/
evel_terminate();