This document discusses using Ansible to automate the migration of network policies from Cisco Tetration Analytics to a new Application Centric Infrastructure (ACI) fabric at World Wide Technology (WWT). Tetration provides visibility into application communication and generates network policies that can be extracted as variables for Ansible playbooks. This allows Ansible to programmatically configure the new ACI fabric with the migrated policies rather than manually creating them in the ACI GUI.

1. AnsibleDurham
17April 2019
Joel W. King Engineering and Innovations
Network Solutions
Enabling policy migration in the Data Center with Ansible

2. @joelwking
@joel_w_king
@programmablenetworks/
Enabling policy migration in the Data Center with Ansible
www.slideshare.net/joelwking/
www.meetup.com/Ansible-Durham/events/260264063/

3. At AnsibleFest Austin 2018, we demonstrated using Ansible to
extract policy from Cisco Tetration Analytics and expose it as
variables to playbooks.
The internal World Wide Technology IT department is
migrating from a traditional Nexus fabric to Application Centric
Infrastructure (ACI).
This talk describes how Ansible is used to migrate policy to,
and automate the configuration of, the new data center fabric.

4.  Background and context
… a bit of history on WWT involvement with Ansible network automation
 What is?
Application Centric Infrastructure
Tetration Analytics
 Workflow
Roles and Playbooks
 Source of Truth
 Data Optimization and Enrichment
 Summary / Resources

5. PROFESSIONAL SERVICES
WWT ACI University – Sept 2015
AnsibleFest
Brooklyn June 2015 ACI Modules added
Ansible version 2.4
December 2017
ansible.com/cisco
AnsibleFest
Austin June 2018
Cisco
Tetration
Analytics™
ADVANCED
TECHNOLOGY
CENTER
IT OPs
Q1 2019
CODE EXCHANGE

6. Fourth-generation programming language (4GL) for Python

7. #SILICONVALLEYINSTL

8. Application Centric Infrastructure
Application Policy Infrastructure Controller
 Cisco’s Software Defined Data Center Fabric
 Software and Hardware tightly coupled
Nexus 9000 family of switches
(APIC) manages and configures switches
 Viewed as one big (stateless) switch
 APIC is the central repository for all config / policies
 Emphasizing ACI as ‘Automation Centric Infrastructure’
 ‘ACI Anywhere’ - including public cloud.

9. Application Centric Infrastructure
Ansible modules for managing Cisco ACI fabrics
Network
Configuration
and
Policy
Multi-Site
Orchestrator (MSO)
3 VM Cluster
in-band
out-of-band
github.com/ansible/community/wiki/Network:-ACI

10. ---
apic:
username: maint
password: !vault | $ANSIBLE_VAULT;1.1;AES256 3836363862353137613630356
ansible-aci-credentials
All roles have been created using signature-based
authentication. Because automation can generate a
high volume of connections to the APIC, password-
based authentication will trigger the anti-DoS feature
introduced on ACI v3.1+, which will randomly generate
HTTP 503 errors, causing the playbook to fail.
/files/ passwords.yml

11. https://apic.example.net/api/mo/uni.json
universe
common small largeTenant(s)
GREEN BLUEVRF1
BD1
Private Networks
(aka VRF)
Bridge Domain
(Layer 2 )
Subnet
(aka VLAN Interface)
BD1 Servers BD1
198.51.100.1/24
192.0.2.1/24
192.0.2.1/24
198.51.100.1/24
198.51.100.1/24
 APIC is based on hierarchical object
model.
 Everything is represented as an object
 Every object can be manipulated by the
ACI_REST module.
• MOs are organized in a tree structure
called the Management Information
Tree
• MO are abstract representations of a
physical or logical entity that contain a
set of configurations and properties
Entire infrastructure operates as a single system . All infrastructure components are open for programmatic access. The GUI (APIC Web) uses same API interface as applications.

12. 23 REP0S
ACI MODULE
ARGS PRIOR #
COMMON TO ALL
ROLES
VARIABLE
NAMES FROM MIT

13.  Existing ACI modules
95 ACI and ACI MultiSite modules in v2.8
 ACI_REST
direct access to the APIC API,
 APIC API is natively idempotent
distributed system, cluster of controllers
 Config / Policy definition is a tree structure
50,000 +/- managed objects
parent, child, logical bindings
ACI FABRIC
https://github.com/ansible/community/wiki/Network:-ACI

14. #SILICONVALLEYINSTL

15. Automates generation
of whitelist policy
Enabling a Zero-trust model for
application segmentation
Tetration
Tetration Appliance
(Tetration, Tetration-M)
Tetration-V
Tetration-SaaS
Apache Hadoop
Distributed File System (HDFS)
Apache Kafka
stream-processing
Google Protocol Buffers
serializing structured data
FORM FACTORS
open-source software
TECHNOLOGY

16. Cisco Tetration
39RU dual-rack option
[ 1/2 large form factor ]
Data Collection Layer
Cisco
Tetration
Analytics™
NETWORKING
[TELEMETRY ONLY]
Data Consumption Layer
REST API
KAFKA MESSAGE BUS

17.  Policy available via
Web GUI
REST API
Network Policy Publisher
 Generate policy with Tetration
 Automate ACI config with Ansible
… I can’t imagine creating fabric policies
manually, using the ACI GUI is impossible …
- Lenny Ilyashov WWT

18. #SILICONVALLEYINSTL

19.  Data center consists of approximately 1,600 servers
 Network Centric with Policy
ACI implementation - hybrid between network centric and
application centric
 Environment based migration
 Default gateways remain on legacy
APRIL
ATC
SANDBOX
Master Data Center
and Hybrid Cloud
DATA CENTER ENVIRONMENTS
DEVELOPMENT TEST PRODUCTION
TETRATION
ADM
Q1 2020
GIT
it-automation-aci

20. Telemetry
Network
Configuration
Policy
Tetration
LEGACY NETWORK
VISIBILITY AGENT
ACI FABRIC
Network
Policy
Configuration
DATA MANIPULATION
DATA ENRICHMENT
/it-automation-aci/
├── ansible-aci-application-profile
├── ansible-aci_automation
├── ansible-aci-automation-package
├── ansible-aci-bridgedomain
├── ansible-aci-contract-filter
├── ansible-aci-credentials
.........
├── ansible-aci-epg
├── ansible-aci-include-data
├── ansible-aci-tenant
├── ansible-aci-vrf
UTILITY AND ACI ROLES

21. PAGE
2 OF 2
FABRIC ACCESS
POLICY CONFIGURATION
VIRTUAL
MACHINE
MANAGERS (VMM)
INTEGRATION

22. PAGE
1 OF 2
TENANT SPECIFIC
CONFIGURATION
TENANT POLICY
CONFIGURATION
TETRATION
POLICY
MIGRATION

23. Contract Subject
(vzSubj)
Contract (vzBrCP)
Filter
(vzFilter)
Filter Entry
(vzEntry)
Bind Filter to
Contract Subject
(vzRsSubjFiltAtt)
Tenant
(fvTenant)
End Point Group
(fvAEPg)
Application Profile
(fvAP)
Bind EPGs to Contracts
(fv:RsCons, fv:RsProv)

24.  Deployment
repos represent
‘customer’
deployments.
 Playbooks
consume ‘utility
roles’ plus one
or more ‘ansible-
aci’ roles from
the Git
Organization
roles/requirements.yml
aci_tenant_master.yml

25. #SILICONVALLEYINSTL

26. Configuration Policy

27. IPAM GLOBAL
CONFIG
YAML
ENCRYPTED
CREDENTIALS
YAML
PLAYBOOK
SERVICE
TICKETING
SYSTEM
VERSION CONTROL
SOURCE OF TRUTH
NETWORK
DEPLOYMENT
ENGINEER
https://github.wwt.com/it-automation-aci/ansible-aci-include-data
MongoDB
BOOTSTRAP
EXTRA_VARS
PRE-PROCESSING
POST-PROCESSING

28. Configuration
 Network Engineers like spreadsheets
 Free and readily available – no training
 YAML, JSON and XML confound
non-programmers
tabular structure to hierarchical

29. Various Sheets for Configuration Data
EPG
(End Point Group)
BD
(Bridge Domain)
Configuration

30. Policy
WEB GUI
REST API
NETWORK POLICY PUBLISHER

31. As we document, we capture things like this:
Tetration was a big unknown when we started this a few months ago. Our thoughts on it now:
We are attempting to use Tetration as the source of truth for policy because, coupled with the automation
tools the GET team has developed, it minimizes manual effort and gives us a head start on automating the
policy creation process.
 Using scopes and workspaces in unique ways that fit our needs gives us the ability to design our own
"WWT" migration methodology.
 The flow data that Tetration collects provides a level of visibility that gives us a good starting point
from which to create policy.
 Tetration gives us the ability to export policy in a format that can be manipulated (if needed) and then
imported into ACI programmatically.
o Adding policy manually into ACI at our scale is a nearly impossible task.
 It also provides an easy to consume view of traffic flow and cross-environment communication for the
purposes of analysis and planning our migration approach.
Best regards,
Lenny Ilyashov
Policy

32. aci-demo.sandbox.wwtatc.local : ok=3287 changed=720 unreachable=0 failed=0

33. Policy TASK [Get facts from CSV file]
************************************************************************************************
ok: [localhost] => {
"ansible_facts": {
"ENTRYs": [
{
"consumer_filter_name": "Default:WWT:EPG-EXT-LGC",
"contract_name": "EPG-EXT-LGC_to_EPG-SND-PRVLIN1",
"contract_subject_name": "SUBJ-EPG-EXT-LGC_to_EPG-SND-PRVLIN1",
"ether_type": "ip",
"ports_from": "27000",
"ports_to": "27000",
"proto": "tcp",
"provider_filter_name": "EPG-SND-PRVLIN1"
},
{
"consumer_filter_name": "EPG-SND-RAZOR",
"contract_name": "EPG-SND-RAZOR_to_EPG-EXT-PRD",
"contract_subject_name": "SUBJ-EPG-SND-RAZOR_to_EPG-EXT-PRD",
"ether_type": "ip",
"ports_from": "17472",
"ports_to": "17472",
"proto": "tcp",
"provider_filter_name": "Default:WWT:EPG-EXT-PRD"
},

34. UPDATE_ENDUPDATE_START

35. Policy
>>>
print(json.dumps(parsed['imdata']['ansible_facts']['inventory_filters'][0],indent=4))
{
"query": "type=and, filter=[{type=eq, field=vrf_id, value=1},{type=eq,
field=user_org, value=wwt},{type=eq, field=user_env, value=snd},type=or,
filter=[{type=eq, field=user_epg, value=EPG-SND-KUBE2},{type=subnet, field=ip,
value=10.5.31.0/24, subnet=10.5.31.0/24},],unsafeSubnetValue=null]",
"id": "5c74b382497d4f610afd2ccb",
"inventory_items": [
{
"end_ip_addr": "10.5.31.5",
"start_ip_addr": "10.5.31.1",
"addr_family": "IPv4"
},
{
"end_ip_addr": "10.5.31.102",
"start_ip_addr": "10.5.31.100",
"addr_family": "IPv4"
},
>>> print(json.dumps(parsed['imdata']['ansible_facts']['intents'][0:2],indent=4))
[
{
"dst_port_start": 161,
"dst_port_end": 161,
"ip_protocol": "udp",
"consumer_filter_id": "5c75720c755f02668c5d8073",
"action": "ALLOW",
"provider_filter_id": "5c8c013e497d4f3ecafd2d20",
"intent_id": "f2465ad8765cf10fdbd9f0ab7a945f5b"
},
{
"dst_port_start": 22,
"dst_port_end": 22,
"ip_protocol": "tcp",
"consumer_filter_id": "5c75718f497d4f0eb2fd2cd7",
"action": "ALLOW",
"provider_filter_id": "5c8c013e497d4f3ecafd2d20",
"intent_id": "456a23f0d3eca1ca6262319170b96ce5"
}
]

36. #SILICONVALLEYINSTL

37.  Configuration Data
Converting tabular data to a hierarchical structure
 Policy Data
Associating the generated policy (contract) with the
appropriate ACI managed objects
(Tenant, Application Profile, EPG) Tetration

38. ./csv_to_mongo.yml -e filename="/it-automation-aci/TEST_DATA/aci_constructs_policies_3.csv" --skip-tags "mongo"
tasks:
- name: Get facts from CSV file
csv_to_facts:
src: '{{ filename }}'
table: spreadsheet
vsheets:
- VRFs:
- VRF
- Tenant
- TENANTs:
- Tenant
- APs:
- Tenant
- AppProfile
- debug:
msg: '{{ TENANTs }}'
TASK [debug]
******************************************************************************************************************
ok: [localhost] => {}
MSG:
[{u'Tenant': u'WWT-INT'}, {u'Tenant': u'WWT-DMZ'}, {u'Tenant': u'WWT_NULL'}]
UNIQUE (SET) OF
TENANTS
PRESENT IN THE
SPREADSHEET
CSV FILE FROM
EXCEL
VIRTUAL SPREADSHEET
https://github.com/joelwking/ansible-nxapi/blob/master/csv_to_facts.py

39. I want to be able to parse two
datasets to optimize the creation
and deletion of objects. For the
creation of new objects, I only
want to send the call to create
the object for objects that do not
already exist.
For the deletion, I want to delete
objects that are no longer
required.

41.  Most Network Engineers are not familiar with data serialization
formats – will need good examples | documentation
 Don’t clutter playbooks with conditionals validating data format
 While ACI is idempotent, optimize data for efficiency, memory,
total execution time
 Create a process flow to identify sources and sinks of data and
tasks within playbooks

42. #SILICONVALLEYINSTL

43.  Team approach
Software repository | Ansible roles are a means to an end
Requires a team of engineers with complementary skills
 Data Analysis: Managing and manipulating data
Generating fabric policies are iterative,
time consuming
 Automation is a requirement, not an option.
… I can’t imagine creating fabric policies manually, using the
ACI GUI is impossible … - Lenny Ilyashov WWT

44. #SILICONVALLEYINSTL

45.  AnsibleFest 2018: Using Ansible Tower to implement security policies and
telemetry streaming for hybrid clouds
https://www.ansible.com/using-ansible-tower-to-implement-security-policies-
telemetry-streaming
 Cisco Code Exchange
https://developer.cisco.com/codeexchange/#search=tetration
 Using Tetration for Application Security and Policy Enforcement
https://blogs.cisco.com/developer/tetration-for-security
 Coders and developers: The new heroes of the network?
https://www.computerweekly.com/news/252457087/Coders-and-developers-the-
new-heroes-of-the-network
 Analytics for Application Security and Policy Enforcement in Cloud Managed
Networks
https://developer.cisco.com/devnetcreate/2019/agenda