NIEM and XML for Architects and Developers

<Insert Picture Here>
Delivering NIEM-aligned data services
David Webber, Information Architect, Oracle Public Sector
NIEM and XML
for Architects
and Developers

The following is intended to outline Oracle general
product direction. It is intended for information
purposes only, and may not be incorporated into any
contract. It is not a commitment to deliver any
material, code, or functionality, and should not be
relied upon in making purchasing decisions.
The development, release, and timing of any features
or functionality described for Oracle’s products
remains at the sole discretion of Oracle.
Disclaimer Notice
Slide 2

© 2012/13 Oracle Corporation – Restricted Distribution 3
Agenda
• Background - What are the business drivers and needs?
• NIEM today – Technical Introduction
• The Delivery Life Cycle – Lessons Learned
• Software Delivery - Anatomy of a NIEM “IEPD”
• Tooling to the Rescue!
• Key Pillar of Dictionaries
• Summary

BACKGROUND
Business Drivers and Needs

XML is everywhere…
• PMIX – Prescription Drug information sharing
• Elections Results Reporting
• Pharmacy and Vaccines locating
• COBie Buildings Information sharing
• PESC – Student Post-Secondary Education
• City 311 Services
• Vehicle spare parts inventory
• Public Safety information sharing
• Healthcare services delivery coordination
• eDevice control templates

Public Sector Challenges
• Cost of building information exchanges
– Reduction of development sunk costs through minimizing coding tasks
– Allowing rapid prototyping and proof of concepts (agile development)
– Flexibility - dynamic adaptable templates instead of rigid fixed code
– Reuse – templates for common systems / solutions easily shared and adapted
• Government transparency and public information sharing
– Structured content from data stores – e.g. election results reporting, monthly cost
reports, scheduled road maintenance; obvious metadata and semantics
– Simple search requests on data – e.g. part numbers, codes, licenses lookup
– Not intended for unstructured and textual content document delivery
• Practical real world data sharing anywhere
– Direct simple tools with short learning curve and plug and play deployment

Technology Capability Requirements
• Delivering on the Open Data Digital Strategy Vision for government
• Open Standards and Open Source based
• Plug and play with code-free templates and rapid development
• Supports NIEM and Open XML exchanges
• Leverages deployed technology today
• Workflow integration
• Secure messaging delivery and partner management
• Delivered across-platform and device
• End-to-end security & governance

NIEM TODAY
Technical Introduction

The NIEM Framework
NIEM connects communities of people who share a common need to
exchange information in order to advance their missions, and provides a
foundation for seamless information exchange between
federal, state, local, and tribal agencies. Much more than a data
model, NIEM offers an active user community as well as a technical and
support framework.
Support FrameworkTechnical FrameworkCommunity
Formal Governance
Processes
Online Repositories
Mission-Oriented Domains
Self-Managing
Domain Stewards
Data Model
XML Design Rules
Development Methodology
Predefined
Deliverables (IEPD)
Tools for Development
and Discovery
Established
Training Program
Implementation Support
Help Desk &
Knowledge Center

The NIEM Data Model
NIEM’s data model is a set of common, controlled, and approved XML
data structures and definitions vetted through the
Federal, State, Local, Tribal and Private Sectors.
Data elements are organized into core and
domain-specific components
Core components are
used by multiple
domains and can be
described by
structure, semantics, a
nd definition
universally
Domain-specific
components are
continually updated by
subject matter experts
that are actual NIEM
participants and
industry experts for
their particular domain
NIEM Naming and
Design Rules (NDR)
specify how each of
these components are
defined and utilized

NIEM’s Domains View
NEWEST DOMAINS
NIEM Government Resource
Management Domain
NIEM Health Domain
NIEM Human Services Domain
DOMAINS IN PLANNING
NIEM Agriculture
NIEM Education
NIEM Transportation
11

Available XML Dictionaries
• NIEM 2.1 dictionaries
– CBRN dictionary
– Emergency dictionary
– Family dictionary
– Immigration dictionary
– Infrastructure dictionary
– Intelligence dictionary
– Justice dictionary
– Maritime dictionary
– Screening dictionary
– Trade dictionary
– Immigration blueprint
– NIEM core dictionary
Available from download site
direct link:
http://sourceforge.net/projects/camprocessor/files
XML XML XML XML XML XML
+ includes spreadsheets and sample models
Note: Those marked in bold are model style dictionaries with recursive components.

NIEM uses W3C Schema Collection
XSD Collection
*Code list xsd
Namespace* Optional
XML Schema
xsd subset *Constraint xsd
*Extension xsd
*Restriction xsd
Industry Domain
schemas
Code list xsd
LEGEND:
Exchange xsd
NIEM
Exchange
Schema

NIEM Past and Present Challenges
• Vertical domain vocabulary for DHS and DOJ
• One dozen domain dictionaries
• Technology limited - built using XSD schema
• Core components highly contextual to DHS/DOJ
• Surprisingly – no actual logical models of information!
• Gap with semantic technologies integration
• Limited data content rules and code lists
• Mapping automation support missing
• Multi-year development life cycles

NIEM is still improving…
• Collections of complex XML Schema
• Verbose components
• Embedded context in names
• Currently facing significant scaling challenges
– Inconsistencies; too much manual management; slow lifecycles
• Dictionary technology incubating
• Enhanced code lists mechanism incubating
• UML profile is evolving initiative with OMG
– Attempt to marry modelling techniques and XSD Schema syntax
• Policy and Security mechanisms incubating
• Need to embrace enabling tools for rapid expansion of
domains and community collaboration

DELIVERY LIFECYCLE
How to build NIEM solutions
Lessons learned
Foundation Pillars

The 8 “D”s and NIEM
• Design
• Develop
• Deploy
• Document
• Dictionaries
• Discovery
• Differentiate
• Diagnose Repeatable, Reusable Process
(Exchange Specification Lifecycle)
NIEM IEPD Process
*IEPD - Information Exchange Package Documentation

©2012 Oracle Corporation 18
NIEM IEPD
Reports
Dictionaries
Discovery
NIEM IEPD / Exchange Delivery Lifecycle
Diagnose
Differentiate
Design
Develop
Validated
Templates /
Schema
XSD
Schema
XML
Samples
XMI / UML
Models
Production
Results
Requirements
Updates
Document
ation
Drag and Drop
Visual
Designer
DocumentDeploy
Exchange
Templates 1
2
3
4
5
Required
IEPD artifacts

Integrating into your existing processes and control
• Critical to manage and steer use of NIEM
• Ensure business outcomes match requirements
• Lessons learned with NIEM
• The earlier in the software development process that NIEM is
introduced the better the overall outcomes
• Each step has deliverables – business plan, delivery
architecture, data model, alignment reports, schema, test results
that can be monitored in your SDLC*
• Monitoring products reinforces reuse as best practice
• Tools essential in review processes
• Collaboration tools can coordinate artifacts across
projects and participants
• Test bed tools and open APIs reduce integration
alignment costs *SDLC – Software Development Life Cycle

SOFTWARE DELIVERY
Anatomy of a NIEM “IEPD”
It is not just all about the XSD schema

IEPD Components & Requirements
<Exchange_Schema/>
<Extension_Schema/>
<Subset_Schema/>
IEPD IEM
IEPD MPD
NIEM Core
Schema(s)
Domain
Schema(s)
Main Document
Catalog
Change Log
Sample XML
Instance
In order to be NIEM-conformant, the IEPD must adhere to:
1. NIEM Conformance Document
2. NIEM Naming and Design Rules (NDR) v1.3
3. NIEM Model Package Description (MPD) Specification v1.0

Formal NIEM IEPD Packaging Details
IEPD Artefact Description
Exchange Files
Subset Schema (SSGT) Subset of the full NIEM schema—a compressed directory of schemas to distinguish from other schema sets.
Wantlist
User requirements -distinguishes user-required data components from components they depend on for conformance. Generated by and up-loaded to the
Schema Subset Generation Tool (SSGT). This is an open specification and the SSGT is not required to create a wantlist, though it is easier.
Exchange Schema
Base document schema that defines the XML root element and is generally named after the IEPD itself. Also known as the document schema, reference
schema, or root schema.
Constraint Schema Constraints for separate constraint validation path—a compressed directory of schemas to distinguish from other schema sets.
Extension Schema Specification for extended components—separate local name-spaces of components not contained in NIEM.
Sample XML Instance Example instance(s) – may reference optional stylesheet.
Stylesheet (generic) Example stylesheet for display of instance(s).
Documentation
Master Documentation
The Master Document is the main document for which all of the context and details around the exchange are explained. This document includes, the
overview, as well as details surrounding the exchange, business drivers and requirements
Exchange model XMI
Exchange model in standard open format (xmi, vsd, zargo) and standard open graphic (jpg, pdf, etc.) preferably a Unified Modeling Language (UML)
model.
Business Rules
Business rules in one of the following formats: (1) plain or structured English, (2) written into master documentation, (3) Schematron or other formal
business rule language, or (4) generated by a development tool.
Mapping to NIEM Mapping of domain components, tagged with constraints (i.e., cardinality, etc.) to NIEM components; preferably the Component Mapping Tool (CMT).
Extended Components
Components created because they were not in NIEM—may be part of mapping spreadsheet and include structure and definitions of new components. Prefer
CMT.
Change log Record of cumulative changes from previous IEPD versions. The initial IEPD simple records its creation date.
Catalog
Catalog XML file A machine-readable list of artifacts in the IEPD in an open, portable and browser-displayable format (simple manual edit).
Metadata XML file All metadata of owner and domain to be registered with the IEPD (simple manual edit).
Generated
using CAM
Never Used
Word
Document
Manual
Entry

XML Technology Components
NIEM: need to manage
complete aspects of exchange
delivery, not just the structure
and content data model

Swim Lane View
UML Technologies XML TechnologiesW3C Schema
This is what NIEM exchanges
are delivered with on-the-wire!
UML
XMI
XSD Namespace
Enumerations
XSLT
XQuery
XPath
XML
Patterns
Associations
Cardinality
Stereotypes
Inheritance
Facets
Elements
Attributes
Content
Complex Types
Audience:
Data Modellers / Designers
Audience:
Software Developers and
Middleware Engineers
Audience:
General content delivery
and applications; content
engineers; rules
representation systems;
multimedia delivery;
geospatial systems
Objects
Web
HTML
JavaScript
JSON
REST
SQL
SOAP
Audience:
Web
application
developers
Open-XDX
24

TOOLING TO THE RESCUE
Accelerating and Improving the NIEM process

IEPD Delivery Tasks / Tools Matrix
CAM CAMV
Open-XDX
(OPX)
Exchange Schema Development X
Canonical Model Development / Generation X
Exchange Schema Generation X
NIEM Compliance Checking / Reporting X
WSDL / JAXB Binding Generation X
Operation Message Test Case Generation X
Production Services Message Generation X
SQL Schema Mapping X X
NIEM IEPD Artifact Generation X
- Cross-reference spreadsheet X
- UML model X
- Business Rules Report X
- XSD schema package X
Conformance Test Suite X
Run time XML instance validation X
In order to be NIEM-conformant, the IEPD must adhere to:
1. NIEM Conformance Document
2. NIEM Naming and Design Rules (NDR) v1.3
3. NIEM Model Package Description (MPD) Specification v1.0

CAM Toolkit for NIEM
• Concepts behind the Content Assembly Mechanism
approach
• Understanding the steps involved in top down /
bottom up exchange design, testing and delivery
• Applicability to NIEM and IEPD / MPD delivery
* NIEM – National Information Exchange Model
* IEPD – Information Exchange Package Documentation
*MPD – Model Package Description

CAM Templates – WYSIWYG Exchange
Structure Rules
Documentation
CAM Template
http://www.oasis-open.org/committees/cam

INGEST
CAM and W3C Schema Wizards
EXPORT
XSLT tools
process
entire XSD
collection
XSD Collection
Structure
Rules
Documentation
*Code list xsd
Namespace* Optional
XML Schema
xsd subset *Constraint xsd
*Extension xsd
*Restriction xsd
Industry Domain
schemas
Code list xsd
LEGEND:
Exchange xsd
Exchange
Schema
CAM Template

http://www.cameditor.org CAM Editor – Main Screens

CAM Toolkit Features
• Dictionary “Drag and Drop” of components to build exchanges
• Visual editor and structure editing
• Cross-platform deploy - Windows, Ubuntu/LINUX, Mac; 32/64bit
• Mind map models – integration with Freemind tool
• UML/XMI export – import to UML tools / ArgoUML
• Code list table lookups – rule based and selectable
• This allows people to fully integrate their exchange development
from dictionaries through to working deployed exchange
schema.
31

XML Exchange Package Artifacts
• Template Evaluation Report
– Naming and Design Rules assessment
– Interoperability factors review
– Spell checking
• Comparison to Dictionary
– Reuse report, cross-reference spreadsheet, subset “wantlist”
• Generate Schema Bundle with layout styles (e.g. NIEM)
• Realistic test samples generation
• Testing with runtime validation engine (CAMV)
• Business information exchange documentation
• Models – XMI and mind maps

CAM Template Evaluation Report
• The purpose of the CAM template evaluation report is
to provide a suite of checks against a template that
are difficult, tedious or otherwise to determine by
manual inspection of a template structure and its
associated rules and annotation details
• The report covers the following areas:
– alerts users to potential issues with their template and
provides suggestions for improving the functional details
– addresses logical and conceptual issues that simple syntax
checking alone does not flag
– can be used as part of a formal review process to ensure a
baseline for quality control

CAM Toolkit Capabilities

Online Tutorials – www.niemtrainingvideos.org

KEY PILLAR OF DICTIONARIES
Introduction and Concepts

The Semantic Framework and Vision
XML/NIEM
Components in
Neutral
Dictionary
Representation
UML
Technologies
XML
Technologies
W3C Schema
Tools
Semantic
Representations
Rules
Technologies
HTML 5
rendering tools
Security
solutions
Middleware
solutions

Dictionaries: Key Pillar of Information Sharing
Canonical XML Components Dictionary1
Component Associations and Couplings2
Exchange Templates and Rules3
W3C Schema and Model Representations4
Delivery Control, Messaging, Security5
Implementation Artifacts and Examples6
Navigation
and Query
Tools
Domains Reuse Library
Components
Automatic Rendering Tools
Deployment Environments and Middleware
XML
User Inserts
Relationship
Lookups
Template
Catalog
Testing Workbench, Rules Engine, Data Samples, Integration ETL
Schema
XML
Models Component
s
Canonical
Dictionary
Collections

Dictionaries Semantics Approach
• Abstract dictionary representations with strong
semantics, rules and code lists support
• Schema, models, documentation are generated from
dictionary definitions automatically
• Contextual mechanisms allow linear number of true
core components – not exponential growth
• Easier for practitioners to leverage with more
predictable and repeatable results
• Enable domain use horizontally across government
• ETL mapping automation support
• Continuous collaboration environment

Developing domain dictionaries (EIEM)
• Allows domains to manage their components libraries
• Provides consistency for project development teams
• Sets of NIEM consistent XML exchange components
• Aligned to enterprise data stores
• Optimized for reuse and interoperability
• Save time and effort across the enterprise
• Perennial question for developers – when should I use NIEM
components, and when our own local ones?
• Are there components already available for that purpose?
• Provide formal mechanisms and procedures to share
components and collaborate across SDLC process
• Provide external parties consistent data views
*EIEM – Enterprise Information Exchange Model

EIEM/BIEC
• Business Information Exchange
Components (BIEC)
• NIEM-conforming XML schema data
component definition that meets a
particular recurring business
requirement for an enterprise
• Enterprise Information Exchange
Model (EIEM)
• NIEM-conforming schemas that define
data components to be reused in
IEPDs developed by an enterprise;
collection of enterprise BIECs
organized into a subset and one or
more extension schemas

Managed Dictionaries – Authoritative Sources
Slide 42
Centralized
Distributed
schema
SFIS Values Library Service, NIEM, Metadata Registry
ALLOWEDVALUES
Managed
Dictionaries
Publish

Why CAM and NIEM?
“Can you give us a brief overview of how you expect IT
professionals and line of business managers to use
CAM*?”
– Van Ristau, CTO DLT Inc.
*CAM editor toolkit – http://www.cameditor.org and OASIS Content Assembly Mechanism standard – http://docs.oasis-open.org/cam

Response
• Line of business managers can use CAM to drive better
alignment to NIEM and faster more consistent and
predictable delivery of NIEM information exchanges
• CAM provides tools that allow business analysts to work
collaboratively with software developers to build exchanges
• Teams use CAM to perform the 8 “D’s” tasks when
implementing NIEM exchanges
• CAM automates the delivery of key XML artifacts for NIEM
exchange services
• XML validation can be deployed using the CAMV engine
• Open-XDX provides rapid flexible open data web services

Public Sector Open Data Services Delivery
Open Platform Solutions
• Visual Editor
• SQL Browser
• Dictionaries
• Drag and Drop
• Query/Response
• Generate Data
• Validate Data
• Send Data
• JAX-WS
• soapUI
• CAM Editor
• Content Models
• OER Integration
• Web Forms UI
Open Data Test Bed Services Dictionaries
DB
Web
Open XDX
CAMV
Open-XDX / CAM Editor
Dictionary
Security
SQL
XML
XML
SQL
Repository
• CAM Editor
• Templates
• Policy Rules
• XACML
Templates
XACML

Technology Components and Features
• Providing open data sharing
– Open-XDX component accelerates deployment of interfaces
• Aligning meaning and definitions
– Collaboration using XML component dictionaries and NIEM
• Formal information verification and content rules
– CAMV rules engine provides runtime handling
– Industry domain test bed facilities
– Rules documentation and sharing
• Business process integration
– Oracle BPM and Forms combined with CAMV and Open-XDX
• In progress – Security profiles and policy templates
– Upcoming release of CAM Editor and XACML integration

CAMV
interactive
testing
Cloud Services Deployment
Open-XDX
web
services
XML
Technologies
W3C Schema
Tools
Semantic
Representations
Rules
Technologies
Security
solutions
Middleware solutions
CAMV test
bed for
industries
Registry for
Dictionaries
Security
Profile
Templates
Web site and collaboration tools
Policy
Rules

CAMeditor.ORG Project Statistics
SNAPSHOT OF
PROJECT ACTIVITIES
135,000+ CAMeditor.org
page visits to site
165+ countries have
downloaded tools;
30% of visitors are from
U.S.;
800+ downloads weekly
2000+ student views of
online video training
resources
8 languages now
available
www.cameditor.org
www.niemtrainingvideos.org

NIEM and XML for Architects and Developers

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (9)

Semelhante a NIEM and XML for Architects and Developers

Semelhante a NIEM and XML for Architects and Developers (20)

Mais de Bizagi Inc

Mais de Bizagi Inc (11)

Último

Último (20)

NIEM and XML for Architects and Developers