SlideShare uma empresa Scribd logo
1 de 45
Baixar para ler offline
1© 2019 IDERA, Inc. All rights reserved.
LEAN DATA MODELING FOR ANY METHODOLOGY
JUNE 18, 2019
Ron Huizenga
Senior Product Manager, Enterprise Architecture & Modeling
@DataAviator
2© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 2© 2019 IDERA, Inc. All rights reserved.
PRE-FLIGHT BRIEFING
▪ A brief history lesson
▪ Methodology contrast
▪ The human factor
▪ Data modeling’s increasing value
▪ Case study
• Plan vs. reality
• Quality metrics
• Data modeling impact
▪ Lean principles
• And how to apply them to data
▪ Summary
3© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 3© 2019 IDERA, Inc. All rights reserved.
A BRIEF HISTORY LESSON (PART 1)
TOTAL QUALITY MANAGEMENT
(TQM)
1980’s & 1990’s
Industrialization (manufacturing) is the basis for systems development:
4© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 4© 2019 IDERA, Inc. All rights reserved.
A BRIEF HISTORY LESSON (PART 2)
PREDICTIVE ADAPTIVE
5© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 5© 2019 IDERA, Inc. All rights reserved.
METHODOLOGIES AND DEFINITIONS
▪ Waterfall
• A linear, sequential approach to the software development life cycle (SDLC)
• Used in software engineering and product development.
• Emphasizes a logical progression of steps.
• Requirements -> Analysis -> Design -> Develop -> Test -> Deploy -> Maintain
▪ Agile
• Software development based on iterative development
• Requirements and solutions evolve through collaboration
• Self-organizing, cross-functional teams
• “Increases productivity and reduces time to benefits relative to waterfall”
• Variants
• SCRUM
• Extreme Programming (XP)
6© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 6© 2019 IDERA, Inc. All rights reserved.
SCRUM
▪ A lightweight process framework for Agile software development
▪ Fixed duration iterations called Sprints (30 days)
▪ Product backlog
▪ Sprint backlog
▪ Self organizing team
• Product Owner
• Keeper of the requirements
• SCRUM Master
• Keeper of the process
▪ Daily SCRUM meetings
▪ Sprint kickoff, Sprint Retrospective
7© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 7© 2019 IDERA, Inc. All rights reserved.
EXTREME PROGRAMMING (XP)
▪ The most specific (Radical) of the agile software development frameworks
▪ Five values of XP:
• Communication - face to face discussion with white board
• Simplicity - “what is the simplest thing that will work?”
• Constant Feedback - build – feedback – adjust
• Courage - “effective action in the face of fear”
• Respect – respectful collaboration in the team
▪ Practices
• User stories
• Paired programming
• Small Releases
• Simple Design
• Refactoring
• Continuous integration
• 40 hour work (maximum)
8© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 8© 2019 IDERA, Inc. All rights reserved.
WATERFALL VS AGILE
Data Modeling
9© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 9© 2019 IDERA, Inc. All rights reserved.
AGILE CYCLES
10© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 10© 2019 IDERA, Inc. All rights reserved.
AGILE MISINTERPRETED AND MISALIGNED
▪ Short term project perspective vs. longer term organizational benefits
▪ It’s all about producing usable software in every iteration
• Often used as an excuse to shortcut or omit other important deliverables
• Data architecture/integration
• Documentation
• Decommissioning of replaced applications/systems
• Sound architecture often overlooked because “the business user didn’t tell us that”
• Requirements interpreted too literally
▪ Blind focus on software only
• “Models are good documentation, but they are immediately obsolete.”
11© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 11© 2019 IDERA, Inc. All rights reserved.
THE HUMAN FACTOR
▪ Scrum vs. Extreme
▪ Self-organizing team concept
• Often misinterpreted as role-less (extreme)
• Any person can perform any role
• Can switch from sprint to sprint (iteration)
• No specialization
• Reality
• A formula for disaster in all but the simplest of projects
▪ Often accompanied by attitude of disdain for data modelers
• “They just slow us down”
• “We don’t need a data model”
▪ Short-sighted management
• Long term compromised in favor of short term project goals.
12© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 12© 2019 IDERA, Inc. All rights reserved.
DATA ARCHITECT/ MODELER IN AGILE
▪ Enterprise data perspective
▪ Facilitator
• Enabler vs. Gatekeeper
▪ Full engagement in sprint planning
• Ensure completeness of deliverables
• Prioritization of dependencies
▪ Iterative work style
• Many simultaneous deliverables
▪ Collaboration
• Work with multiple teams simultaneously
• Cross-project focus
vs.
13© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 13© 2019 IDERA, Inc. All rights reserved.
MODEL TYPES & DIAGRAMS
▪ Data Model Separation
• Conceptual Models
• Logical Models
• Physical Models
▪ Specialized Data Models
• Dimensional
• NoSQL
▪ Data lineage
▪ Business process models
• Provide context
14© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 14© 2019 IDERA, Inc. All rights reserved.
DATA MODEL CONSTRUCTS
▪ Full Specification
• Logical
• Physical
▪ Persistence Boundaries
• Business Data Objects
▪ Descriptive metadata
• Names
• Definitions (data dictionary)
• Notes
▪ Implementation characteristics
• Data types
• Keys
• Indexes
• Views
▪ Business Rules
• Relationships (referential constraints)
• Value Restrictions (constraints)
▪ Security Classifications + Rules
▪ Governance Metadata
• Master Data Management classes
• Data Quality classifications
• Retention policies
15© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 15© 2019 IDERA, Inc. All rights reserved.
GOVERNANCE METADATA
16© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 16© 2019 IDERA, Inc. All rights reserved.
CASE STUDY– AS PLANNED
▪ Supply Chain – Commercial Application Suite
▪ 1 Common Database
▪ 4 Parallel Development Streams
• By functional area
▪ Planned Duration: 1 year
▪ Planned Cost: $6,000,000
▪ Agile Methodology (Extreme & Scrum)
• Developers responsible for all design/development
• 2 week sprints (iterations)
▪ Weekly budgeted direct staffing costs: $ 92,800
• Did not include business SMEs as they were covered separately in corporate budget
17© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 17© 2019 IDERA, Inc. All rights reserved.
INITIAL WEEKS
▪ High defect rate
▪ Backlog growing rapidly
▪ By week 16, 50% of effort being spent addressing defects
• Direct cost $46,400/week
• Without being addressed, project schedule would need to be extended 40 weeks
(additional cost of $ 3.7 million)
Excitement!
Anticipation!
Reality:
18© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 18© 2019 IDERA, Inc. All rights reserved.
PROBLEM ASSESSMENT
▪ Define
• Defect categories
▪ Measure
• Discrete vs. weighted impact
• Linear vs. cumulative measurement
▪ Analyze
• Time series distribution
• Defects per object
• Defects vs. opportunities
▪ Improve
• Remediation strategy
▪ Control
• Comparative metrics
19© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 19© 2019 IDERA, Inc. All rights reserved.
System Defects
& Rework
Requirements
Database & Persistence
User Interface
Business Services
Incomplete user stories
Incorrect business analysis documents
Missing foreign key constraints
Missing check constraints
Missing default values
Incorrect data type
Missing index
Missing audit columns
Incorrect table name
Incorrect column name
Tables not in 3rd
Normal Form Incorrect state transition
Calculation Error
Logic construct error
Incorrect looping or branching
Services not invoked
Messages
Navigation flow
Values not sorted in dropdowns
Subfile/list overflow
Controls not working
Missing prompts
Screens not user friendly
Incorrect service invoked
Entity Framework mapping error
Business Process has changed
Incorrect test cases
Defective unit tests
3rd
party widget
integration problems
Missing processes
Inadequate Subject Matter Expert Knowledge
DEFINE: DEFECT CATEGORIES & IMPACT
20© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 20© 2019 IDERA, Inc. All rights reserved.
CUMULATIVE DEFECTS
21© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 21© 2019 IDERA, Inc. All rights reserved.
WEIGHTED DEFECT CATEGORIES
Defect Category Primary Layer
Impact
Comments Defect
Count
Cumulative
Count
Defect % Cumulative
Defect %
Database &
Persistence
Data Layer Database and persistence errors can be very
problematic, time consuming and expensive
to correct. There is always an impact to the
persistence mapping in the business services
layer which must be corrected. In addition,
changes may also ripple to the User Interface
layer.
243 243 35.01% 35.01%
Business services Business Layer Errors in business services are typically
problems in logic, calculations etc. This could
cause erronious data. However, the
corrections are usually limited to the
business layer itself, and do not require
structural changes (and hence mapping
changes to the data layer).
212 455 30.55% 65.56%
User Interface Presentation Layer UI errors are almost always isolated to the
presentation layer and generally fairly straigh
forward to fix.
197 652 28.39% 93.95%
Requirements any Requirments errors could impact any and all
layers, depending upon the severity or scope
of the error. They can not be quantified in
general and must be examined on a case by
case basis to determine impact.
42 694 6.05% 100.00%
Total 694 694 100.00% 100.00%
Severity
Points
Weighted
Score
Cumulative
Score
Score % Cumulative
Score %
7 1,701 1,701 62.95% 62.95%
3 636 2,337 23.54% 86.49%
1 197 2,534 7.29% 93.78%
4 168 2,702 6.22% 100.00%
2702 2,702 100.00% 100.00%
A x B =
22© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 22© 2019 IDERA, Inc. All rights reserved.
CUMULATIVE DEFECT SEVERITY
23© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 23© 2019 IDERA, Inc. All rights reserved.
SPECIFIC DATABASE DEFECT POINT VALUES (SEVERITY)
No. Defect Type Description Points
1 Duplicate table 10
2 Table not normalized 10
3 Primary Key Incorrect 5
4 Missing Foreign Key (relationship) 5
5 Referential Integrity constraint incorrect 3
6 Missing foreign key index 2
7 Audit Column missing 2
8 Check Constraint Missing 1
9 Default value not specified 1
10 Incorrect table naming 3
11 Column data type incorrect 2
12 Column NULL specification incorrect 1
13 Incorrect column naming 2
24© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 24© 2019 IDERA, Inc. All rights reserved.
DATABASE & PERSISTENCE DEFECTS
25© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 25© 2019 IDERA, Inc. All rights reserved.
TIME SERIES DISTRIBUTION OF DEFECTS
26© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 26© 2019 IDERA, Inc. All rights reserved.
SMOOTHING – CUMULATIVE ANALYSIS
27© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 27© 2019 IDERA, Inc. All rights reserved.
REMEDIATION
▪ Apply LEAN principles to:
• Increase efficiency (eliminate waste)
• Build in quality
• Create knowledge
• Optimize
▪ Use Senior Data Architect – Cross Team Focus
• Introduced in week 21 of project
▪ Process Changes
• Model all changes
• Generate DDL from modeling tool
• 1 developer dedicated to persistence mapping
• Works for data architect
▪ Halt functional design/development to reset
• Redesign database
• Sprints dedicated to problem cleanup
▪ Target: Reduce data defects by at least 75% going forward
28© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 28© 2019 IDERA, Inc. All rights reserved.
OBJECTS & DEFECTS/WEEK COMPARISON
29© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 29© 2019 IDERA, Inc. All rights reserved.
DEFECTS PER OBJECT COMPARISON
30© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 30© 2019 IDERA, Inc. All rights reserved.
COMPARISON – CUMULATIVE OBJECTS VS. DEFECTS
31© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 31© 2019 IDERA, Inc. All rights reserved.
COMPARATIVE
Measurement
Measurement Period
(Weeks 1 -20)
Control Period (Weeks
21 - 31)
Performance
Improvement
Interval Length (weeks) 20 11
Objects Created 957 1,083
Defects 1,077 38
Defect Opportunities 4,090 4,333
Defect Points 1,696 87
Defect Point Opportunities 8,886 8,991
Average Objects/week 47.85 98.45 205.76%
Average Defects/week 53.85 3.45 1558.82%
Average Defect Points/week 84.80 7.91 1072.18%
Average defects/object 1.13 0.04 3207.37%
Average Defect Opportunities/Week 204.50 393.91
Defects/Opportunity 0.263 0.009 3002.60%
Defect Points/Opportunity 0.191 0.010 1972.46%
32© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 32© 2019 IDERA, Inc. All rights reserved.
THE BOTTOM LINE
▪ On time completion
▪ Avoided $3.7 million overrun
▪ Senior Enterprise Data Architect + Modeling Tools $200K
• Duration of project
▪ ROI: ($3.7 million – $200K)/$200K = 1,750%
• Had this been done at the beginning of the project, returns would have been even
greater
33© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 33© 2019 IDERA, Inc. All rights reserved.
WHAT IS LEAN?
▪ Has it’s basis in manufacturing, and has been adapted to
knowledge work
• Toyota Production System (TPS)
▪ Organizational focus vs. Agile’s software focus
▪ Repeatable process to minimize waste, maximize value
▪ Requires
• Quality standards
• Collaboration of specialized workers
▪ Kaizen
• “kai-” (change) “-zen” (good)
• “continuous improvement” or “small incremental
improvements” of all areas of a company
34© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 34© 2019 IDERA, Inc. All rights reserved.
LEAN PRINCIPLES
▪ Eliminate waste
• Eliminate anything that does not add value
▪ Build quality in
• Quality is everybody’s job!
• Test driven, incremental development with constant feedback
• Automate processes prone to human error
▪ Create knowledge
• Properly document and retain valuable learning
▪ Deliver fast
• Remove blockers
• Don’t over-engineer
▪ Respect people
• All aspects: communication, handle conflict, onboarding, process improvement
• Empowerment
▪ Optimize the whole
• Don’t sacrifice quality for speed
• Understand capacity and downstream impact of all work
• Identify and optimize value streams
35© 2019 IDERA, Inc. All rights reserved.
AGILE VS. LEAN
▪ Agile
▪ Proposed as “a better way of
developing software
▪ Bottom-up focus
• Short cycle, frequent delivery
▪ Kanban usage
• Fixed duration iterations
• Limit time of development
• Each iteration begins with a fresh
board
▪ Focus is delivering software
▪ Lean
▪ Strategic as well as operational
• Improve IT’s value to the organization
▪ Top-down, End-to-End Focus (E2E)
• “See the whole”
▪ Kanban usage
• Continuous flow
• Limit work-in-progress
• When a task completes, PULL the next
in sequence
▪ Focus is delivering real value
• (not just software)
“Agile is the new Waterfall”
36© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 36© 2019 IDERA, Inc. All rights reserved.
START OF ITERATION
▪ Participate fully in iteration planning
▪ Ensure there is a “Named Release” as of completion of previous iteration
• Always have a baseline for compare/merge !
▪ Submodels
• Structure by relevant topic/subject area
• At story level if necessary to facilitate communication
• Roll up to parent level submodels
37© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 37© 2019 IDERA, Inc. All rights reserved.
MANAGING ITERATIONS
▪ Always have a baseline for compare/merge !
▪ Within iteration workflow
• Model each change, associating with appropriate task/user story
• Generate incremental DDL script(s) and stage to build server
• Use a robust script naming convention, particularly if utilizing automated build systems
• 1 data modeler may be working with multiple dev teams simultaneously
• Some designs will be originated by data modeler
• Others may be from developer “sandbox”
− Compare/merge and redesign as appropriate
− Ensure developer uses the officially sanctioned script
• Create “Named Release” at end of iteration
• Create delta script by using compare/merge
• Based on Named Release from the previous iteration
▪ Use sub-models for audience specific perspective
▪ Maintain the discipline!
▪ Participate fully in iteration planning and retrospectives
38© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 38© 2019 IDERA, Inc. All rights reserved.
ER/STUDIO: CHANGE MANAGEMENT CENTER - TRACEABILITY
39© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 39© 2019 IDERA, Inc. All rights reserved.
MANAGING COMPLEXITY
▪ Have an overall plan guiding the initiative
• Usually requires analysis and some modeling BEFORE development starts
▪ Some areas may be very complex, requiring multiple iterations to design/develop
▪ Use data model design patterns as a starting point
▪ The “wave” approach
• Data modelers working on some items 1 or 2 iterations ahead of the development team
• Logical / Physical modeling separation facilitates this
• Make changes to logical model in advance
• Compare/merge appropriate changes to physical at the right time
• Enterprise perspective of the data
▪ Fully documented data models!!
• Data dictionary definitions
• Documented relationships/role names
• The physical model IS the implementation
• ALL physical constructs
40© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 40© 2019 IDERA, Inc. All rights reserved.
ER/STUDIO – COMPARE AND MERGE
41© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 41© 2019 IDERA, Inc. All rights reserved.
GENERATE SCRIPT
42© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 42© 2019 IDERA, Inc. All rights reserved.
END OF ITERATION WRAP-UP
▪ Create “Named Release” at completion
• Serves as baseline for start of next iteration
• Serves as baseline for comparison at ANY later point
▪ Create delta DDL script by using compare/merge
• Based on Named Release from end of the previous iteration
▪ Create full database DDL script
• Can be used to easily create “sandbox” databases quickly
▪ Ensure the model(s) have been published
▪ Participate fully in planning and retrospective meetings
• Lessons learned
• Celebrate the successes
43© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 43© 2019 IDERA, Inc. All rights reserved.
AUTOMATED BUILD SYSTEM CONSIDERATIONS
▪ Require synchronized deliverables
▪ Database (DDL)
▪ Application code
▪ Persistence
• Data services
• Framework updates
44© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 44© 2019 IDERA, Inc. All rights reserved.
POST FLIGHT DE-BRIEF
▪ Systems development is continually evolving and improving
• There have been no brand new, groundbreaking ideas
• Derived from manufacturing principles and practices proven to deliver business value
• Learn and adapt based on the cumulative body of knowledge
• And fit to suit organizational culture
▪ DATA has ALWAYS been important. More companies are recognizing that.
• Applications come and go
• Companies always want to retain the data!
• Data models are more important than ever in order to
• Manage complexity
• Increase quality
• Deliver value
• Avoid failure.
▪ Lean principles improve systems development
• Value focus
• Efficiency
• Waste reduction
• Customer Satisfaction
▪ Approaches utilizing lean are the most successful
• Predominantly adaptive
• With predictive capabilities incorporated
• Best of both worlds
45© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 45© 2019 IDERA, Inc. All rights reserved.
THANKS!
Any questions?
You can find me at:
ron.huizenga@idera.com
@DataAviator

Mais conteúdo relacionado

Mais procurados

BigQuery implementation
BigQuery implementationBigQuery implementation
BigQuery implementationSimon Su
 
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...Databricks
 
Apache Flink Adoption at Shopify
Apache Flink Adoption at ShopifyApache Flink Adoption at Shopify
Apache Flink Adoption at ShopifyYaroslav Tkachenko
 
Standards Metadata Management (system)
Standards Metadata Management (system)Standards Metadata Management (system)
Standards Metadata Management (system)Kevin Lee
 
Using the FLaNK Stack for edge ai (flink, nifi, kafka, kudu)
Using the FLaNK Stack for edge ai (flink, nifi, kafka, kudu)Using the FLaNK Stack for edge ai (flink, nifi, kafka, kudu)
Using the FLaNK Stack for edge ai (flink, nifi, kafka, kudu)Timothy Spann
 
0-60: Tesla's Streaming Data Platform ( Jesse Yates, Tesla) Kafka Summit SF 2019
0-60: Tesla's Streaming Data Platform ( Jesse Yates, Tesla) Kafka Summit SF 20190-60: Tesla's Streaming Data Platform ( Jesse Yates, Tesla) Kafka Summit SF 2019
0-60: Tesla's Streaming Data Platform ( Jesse Yates, Tesla) Kafka Summit SF 2019confluent
 
Automate data warehouse etl testing and migration testing the agile way
Automate data warehouse etl testing and migration testing the agile wayAutomate data warehouse etl testing and migration testing the agile way
Automate data warehouse etl testing and migration testing the agile wayTorana, Inc.
 
SAP S4/HANA meetup overview
SAP S4/HANA meetup overview SAP S4/HANA meetup overview
SAP S4/HANA meetup overview Accenture Hungary
 
Decoding SAP S/4HANA System Conversion
Decoding SAP S/4HANA System ConversionDecoding SAP S/4HANA System Conversion
Decoding SAP S/4HANA System ConversionAkilesh Kumaran
 
Accelerate and modernize your data pipelines
Accelerate and modernize your data pipelinesAccelerate and modernize your data pipelines
Accelerate and modernize your data pipelinesPaul Van Siclen
 
Scaling up uber's real time data analytics
Scaling up uber's real time data analyticsScaling up uber's real time data analytics
Scaling up uber's real time data analyticsXiang Fu
 
Sap S4 HANA Everything You Need To Know
Sap S4 HANA Everything You Need To Know Sap S4 HANA Everything You Need To Know
Sap S4 HANA Everything You Need To Know Soumya De
 
Building an open data platform with apache iceberg
Building an open data platform with apache icebergBuilding an open data platform with apache iceberg
Building an open data platform with apache icebergAlluxio, Inc.
 
Lessons learnt on setting up and scaling an automation CoE
Lessons learnt on setting up and scaling an automation CoELessons learnt on setting up and scaling an automation CoE
Lessons learnt on setting up and scaling an automation CoEMindfields Global
 
Cloud DW technology trends and considerations for enterprises to apply snowflake
Cloud DW technology trends and considerations for enterprises to apply snowflakeCloud DW technology trends and considerations for enterprises to apply snowflake
Cloud DW technology trends and considerations for enterprises to apply snowflakeSANG WON PARK
 
Pentaho Data Integration Introduction
Pentaho Data Integration IntroductionPentaho Data Integration Introduction
Pentaho Data Integration Introductionmattcasters
 
Parquet performance tuning: the missing guide
Parquet performance tuning: the missing guideParquet performance tuning: the missing guide
Parquet performance tuning: the missing guideRyan Blue
 

Mais procurados (20)

BigQuery implementation
BigQuery implementationBigQuery implementation
BigQuery implementation
 
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
Solving Data Discovery Challenges at Lyft with Amundsen, an Open-source Metad...
 
Apache Flink Adoption at Shopify
Apache Flink Adoption at ShopifyApache Flink Adoption at Shopify
Apache Flink Adoption at Shopify
 
Standards Metadata Management (system)
Standards Metadata Management (system)Standards Metadata Management (system)
Standards Metadata Management (system)
 
FLiP Into Trino
FLiP Into TrinoFLiP Into Trino
FLiP Into Trino
 
Using the FLaNK Stack for edge ai (flink, nifi, kafka, kudu)
Using the FLaNK Stack for edge ai (flink, nifi, kafka, kudu)Using the FLaNK Stack for edge ai (flink, nifi, kafka, kudu)
Using the FLaNK Stack for edge ai (flink, nifi, kafka, kudu)
 
0-60: Tesla's Streaming Data Platform ( Jesse Yates, Tesla) Kafka Summit SF 2019
0-60: Tesla's Streaming Data Platform ( Jesse Yates, Tesla) Kafka Summit SF 20190-60: Tesla's Streaming Data Platform ( Jesse Yates, Tesla) Kafka Summit SF 2019
0-60: Tesla's Streaming Data Platform ( Jesse Yates, Tesla) Kafka Summit SF 2019
 
Automate data warehouse etl testing and migration testing the agile way
Automate data warehouse etl testing and migration testing the agile wayAutomate data warehouse etl testing and migration testing the agile way
Automate data warehouse etl testing and migration testing the agile way
 
SAP S4/HANA meetup overview
SAP S4/HANA meetup overview SAP S4/HANA meetup overview
SAP S4/HANA meetup overview
 
Decoding SAP S/4HANA System Conversion
Decoding SAP S/4HANA System ConversionDecoding SAP S/4HANA System Conversion
Decoding SAP S/4HANA System Conversion
 
Accelerate and modernize your data pipelines
Accelerate and modernize your data pipelinesAccelerate and modernize your data pipelines
Accelerate and modernize your data pipelines
 
Scaling up uber's real time data analytics
Scaling up uber's real time data analyticsScaling up uber's real time data analytics
Scaling up uber's real time data analytics
 
SAP API Business Hub
SAP API Business HubSAP API Business Hub
SAP API Business Hub
 
Sap S4 HANA Everything You Need To Know
Sap S4 HANA Everything You Need To Know Sap S4 HANA Everything You Need To Know
Sap S4 HANA Everything You Need To Know
 
Building an open data platform with apache iceberg
Building an open data platform with apache icebergBuilding an open data platform with apache iceberg
Building an open data platform with apache iceberg
 
Lessons learnt on setting up and scaling an automation CoE
Lessons learnt on setting up and scaling an automation CoELessons learnt on setting up and scaling an automation CoE
Lessons learnt on setting up and scaling an automation CoE
 
Cloud DW technology trends and considerations for enterprises to apply snowflake
Cloud DW technology trends and considerations for enterprises to apply snowflakeCloud DW technology trends and considerations for enterprises to apply snowflake
Cloud DW technology trends and considerations for enterprises to apply snowflake
 
Introduction to ETL and Data Integration
Introduction to ETL and Data IntegrationIntroduction to ETL and Data Integration
Introduction to ETL and Data Integration
 
Pentaho Data Integration Introduction
Pentaho Data Integration IntroductionPentaho Data Integration Introduction
Pentaho Data Integration Introduction
 
Parquet performance tuning: the missing guide
Parquet performance tuning: the missing guideParquet performance tuning: the missing guide
Parquet performance tuning: the missing guide
 

Semelhante a Lean Modeling for Any Methodology

Data Architecture - The Foundation for Enterprise Architecture and Governance
Data Architecture - The Foundation for Enterprise Architecture and GovernanceData Architecture - The Foundation for Enterprise Architecture and Governance
Data Architecture - The Foundation for Enterprise Architecture and GovernanceDATAVERSITY
 
Slides: The Business Value of Data Modeling
Slides: The Business Value of Data ModelingSlides: The Business Value of Data Modeling
Slides: The Business Value of Data ModelingDATAVERSITY
 
Strategic imperative the enterprise data model
Strategic imperative the enterprise data modelStrategic imperative the enterprise data model
Strategic imperative the enterprise data modelDATAVERSITY
 
MongoDB World 2018: From Disruption to Transformation: Document Databases, Do...
MongoDB World 2018: From Disruption to Transformation: Document Databases, Do...MongoDB World 2018: From Disruption to Transformation: Document Databases, Do...
MongoDB World 2018: From Disruption to Transformation: Document Databases, Do...MongoDB
 
Data Maturity - A Balanced Approach
Data Maturity - A Balanced ApproachData Maturity - A Balanced Approach
Data Maturity - A Balanced ApproachDATAVERSITY
 
IDERA Live | Databases Don't Build and Populate Themselves
IDERA Live | Databases Don't Build and Populate ThemselvesIDERA Live | Databases Don't Build and Populate Themselves
IDERA Live | Databases Don't Build and Populate ThemselvesIDERA Software
 
IDERA Live | Business Value Metrics for Data Governance
IDERA Live | Business Value Metrics for Data GovernanceIDERA Live | Business Value Metrics for Data Governance
IDERA Live | Business Value Metrics for Data GovernanceIDERA Software
 
Office 365 Monitoring Best Practices
Office 365 Monitoring Best PracticesOffice 365 Monitoring Best Practices
Office 365 Monitoring Best PracticesThousandEyes
 
Agile in corporate environment with JIRA by Feras El Hajjar
Agile in corporate environment with JIRA by Feras El HajjarAgile in corporate environment with JIRA by Feras El Hajjar
Agile in corporate environment with JIRA by Feras El HajjarAgile ME
 
Deliver Unrivaled End-User Experience With Confidence - How Synthetic Monitor...
Deliver Unrivaled End-User Experience With Confidence - How Synthetic Monitor...Deliver Unrivaled End-User Experience With Confidence - How Synthetic Monitor...
Deliver Unrivaled End-User Experience With Confidence - How Synthetic Monitor...DevOps.com
 
Business Value Metrics for Data Governance
Business Value Metrics for Data GovernanceBusiness Value Metrics for Data Governance
Business Value Metrics for Data GovernanceDATAVERSITY
 
Why Your Data Management Strategy Isn't Working (and How to Fix It)
Why Your Data Management Strategy Isn't Working (and How to Fix It)Why Your Data Management Strategy Isn't Working (and How to Fix It)
Why Your Data Management Strategy Isn't Working (and How to Fix It)DATAVERSITY
 
Supply Chain Visualization
Supply Chain VisualizationSupply Chain Visualization
Supply Chain VisualizationSreenivasa Setty
 
Best Practices for Managing IaaS, PaaS, and Container-Based Deployments - App...
Best Practices for Managing IaaS, PaaS, and Container-Based Deployments - App...Best Practices for Managing IaaS, PaaS, and Container-Based Deployments - App...
Best Practices for Managing IaaS, PaaS, and Container-Based Deployments - App...AppDynamics
 
Human Factors in Data Governance
Human Factors in Data GovernanceHuman Factors in Data Governance
Human Factors in Data GovernanceDATAVERSITY
 
IDERA Live | To Force Plans, Or Not to Force Plans, That Is The Question
IDERA Live | To Force Plans, Or Not to Force Plans, That Is The QuestionIDERA Live | To Force Plans, Or Not to Force Plans, That Is The Question
IDERA Live | To Force Plans, Or Not to Force Plans, That Is The QuestionIDERA Software
 
Webiplex SuiteWorld Gold Sponsor Presentation
Webiplex SuiteWorld Gold Sponsor PresentationWebiplex SuiteWorld Gold Sponsor Presentation
Webiplex SuiteWorld Gold Sponsor PresentationClintHofer1
 
MongoDB World 2019: From Transformation to Innovation: Lean-teams, Continuous...
MongoDB World 2019: From Transformation to Innovation: Lean-teams, Continuous...MongoDB World 2019: From Transformation to Innovation: Lean-teams, Continuous...
MongoDB World 2019: From Transformation to Innovation: Lean-teams, Continuous...MongoDB
 
AAC2018_We're all just doing waterfall really with Iain McKenna
AAC2018_We're all just doing waterfall really with Iain McKennaAAC2018_We're all just doing waterfall really with Iain McKenna
AAC2018_We're all just doing waterfall really with Iain McKennaAgile Austria Conference
 
HOW TO SAVE PILEs of $$$ BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...
HOW TO SAVE  PILEs of $$$BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...HOW TO SAVE  PILEs of $$$BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...
HOW TO SAVE PILEs of $$$ BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...Kent Graziano
 

Semelhante a Lean Modeling for Any Methodology (20)

Data Architecture - The Foundation for Enterprise Architecture and Governance
Data Architecture - The Foundation for Enterprise Architecture and GovernanceData Architecture - The Foundation for Enterprise Architecture and Governance
Data Architecture - The Foundation for Enterprise Architecture and Governance
 
Slides: The Business Value of Data Modeling
Slides: The Business Value of Data ModelingSlides: The Business Value of Data Modeling
Slides: The Business Value of Data Modeling
 
Strategic imperative the enterprise data model
Strategic imperative the enterprise data modelStrategic imperative the enterprise data model
Strategic imperative the enterprise data model
 
MongoDB World 2018: From Disruption to Transformation: Document Databases, Do...
MongoDB World 2018: From Disruption to Transformation: Document Databases, Do...MongoDB World 2018: From Disruption to Transformation: Document Databases, Do...
MongoDB World 2018: From Disruption to Transformation: Document Databases, Do...
 
Data Maturity - A Balanced Approach
Data Maturity - A Balanced ApproachData Maturity - A Balanced Approach
Data Maturity - A Balanced Approach
 
IDERA Live | Databases Don't Build and Populate Themselves
IDERA Live | Databases Don't Build and Populate ThemselvesIDERA Live | Databases Don't Build and Populate Themselves
IDERA Live | Databases Don't Build and Populate Themselves
 
IDERA Live | Business Value Metrics for Data Governance
IDERA Live | Business Value Metrics for Data GovernanceIDERA Live | Business Value Metrics for Data Governance
IDERA Live | Business Value Metrics for Data Governance
 
Office 365 Monitoring Best Practices
Office 365 Monitoring Best PracticesOffice 365 Monitoring Best Practices
Office 365 Monitoring Best Practices
 
Agile in corporate environment with JIRA by Feras El Hajjar
Agile in corporate environment with JIRA by Feras El HajjarAgile in corporate environment with JIRA by Feras El Hajjar
Agile in corporate environment with JIRA by Feras El Hajjar
 
Deliver Unrivaled End-User Experience With Confidence - How Synthetic Monitor...
Deliver Unrivaled End-User Experience With Confidence - How Synthetic Monitor...Deliver Unrivaled End-User Experience With Confidence - How Synthetic Monitor...
Deliver Unrivaled End-User Experience With Confidence - How Synthetic Monitor...
 
Business Value Metrics for Data Governance
Business Value Metrics for Data GovernanceBusiness Value Metrics for Data Governance
Business Value Metrics for Data Governance
 
Why Your Data Management Strategy Isn't Working (and How to Fix It)
Why Your Data Management Strategy Isn't Working (and How to Fix It)Why Your Data Management Strategy Isn't Working (and How to Fix It)
Why Your Data Management Strategy Isn't Working (and How to Fix It)
 
Supply Chain Visualization
Supply Chain VisualizationSupply Chain Visualization
Supply Chain Visualization
 
Best Practices for Managing IaaS, PaaS, and Container-Based Deployments - App...
Best Practices for Managing IaaS, PaaS, and Container-Based Deployments - App...Best Practices for Managing IaaS, PaaS, and Container-Based Deployments - App...
Best Practices for Managing IaaS, PaaS, and Container-Based Deployments - App...
 
Human Factors in Data Governance
Human Factors in Data GovernanceHuman Factors in Data Governance
Human Factors in Data Governance
 
IDERA Live | To Force Plans, Or Not to Force Plans, That Is The Question
IDERA Live | To Force Plans, Or Not to Force Plans, That Is The QuestionIDERA Live | To Force Plans, Or Not to Force Plans, That Is The Question
IDERA Live | To Force Plans, Or Not to Force Plans, That Is The Question
 
Webiplex SuiteWorld Gold Sponsor Presentation
Webiplex SuiteWorld Gold Sponsor PresentationWebiplex SuiteWorld Gold Sponsor Presentation
Webiplex SuiteWorld Gold Sponsor Presentation
 
MongoDB World 2019: From Transformation to Innovation: Lean-teams, Continuous...
MongoDB World 2019: From Transformation to Innovation: Lean-teams, Continuous...MongoDB World 2019: From Transformation to Innovation: Lean-teams, Continuous...
MongoDB World 2019: From Transformation to Innovation: Lean-teams, Continuous...
 
AAC2018_We're all just doing waterfall really with Iain McKenna
AAC2018_We're all just doing waterfall really with Iain McKennaAAC2018_We're all just doing waterfall really with Iain McKenna
AAC2018_We're all just doing waterfall really with Iain McKenna
 
HOW TO SAVE PILEs of $$$ BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...
HOW TO SAVE  PILEs of $$$BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...HOW TO SAVE  PILEs of $$$BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...
HOW TO SAVE PILEs of $$$ BY CREATING THE BEST DATA MODEL THE FIRST TIME (Ksc...
 

Mais de DATAVERSITY

Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...DATAVERSITY
 
Data at the Speed of Business with Data Mastering and Governance
Data at the Speed of Business with Data Mastering and GovernanceData at the Speed of Business with Data Mastering and Governance
Data at the Speed of Business with Data Mastering and GovernanceDATAVERSITY
 
Exploring Levels of Data Literacy
Exploring Levels of Data LiteracyExploring Levels of Data Literacy
Exploring Levels of Data LiteracyDATAVERSITY
 
Building a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsBuilding a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsDATAVERSITY
 
Make Data Work for You
Make Data Work for YouMake Data Work for You
Make Data Work for YouDATAVERSITY
 
Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?DATAVERSITY
 
Data Catalogs Are the Answer – What Is the Question?
Data Catalogs Are the Answer – What Is the Question?Data Catalogs Are the Answer – What Is the Question?
Data Catalogs Are the Answer – What Is the Question?DATAVERSITY
 
Data Modeling Fundamentals
Data Modeling FundamentalsData Modeling Fundamentals
Data Modeling FundamentalsDATAVERSITY
 
Showing ROI for Your Analytic Project
Showing ROI for Your Analytic ProjectShowing ROI for Your Analytic Project
Showing ROI for Your Analytic ProjectDATAVERSITY
 
How a Semantic Layer Makes Data Mesh Work at Scale
How a Semantic Layer Makes  Data Mesh Work at ScaleHow a Semantic Layer Makes  Data Mesh Work at Scale
How a Semantic Layer Makes Data Mesh Work at ScaleDATAVERSITY
 
Is Enterprise Data Literacy Possible?
Is Enterprise Data Literacy Possible?Is Enterprise Data Literacy Possible?
Is Enterprise Data Literacy Possible?DATAVERSITY
 
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...DATAVERSITY
 
Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?DATAVERSITY
 
Data Governance Trends - A Look Backwards and Forwards
Data Governance Trends - A Look Backwards and ForwardsData Governance Trends - A Look Backwards and Forwards
Data Governance Trends - A Look Backwards and ForwardsDATAVERSITY
 
Data Governance Trends and Best Practices To Implement Today
Data Governance Trends and Best Practices To Implement TodayData Governance Trends and Best Practices To Implement Today
Data Governance Trends and Best Practices To Implement TodayDATAVERSITY
 
2023 Trends in Enterprise Analytics
2023 Trends in Enterprise Analytics2023 Trends in Enterprise Analytics
2023 Trends in Enterprise AnalyticsDATAVERSITY
 
Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?DATAVERSITY
 
Data Management Best Practices
Data Management Best PracticesData Management Best Practices
Data Management Best PracticesDATAVERSITY
 
MLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive AdvantageMLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive AdvantageDATAVERSITY
 
Keeping the Pulse of Your Data – Why You Need Data Observability to Improve D...
Keeping the Pulse of Your Data – Why You Need Data Observability to Improve D...Keeping the Pulse of Your Data – Why You Need Data Observability to Improve D...
Keeping the Pulse of Your Data – Why You Need Data Observability to Improve D...DATAVERSITY
 

Mais de DATAVERSITY (20)

Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
Architecture, Products, and Total Cost of Ownership of the Leading Machine Le...
 
Data at the Speed of Business with Data Mastering and Governance
Data at the Speed of Business with Data Mastering and GovernanceData at the Speed of Business with Data Mastering and Governance
Data at the Speed of Business with Data Mastering and Governance
 
Exploring Levels of Data Literacy
Exploring Levels of Data LiteracyExploring Levels of Data Literacy
Exploring Levels of Data Literacy
 
Building a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business GoalsBuilding a Data Strategy – Practical Steps for Aligning with Business Goals
Building a Data Strategy – Practical Steps for Aligning with Business Goals
 
Make Data Work for You
Make Data Work for YouMake Data Work for You
Make Data Work for You
 
Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?Data Catalogs Are the Answer – What is the Question?
Data Catalogs Are the Answer – What is the Question?
 
Data Catalogs Are the Answer – What Is the Question?
Data Catalogs Are the Answer – What Is the Question?Data Catalogs Are the Answer – What Is the Question?
Data Catalogs Are the Answer – What Is the Question?
 
Data Modeling Fundamentals
Data Modeling FundamentalsData Modeling Fundamentals
Data Modeling Fundamentals
 
Showing ROI for Your Analytic Project
Showing ROI for Your Analytic ProjectShowing ROI for Your Analytic Project
Showing ROI for Your Analytic Project
 
How a Semantic Layer Makes Data Mesh Work at Scale
How a Semantic Layer Makes  Data Mesh Work at ScaleHow a Semantic Layer Makes  Data Mesh Work at Scale
How a Semantic Layer Makes Data Mesh Work at Scale
 
Is Enterprise Data Literacy Possible?
Is Enterprise Data Literacy Possible?Is Enterprise Data Literacy Possible?
Is Enterprise Data Literacy Possible?
 
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
The Data Trifecta – Privacy, Security & Governance Race from Reactivity to Re...
 
Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?Emerging Trends in Data Architecture – What’s the Next Big Thing?
Emerging Trends in Data Architecture – What’s the Next Big Thing?
 
Data Governance Trends - A Look Backwards and Forwards
Data Governance Trends - A Look Backwards and ForwardsData Governance Trends - A Look Backwards and Forwards
Data Governance Trends - A Look Backwards and Forwards
 
Data Governance Trends and Best Practices To Implement Today
Data Governance Trends and Best Practices To Implement TodayData Governance Trends and Best Practices To Implement Today
Data Governance Trends and Best Practices To Implement Today
 
2023 Trends in Enterprise Analytics
2023 Trends in Enterprise Analytics2023 Trends in Enterprise Analytics
2023 Trends in Enterprise Analytics
 
Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?Who Should Own Data Governance – IT or Business?
Who Should Own Data Governance – IT or Business?
 
Data Management Best Practices
Data Management Best PracticesData Management Best Practices
Data Management Best Practices
 
MLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive AdvantageMLOps – Applying DevOps to Competitive Advantage
MLOps – Applying DevOps to Competitive Advantage
 
Keeping the Pulse of Your Data – Why You Need Data Observability to Improve D...
Keeping the Pulse of Your Data – Why You Need Data Observability to Improve D...Keeping the Pulse of Your Data – Why You Need Data Observability to Improve D...
Keeping the Pulse of Your Data – Why You Need Data Observability to Improve D...
 

Último

The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerPavel Šabatka
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptaigil2
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...PrithaVashisht1
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?sonikadigital1
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxVenkatasubramani13
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionajayrajaganeshkayala
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructuresonikadigital1
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introductionsanjaymuralee1
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)Data & Analytics Magazin
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best PracticesDataArchiva
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Guido X Jansen
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.JasonViviers2
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxDwiAyuSitiHartinah
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024Becky Burwell
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationGiorgio Carbone
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityAggregage
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Vladislav Solodkiy
 

Último (17)

The Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayerThe Universal GTM - how we design GTM and dataLayer
The Universal GTM - how we design GTM and dataLayer
 
MEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .pptMEASURES OF DISPERSION I BSc Botany .ppt
MEASURES OF DISPERSION I BSc Botany .ppt
 
Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...Elements of language learning - an analysis of how different elements of lang...
Elements of language learning - an analysis of how different elements of lang...
 
How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?How is Real-Time Analytics Different from Traditional OLAP?
How is Real-Time Analytics Different from Traditional OLAP?
 
Mapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptxMapping the pubmed data under different suptopics using NLP.pptx
Mapping the pubmed data under different suptopics using NLP.pptx
 
CI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual interventionCI, CD -Tools to integrate without manual intervention
CI, CD -Tools to integrate without manual intervention
 
ChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics InfrastructureChistaDATA Real-Time DATA Analytics Infrastructure
ChistaDATA Real-Time DATA Analytics Infrastructure
 
Virtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product IntroductionVirtuosoft SmartSync Product Introduction
Virtuosoft SmartSync Product Introduction
 
AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)AI for Sustainable Development Goals (SDGs)
AI for Sustainable Development Goals (SDGs)
 
5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices5 Ds to Define Data Archiving Best Practices
5 Ds to Define Data Archiving Best Practices
 
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
Persuasive E-commerce, Our Biased Brain @ Bikkeldag 2024
 
YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.YourView Panel Book.pptx YourView Panel Book.
YourView Panel Book.pptx YourView Panel Book.
 
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptxTINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
TINJUAN PEMROSESAN TRANSAKSI DAN ERP.pptx
 
SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024SFBA Splunk Usergroup meeting March 13, 2024
SFBA Splunk Usergroup meeting March 13, 2024
 
Master's Thesis - Data Science - Presentation
Master's Thesis - Data Science - PresentationMaster's Thesis - Data Science - Presentation
Master's Thesis - Data Science - Presentation
 
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for ClarityStrategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
Strategic CX: A Deep Dive into Voice of the Customer Insights for Clarity
 
Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023Cash Is Still King: ATM market research '2023
Cash Is Still King: ATM market research '2023
 

Lean Modeling for Any Methodology

  • 1. 1© 2019 IDERA, Inc. All rights reserved. LEAN DATA MODELING FOR ANY METHODOLOGY JUNE 18, 2019 Ron Huizenga Senior Product Manager, Enterprise Architecture & Modeling @DataAviator
  • 2. 2© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 2© 2019 IDERA, Inc. All rights reserved. PRE-FLIGHT BRIEFING ▪ A brief history lesson ▪ Methodology contrast ▪ The human factor ▪ Data modeling’s increasing value ▪ Case study • Plan vs. reality • Quality metrics • Data modeling impact ▪ Lean principles • And how to apply them to data ▪ Summary
  • 3. 3© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 3© 2019 IDERA, Inc. All rights reserved. A BRIEF HISTORY LESSON (PART 1) TOTAL QUALITY MANAGEMENT (TQM) 1980’s & 1990’s Industrialization (manufacturing) is the basis for systems development:
  • 4. 4© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 4© 2019 IDERA, Inc. All rights reserved. A BRIEF HISTORY LESSON (PART 2) PREDICTIVE ADAPTIVE
  • 5. 5© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 5© 2019 IDERA, Inc. All rights reserved. METHODOLOGIES AND DEFINITIONS ▪ Waterfall • A linear, sequential approach to the software development life cycle (SDLC) • Used in software engineering and product development. • Emphasizes a logical progression of steps. • Requirements -> Analysis -> Design -> Develop -> Test -> Deploy -> Maintain ▪ Agile • Software development based on iterative development • Requirements and solutions evolve through collaboration • Self-organizing, cross-functional teams • “Increases productivity and reduces time to benefits relative to waterfall” • Variants • SCRUM • Extreme Programming (XP)
  • 6. 6© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 6© 2019 IDERA, Inc. All rights reserved. SCRUM ▪ A lightweight process framework for Agile software development ▪ Fixed duration iterations called Sprints (30 days) ▪ Product backlog ▪ Sprint backlog ▪ Self organizing team • Product Owner • Keeper of the requirements • SCRUM Master • Keeper of the process ▪ Daily SCRUM meetings ▪ Sprint kickoff, Sprint Retrospective
  • 7. 7© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 7© 2019 IDERA, Inc. All rights reserved. EXTREME PROGRAMMING (XP) ▪ The most specific (Radical) of the agile software development frameworks ▪ Five values of XP: • Communication - face to face discussion with white board • Simplicity - “what is the simplest thing that will work?” • Constant Feedback - build – feedback – adjust • Courage - “effective action in the face of fear” • Respect – respectful collaboration in the team ▪ Practices • User stories • Paired programming • Small Releases • Simple Design • Refactoring • Continuous integration • 40 hour work (maximum)
  • 8. 8© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 8© 2019 IDERA, Inc. All rights reserved. WATERFALL VS AGILE Data Modeling
  • 9. 9© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 9© 2019 IDERA, Inc. All rights reserved. AGILE CYCLES
  • 10. 10© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 10© 2019 IDERA, Inc. All rights reserved. AGILE MISINTERPRETED AND MISALIGNED ▪ Short term project perspective vs. longer term organizational benefits ▪ It’s all about producing usable software in every iteration • Often used as an excuse to shortcut or omit other important deliverables • Data architecture/integration • Documentation • Decommissioning of replaced applications/systems • Sound architecture often overlooked because “the business user didn’t tell us that” • Requirements interpreted too literally ▪ Blind focus on software only • “Models are good documentation, but they are immediately obsolete.”
  • 11. 11© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 11© 2019 IDERA, Inc. All rights reserved. THE HUMAN FACTOR ▪ Scrum vs. Extreme ▪ Self-organizing team concept • Often misinterpreted as role-less (extreme) • Any person can perform any role • Can switch from sprint to sprint (iteration) • No specialization • Reality • A formula for disaster in all but the simplest of projects ▪ Often accompanied by attitude of disdain for data modelers • “They just slow us down” • “We don’t need a data model” ▪ Short-sighted management • Long term compromised in favor of short term project goals.
  • 12. 12© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 12© 2019 IDERA, Inc. All rights reserved. DATA ARCHITECT/ MODELER IN AGILE ▪ Enterprise data perspective ▪ Facilitator • Enabler vs. Gatekeeper ▪ Full engagement in sprint planning • Ensure completeness of deliverables • Prioritization of dependencies ▪ Iterative work style • Many simultaneous deliverables ▪ Collaboration • Work with multiple teams simultaneously • Cross-project focus vs.
  • 13. 13© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 13© 2019 IDERA, Inc. All rights reserved. MODEL TYPES & DIAGRAMS ▪ Data Model Separation • Conceptual Models • Logical Models • Physical Models ▪ Specialized Data Models • Dimensional • NoSQL ▪ Data lineage ▪ Business process models • Provide context
  • 14. 14© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 14© 2019 IDERA, Inc. All rights reserved. DATA MODEL CONSTRUCTS ▪ Full Specification • Logical • Physical ▪ Persistence Boundaries • Business Data Objects ▪ Descriptive metadata • Names • Definitions (data dictionary) • Notes ▪ Implementation characteristics • Data types • Keys • Indexes • Views ▪ Business Rules • Relationships (referential constraints) • Value Restrictions (constraints) ▪ Security Classifications + Rules ▪ Governance Metadata • Master Data Management classes • Data Quality classifications • Retention policies
  • 15. 15© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 15© 2019 IDERA, Inc. All rights reserved. GOVERNANCE METADATA
  • 16. 16© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 16© 2019 IDERA, Inc. All rights reserved. CASE STUDY– AS PLANNED ▪ Supply Chain – Commercial Application Suite ▪ 1 Common Database ▪ 4 Parallel Development Streams • By functional area ▪ Planned Duration: 1 year ▪ Planned Cost: $6,000,000 ▪ Agile Methodology (Extreme & Scrum) • Developers responsible for all design/development • 2 week sprints (iterations) ▪ Weekly budgeted direct staffing costs: $ 92,800 • Did not include business SMEs as they were covered separately in corporate budget
  • 17. 17© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 17© 2019 IDERA, Inc. All rights reserved. INITIAL WEEKS ▪ High defect rate ▪ Backlog growing rapidly ▪ By week 16, 50% of effort being spent addressing defects • Direct cost $46,400/week • Without being addressed, project schedule would need to be extended 40 weeks (additional cost of $ 3.7 million) Excitement! Anticipation! Reality:
  • 18. 18© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 18© 2019 IDERA, Inc. All rights reserved. PROBLEM ASSESSMENT ▪ Define • Defect categories ▪ Measure • Discrete vs. weighted impact • Linear vs. cumulative measurement ▪ Analyze • Time series distribution • Defects per object • Defects vs. opportunities ▪ Improve • Remediation strategy ▪ Control • Comparative metrics
  • 19. 19© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 19© 2019 IDERA, Inc. All rights reserved. System Defects & Rework Requirements Database & Persistence User Interface Business Services Incomplete user stories Incorrect business analysis documents Missing foreign key constraints Missing check constraints Missing default values Incorrect data type Missing index Missing audit columns Incorrect table name Incorrect column name Tables not in 3rd Normal Form Incorrect state transition Calculation Error Logic construct error Incorrect looping or branching Services not invoked Messages Navigation flow Values not sorted in dropdowns Subfile/list overflow Controls not working Missing prompts Screens not user friendly Incorrect service invoked Entity Framework mapping error Business Process has changed Incorrect test cases Defective unit tests 3rd party widget integration problems Missing processes Inadequate Subject Matter Expert Knowledge DEFINE: DEFECT CATEGORIES & IMPACT
  • 20. 20© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 20© 2019 IDERA, Inc. All rights reserved. CUMULATIVE DEFECTS
  • 21. 21© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 21© 2019 IDERA, Inc. All rights reserved. WEIGHTED DEFECT CATEGORIES Defect Category Primary Layer Impact Comments Defect Count Cumulative Count Defect % Cumulative Defect % Database & Persistence Data Layer Database and persistence errors can be very problematic, time consuming and expensive to correct. There is always an impact to the persistence mapping in the business services layer which must be corrected. In addition, changes may also ripple to the User Interface layer. 243 243 35.01% 35.01% Business services Business Layer Errors in business services are typically problems in logic, calculations etc. This could cause erronious data. However, the corrections are usually limited to the business layer itself, and do not require structural changes (and hence mapping changes to the data layer). 212 455 30.55% 65.56% User Interface Presentation Layer UI errors are almost always isolated to the presentation layer and generally fairly straigh forward to fix. 197 652 28.39% 93.95% Requirements any Requirments errors could impact any and all layers, depending upon the severity or scope of the error. They can not be quantified in general and must be examined on a case by case basis to determine impact. 42 694 6.05% 100.00% Total 694 694 100.00% 100.00% Severity Points Weighted Score Cumulative Score Score % Cumulative Score % 7 1,701 1,701 62.95% 62.95% 3 636 2,337 23.54% 86.49% 1 197 2,534 7.29% 93.78% 4 168 2,702 6.22% 100.00% 2702 2,702 100.00% 100.00% A x B =
  • 22. 22© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 22© 2019 IDERA, Inc. All rights reserved. CUMULATIVE DEFECT SEVERITY
  • 23. 23© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 23© 2019 IDERA, Inc. All rights reserved. SPECIFIC DATABASE DEFECT POINT VALUES (SEVERITY) No. Defect Type Description Points 1 Duplicate table 10 2 Table not normalized 10 3 Primary Key Incorrect 5 4 Missing Foreign Key (relationship) 5 5 Referential Integrity constraint incorrect 3 6 Missing foreign key index 2 7 Audit Column missing 2 8 Check Constraint Missing 1 9 Default value not specified 1 10 Incorrect table naming 3 11 Column data type incorrect 2 12 Column NULL specification incorrect 1 13 Incorrect column naming 2
  • 24. 24© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 24© 2019 IDERA, Inc. All rights reserved. DATABASE & PERSISTENCE DEFECTS
  • 25. 25© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 25© 2019 IDERA, Inc. All rights reserved. TIME SERIES DISTRIBUTION OF DEFECTS
  • 26. 26© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 26© 2019 IDERA, Inc. All rights reserved. SMOOTHING – CUMULATIVE ANALYSIS
  • 27. 27© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 27© 2019 IDERA, Inc. All rights reserved. REMEDIATION ▪ Apply LEAN principles to: • Increase efficiency (eliminate waste) • Build in quality • Create knowledge • Optimize ▪ Use Senior Data Architect – Cross Team Focus • Introduced in week 21 of project ▪ Process Changes • Model all changes • Generate DDL from modeling tool • 1 developer dedicated to persistence mapping • Works for data architect ▪ Halt functional design/development to reset • Redesign database • Sprints dedicated to problem cleanup ▪ Target: Reduce data defects by at least 75% going forward
  • 28. 28© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 28© 2019 IDERA, Inc. All rights reserved. OBJECTS & DEFECTS/WEEK COMPARISON
  • 29. 29© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 29© 2019 IDERA, Inc. All rights reserved. DEFECTS PER OBJECT COMPARISON
  • 30. 30© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 30© 2019 IDERA, Inc. All rights reserved. COMPARISON – CUMULATIVE OBJECTS VS. DEFECTS
  • 31. 31© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 31© 2019 IDERA, Inc. All rights reserved. COMPARATIVE Measurement Measurement Period (Weeks 1 -20) Control Period (Weeks 21 - 31) Performance Improvement Interval Length (weeks) 20 11 Objects Created 957 1,083 Defects 1,077 38 Defect Opportunities 4,090 4,333 Defect Points 1,696 87 Defect Point Opportunities 8,886 8,991 Average Objects/week 47.85 98.45 205.76% Average Defects/week 53.85 3.45 1558.82% Average Defect Points/week 84.80 7.91 1072.18% Average defects/object 1.13 0.04 3207.37% Average Defect Opportunities/Week 204.50 393.91 Defects/Opportunity 0.263 0.009 3002.60% Defect Points/Opportunity 0.191 0.010 1972.46%
  • 32. 32© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 32© 2019 IDERA, Inc. All rights reserved. THE BOTTOM LINE ▪ On time completion ▪ Avoided $3.7 million overrun ▪ Senior Enterprise Data Architect + Modeling Tools $200K • Duration of project ▪ ROI: ($3.7 million – $200K)/$200K = 1,750% • Had this been done at the beginning of the project, returns would have been even greater
  • 33. 33© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 33© 2019 IDERA, Inc. All rights reserved. WHAT IS LEAN? ▪ Has it’s basis in manufacturing, and has been adapted to knowledge work • Toyota Production System (TPS) ▪ Organizational focus vs. Agile’s software focus ▪ Repeatable process to minimize waste, maximize value ▪ Requires • Quality standards • Collaboration of specialized workers ▪ Kaizen • “kai-” (change) “-zen” (good) • “continuous improvement” or “small incremental improvements” of all areas of a company
  • 34. 34© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 34© 2019 IDERA, Inc. All rights reserved. LEAN PRINCIPLES ▪ Eliminate waste • Eliminate anything that does not add value ▪ Build quality in • Quality is everybody’s job! • Test driven, incremental development with constant feedback • Automate processes prone to human error ▪ Create knowledge • Properly document and retain valuable learning ▪ Deliver fast • Remove blockers • Don’t over-engineer ▪ Respect people • All aspects: communication, handle conflict, onboarding, process improvement • Empowerment ▪ Optimize the whole • Don’t sacrifice quality for speed • Understand capacity and downstream impact of all work • Identify and optimize value streams
  • 35. 35© 2019 IDERA, Inc. All rights reserved. AGILE VS. LEAN ▪ Agile ▪ Proposed as “a better way of developing software ▪ Bottom-up focus • Short cycle, frequent delivery ▪ Kanban usage • Fixed duration iterations • Limit time of development • Each iteration begins with a fresh board ▪ Focus is delivering software ▪ Lean ▪ Strategic as well as operational • Improve IT’s value to the organization ▪ Top-down, End-to-End Focus (E2E) • “See the whole” ▪ Kanban usage • Continuous flow • Limit work-in-progress • When a task completes, PULL the next in sequence ▪ Focus is delivering real value • (not just software) “Agile is the new Waterfall”
  • 36. 36© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 36© 2019 IDERA, Inc. All rights reserved. START OF ITERATION ▪ Participate fully in iteration planning ▪ Ensure there is a “Named Release” as of completion of previous iteration • Always have a baseline for compare/merge ! ▪ Submodels • Structure by relevant topic/subject area • At story level if necessary to facilitate communication • Roll up to parent level submodels
  • 37. 37© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 37© 2019 IDERA, Inc. All rights reserved. MANAGING ITERATIONS ▪ Always have a baseline for compare/merge ! ▪ Within iteration workflow • Model each change, associating with appropriate task/user story • Generate incremental DDL script(s) and stage to build server • Use a robust script naming convention, particularly if utilizing automated build systems • 1 data modeler may be working with multiple dev teams simultaneously • Some designs will be originated by data modeler • Others may be from developer “sandbox” − Compare/merge and redesign as appropriate − Ensure developer uses the officially sanctioned script • Create “Named Release” at end of iteration • Create delta script by using compare/merge • Based on Named Release from the previous iteration ▪ Use sub-models for audience specific perspective ▪ Maintain the discipline! ▪ Participate fully in iteration planning and retrospectives
  • 38. 38© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 38© 2019 IDERA, Inc. All rights reserved. ER/STUDIO: CHANGE MANAGEMENT CENTER - TRACEABILITY
  • 39. 39© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 39© 2019 IDERA, Inc. All rights reserved. MANAGING COMPLEXITY ▪ Have an overall plan guiding the initiative • Usually requires analysis and some modeling BEFORE development starts ▪ Some areas may be very complex, requiring multiple iterations to design/develop ▪ Use data model design patterns as a starting point ▪ The “wave” approach • Data modelers working on some items 1 or 2 iterations ahead of the development team • Logical / Physical modeling separation facilitates this • Make changes to logical model in advance • Compare/merge appropriate changes to physical at the right time • Enterprise perspective of the data ▪ Fully documented data models!! • Data dictionary definitions • Documented relationships/role names • The physical model IS the implementation • ALL physical constructs
  • 40. 40© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 40© 2019 IDERA, Inc. All rights reserved. ER/STUDIO – COMPARE AND MERGE
  • 41. 41© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 41© 2019 IDERA, Inc. All rights reserved. GENERATE SCRIPT
  • 42. 42© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 42© 2019 IDERA, Inc. All rights reserved. END OF ITERATION WRAP-UP ▪ Create “Named Release” at completion • Serves as baseline for start of next iteration • Serves as baseline for comparison at ANY later point ▪ Create delta DDL script by using compare/merge • Based on Named Release from end of the previous iteration ▪ Create full database DDL script • Can be used to easily create “sandbox” databases quickly ▪ Ensure the model(s) have been published ▪ Participate fully in planning and retrospective meetings • Lessons learned • Celebrate the successes
  • 43. 43© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 43© 2019 IDERA, Inc. All rights reserved. AUTOMATED BUILD SYSTEM CONSIDERATIONS ▪ Require synchronized deliverables ▪ Database (DDL) ▪ Application code ▪ Persistence • Data services • Framework updates
  • 44. 44© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 44© 2019 IDERA, Inc. All rights reserved. POST FLIGHT DE-BRIEF ▪ Systems development is continually evolving and improving • There have been no brand new, groundbreaking ideas • Derived from manufacturing principles and practices proven to deliver business value • Learn and adapt based on the cumulative body of knowledge • And fit to suit organizational culture ▪ DATA has ALWAYS been important. More companies are recognizing that. • Applications come and go • Companies always want to retain the data! • Data models are more important than ever in order to • Manage complexity • Increase quality • Deliver value • Avoid failure. ▪ Lean principles improve systems development • Value focus • Efficiency • Waste reduction • Customer Satisfaction ▪ Approaches utilizing lean are the most successful • Predominantly adaptive • With predictive capabilities incorporated • Best of both worlds
  • 45. 45© 2016 IDERA, Inc. All rights reserved. Proprietary and confidential. 45© 2019 IDERA, Inc. All rights reserved. THANKS! Any questions? You can find me at: ron.huizenga@idera.com @DataAviator