TCO: An Achilles Heel of Hand-Built Data Warehouses
•Transferir como PPTX, PDF•
1 gostou•1,332 visualizações
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (6)
Destaque
Destaque (20)
Semelhante a TCO: An Achilles Heel of Hand-Built Data Warehouses
Semelhante a TCO: An Achilles Heel of Hand-Built Data Warehouses (20)
Mais de Kalido
Mais de Kalido (12)
Último
Último (20)
TCO: An Achilles Heel of Hand-Built Data Warehouses
- 1. 1 July 9, 2013© Kalido I Kalido Confidential I July 9, 2013 TCO: An Achilles Heel of Hand-Built Data Warehouses
- 2. 2 July 9, 2013© Kalido I Kalido Confidential I July 9, 2013 Three Data Warehousing Achilles Heels Time-to-Value Time-to-Respond to Change Total Cost of Ownership
- 3. 3 July 9, 2013© Kalido I Kalido Confidential I July 9, 2013 High TCO Is A Cause/Effect Outcome Lengthy implementation cycle Built on 1-2 year old requirements and obsolete the day it goes live Time-to-adapt to change is just as cumbersome as the initial implementation “The business changes its mind too quickly for IT to keep up.” n= 544 responses from 430 companies “The Gap”
- 4. 4 July 9, 2013© Kalido I Kalido Confidential I July 9, 2013 Drivers Of High TCO People costs are not reduced over time Lack of automation means hand coding changes as occur Frequency of changes drives higher costs in a hand-built warehouse Under 100K $100-249K $250-499K $500-999K Over $1M 77 110 135 68 137 “How much do you spend annually to support the data warehouse?” n= 527 responses n= 546 responses 430 companies “How much did you spend on ETL tools to support you B.I. or data warehouse project?” • Nearly 50% spent over $500K • Nearly 40% spent over $1M 2012 online poll
- 5. 5 July 9, 2013© Kalido I Kalido Confidential I July 9, 2013 Two Ways You And We Drive TCO Down
- 6. 6 July 9, 2013© Kalido I Kalido Confidential I July 9, 2013 Delivering A Better Outcome
- 7. 7 July 9, 2013© Kalido I Kalido Confidential I July 9, 2013 Business requirements that change? “That does not keep me up at night” Nik Green
- 8. Preparing Your Organization for BI Automation Kalido Summer Series: Find Hidden Costs in Your Business Before They Find You
- 9. Framing the Conversation & The Power of Relativity “…humans rarely choose things in absolute terms. We don’t have an internal value meter that tells us how much things are worth. Rather, we focus on the relative advantage of one thing over another, and estimate value accordingly.” – Predictably Irrational, Dan Ariely Which one of these circles is bigger than the other?
- 10. What constitutes art is relative…
- 11. especially when your kid is the artist
- 12. Nik: “Do you have slowly-changing dimensions?” Local DW Guru: “What’s that?” YOU DON’T HAVE A DATA WAREHOUSE!!! Nik: “How do you handle surrogate key management?” Local DW Guru: “Why would I use surrogate keys?” Nik: “How do you manage referential integrity?” Local DW Guru: “The database handles it.” Nik: “How many subject areas do you manage?” Local DW Guru: “One.” Nik: “Are your dimensions conformed?” Local DW Guru: “Heh?” A Data Warehouse Is Not a Relative Concept
- 13. “I don’t mean Data Warehouse the way you do.” Offering to automate what I don’t do today or even understand is a silly proposition. We Have to Baseline Our Organizations on a Definition “Can I go see it, the Data Warehouse?” “Teradata, our Data Warehouse…” “Nik, I know you’re going to get mad, but we should just…” “You know, Data Warehouse, a big database”
- 14. Modeling Star and Snowflake Schema Physical Schema Management Slowly Changing Dimensions Data Mart and Aggregates Data Load and Index Management Rollup Path Awareness Incremental Summary Generation Convert Existing Logical Models Name & Label Management Release to Production Version Management Object Level Change Management Model Migration Generic Export/Import for Data Migration Model Comparison Report Object Level Dependency for Migration Versions Testing Built-in Integrity Checking Aggregate Task Results Excel Integration for User Reconciliation Data rollback and Batch Reload User Interface for Data BrowsingOperations Task Execution & Monitoring Deployment & Migration Audit and Logging Process Automation Archiving Job Definition with Dependency Data Integration Data Sourcing and Field Mapping Data Detection Data Validation Code Management and Lookup Suspense and Exception Handling Currency and Units of Measure System Key Management Post Processing Housekeeping BI Delivery Native XLS Pivot Table Generation Metadata Management Report-Time Formula Management These capabilities enable building a scalable Business Intelligence platform.
- 15. Imagine extolling the value of a snow blower to him…
- 16. Now try him…
- 17. Business Engagement 10% Logical Modeling 20% Physical Modeling 40% BI Metadata Development 15% Mapping 15% Model Distribution of Effort
- 18. Business Engagement 10% Logical Modeling 20% Physical Modeling 40% BI Metadata Development 15% Mapping 15% 55% Effort Reduction
- 19. …humans have consistently demonstrated an ability to find new things to do that are of greater value when jobs have been outsourced or automated. – Philip Rosedale (Creator of Second Life), Abundance
- 20. Business Engagement 55% Logical Modeling 30% Mapping 15% Reinvest this time in engaging the business and building a better model.
- 21. Shifting Your Talent Business Acumen Communication ETL SQL Dimensional Modeling Data Analysis RDBMS Business Acumen Communication ETL SQL Dimensional Modeling Data Analysis RDBMS
- 22. • Frame the conversation…make sure everyone is talking about the same thing. • Reinvest your time-savings in developing a better solution. • Re-tool your people.
- 23. © 2013 Kalido I Kalido Confidential I July 9, 201323 Thank You! Is a relatively high Total Cost of Ownership still an expected consequence of an Enterprise Data Warehouse? TCO Summer Webinar Series Cost Per Release Cycle – 7/19 Automation to Reduce Operating Costs – 7/23 Reduce Tool Cost – 7/30 Scale Drives Cost Reductions – 8/6
Notas do Editor
- These three variables seem nearly always to be in a discussion about failed or sub-optimal data warehouse implementations. Too long to get to value. Too slow to adapt to change. And too expensive over a long period of time.Time to value is often measured in years rather than months. In other words that measuring stick of value-----business improvement------which is enabled by the best information being available for consumption by business processes or analytics, takes too long to get to. If ever. Responding quickly to change is an aspirational hallmark of most organizations today. Whether the change is market driven, or merger/divestiture driven, or driven by a change in the fundamental structure of a business it is the speed that effects of change can be interpreted and acted on that reflects a key element of value in a data warehouse. If the reports or analytics can’t be updated in time to assess and exploit opportunity or challenge while it is there, the value is diminished.And finally, TCO. Hand built warehouses are expensive to build and maintain, because adapting one to accommodate change in the business is pretty much the same as implementing one. So you need the same talent pool of ETL programmers and support people that you needed to build it in the first place. You don’t accrue economies over time through incremental changes and adjustments made by two or three people--------you accrue costs to rebuild each time you make a change.Experts agree that hand building data warehouses is expensive, time consuming and complex. Speeding the process up in the words of one of those experts Ralph Hughes is “simple but not easy”.(NEXT SLIDE)
- That Gap is still alive and well when we talk about creating a data warehouse with value for the business. In other words, one that meets CURRENT requirements rather than those that were generated years ago.We wanted to find out how prevalent this gap was, so we went and asked. We polled nearly 550 practitioners and business people at TDWI events throughout the year.Data validates what we intuitively know. Business change occurs at a pace that IT capabilities have trouble matching. Some of the comments that we picked up along the way included:“We are pretty well aligned with our IT group, but either the technologies or processes they are using can’t provide the response we need.”“We use the word “agile” a lot, but in reality if we can’t extensively plan for a change, it is difficult to make one.”“We set up our own data engine to support our most critical needs.” (shadow IT)In the research we’ve conducted and sponsored and the interactions throughout the year with clients, three themes really do continue to come up over and over again.We refer to these as the Achilles Heels of Data Warehousing. (NEXT SLIDE)
- Talk About Ralph’s 3-2-1We actually questioned the responses on this slide a bit. Many of the people we polled shared with us anecdotally:“I’ll be guessing at costs, because a lot of the people aren’t in my department.”“I’m pretty sure that we’re way over a million dollars/year, but I’m embarrassed to say that.”“We set up our own data mart because IT couldn’t deliver and we have two guys running it.”“I know there are a whole bunch of consultants in the building but I don’t know what they get paid.”“I’m pretty sure the warehouse was never finished so I guess we don’t spend anything anymore.”Scary stuff. A lot of people just don’t know what their costs are, and then there are those who have divorced themselves entirely from an IT sponsored and delivered data warehouse because it doesn’t meet requirements. In either case it’s a net loss for the organization.Here’s some sobering data though….. The costs of maintaining a hand built data warehouse generally do NOT reduce over time. There are very few efficiencies built into the warehouse because it must be hand built every time a change occurs. The frequency of changes directly drive those costs. There is no such thing as an incremental change and incremental expense.The simple fact is that through better use of automation the dependency on staff is reduced and the dependency on 3rd party tools to support hand built development is reduced.Here are two data points from two studies.First is a poll that Kalido ran. We asked the questionThrough another piece of research we sponsored with Nucleus Research, we found that for a company of 2500 employees who spend 15% of their time locating, assembling and analyzing data and where the efficiency of the processes is “fair” where “fair” is just below the midpoint for efficiency, the negative economic impact on OPERATIONS is $2.8M per year. That’s on business operations alone without any data warehousing operations or technology costs factored in. So making incremental corrections to how efficiently your company manages its data, integrates new sources, adapts quickly to change…….all that compounds into some rather large numbers.
- Kalido customers are nearly 4X as likely to spend less than $100K on additional software for the warehouse than the TDWI World Conference survey respondents.TDWI World Conference respondents are nearly 4X more to spend over $1M.
- Without a frame of reference, the conversation has to be different. This guy is unfamiliar with the plight of the …, so first you have to explain the problem that you are trying to solve, and then you can discuss solving that problem faster and cheaper.Transition: Now that we have framed the conversation, we can talk about automation.
- Relay story of current Data Warehouse project.Started with a scope that did not include Customer or CRMBudget was already lockedBrought in automation capability and expanded the scope.Enabled us to move in lock-step with the development of requirements