Seven Signs You Need a Data Warehouse
•Transferir como PPTX, PDF•
0 gostou•406 visualizações
The document discusses signs that an organization needs a data warehouse. It lists 7 signs: having multiple application databases, inconsistent or duplicate data across systems, complex or changing reporting needs, data quality issues, a chaotic reporting environment with untrusted reports, and poor data that can lead to financial or operational issues. It then introduces the SEQUEL Data Warehouse as a solution that can integrate data from different sources, perform ETL to clean and transform the data, and support analytics and business intelligence tools for improved decision making.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (10)
Destaque
Destaque (12)
Semelhante a Seven Signs You Need a Data Warehouse
Semelhante a Seven Signs You Need a Data Warehouse (20)
Mais de HelpSystems
Mais de HelpSystems (20)
Último
Último (20)
Seven Signs You Need a Data Warehouse
- 1. 7 Signs You Need a Data Warehouse
- 2. Heath Kath Sr. Data Access Consultant Mike Stegeman Sr. Data Access Consultant Today’s speakers: Welcome!
- 3. Today’s Agenda • Why use a data warehouse on IBM i? • 7 signs you need a data warehouse • SEQUEL Data Warehouse
- 4. Why Use a Data Warehouse on IBM i? How does a data warehouse boost your business intelligence strategy? Sales Data Budgets Customers, Inventory, Financials Industry Data
- 5. Why would you need more than just a query tool? • You have multiple application databases • The database is complex and not well designed • Your data contains errors • Your reporting needs are complex • You have many query users What’s real? Data is growing, data is becoming more complex, and users need and want more reliable information. Why Use a Data Warehouse on IBM i?
- 6. Multiple Application Databases Sales (DB2 for i5/OS) BI Reporting Financials (DB2 for i5/OS) POS System (SQL Server) Purchasing (Oracle) You Need a Data Warehouse: Sign 1
- 7. You Need a Data Warehouse: Sign 2 CUSTNO CUSTNAME 1001 John Smith 1002 Mary Jones 1003 Chris Anderson 1004 David Perry Customer File - US CUSTNO CUSTNAME 1001 Harry Potter 1002 Jeremy Carr 1003 Penny Hayes 1004 Debbie Thornton Customer File - Canada CUSTID CUSTNAM AA234 Julie Johnson AA235 Fred Hunter AB670 John Smith BD309 Alan Jordan Customer File - Canada CUSTNO CUSTNAME 1001 John Smith 1002 Mary Jones 1003 Chris Anderson 1004 David Perry Customer File - US Files (tables) are the same, but different… Multiple instances of the same table, with duplicate key values Or different versions of the same entity with incompatible data types
- 8. Poor performance You Need a Data Warehouse: Sign 3 100M rows
- 9. Changing dimensions You Need a Data Warehouse: Sign 4 100 Smith & Jones Electrical Small Retailer Jenny Brown 100 Smith & Jones Electrical Major Retailer Rob McAdam 100 Smith & Jones Electrical Major Retailer Jenny Brown 2012 2013 2014
- 10. You Need a Data Warehouse: Sign 5 Data errors • Failed joins • Invalid dates • Missing values Difficult dates • Date format • Multiple fields for Year, Month, Day • Field types Hidden meanings • Second character of column X means… • If column Y = ‘C’ then Z * -1 • If this…then… Data challenges
- 11. You Need a Data Warehouse: Sign 6 A chaotic reporting environment! Sales FinancialsPurchasing GL Summary (Excel) Summary Sales by Customer/Brand Profitability Extract Summary Sales by Region Purchasing extract (MS Access) ~~~~~~~ ~~~~~~~~ ~~~~~~~ ~~~~~~~~ ~~~~~~~ ~~~~~~~~ ~~~~~~~ ~~~~~~~~ ~~~~~~~ ~~~~~~~~ ~~~~~~~ ~~~~~~~~ Joe downloads this manually via Client Access every Monday...except when he’s on vacation or out with the flu! Mary wrote this extract. She left last year and no-one knows how it works. The Net Sales calculation in this extract is different to Mary’s. No one has yet realized that this is loaded incorrectly. The auditors will be the first to discover the problem. These reports don’t bala These reports don’t balance with each other.No one trusts this report. John spends 5 days every month generating this and massaging the numbers until he thinks it is correct.
- 12. Poor data quality For example: • Property assessment incorrectly changed to $400M • Property tax revenue of $8M was included in the county budget • County had a huge revenue shortfall, resulting in lots of cuts • The school district forced to return $2.7M You Need a Data Warehouse: Sign 7
- 13. Quick Poll… Why do you need a Data Warehouse? Multiple databases Inconsistent data Complexity
- 14. Without a Data Warehouse… • 96% of data marts require change in the first year • 75% of independent data marts do not survive > 2 years • 60% of companies abandon their BI investment within 5 years
- 15. SOLUTION: SEQUEL Data Warehouse
- 16. SEQUEL Data Warehouse ETL Tool Data Management Manage Development IBM i Environment Metadata Repository Data Warehouse
- 17. SEQUEL Data Warehouse Development Client
- 19. SALES PURCHASING FINANCIALS O P E R A T I O N A L S Y S T E M S SEQUEL Implemented Against Operational Data Simple Implementation—Without SDW
- 20. Front End Tools Implemented Against DW/DM Tables Data Warehouse/Data Marts O P E R A T I O N A L S Y S T E M S SALES PURCHASING FINANCIALS Simple Implementation—With SDW
- 21. SEQUEL Data Warehouse: Data Access + Non-DB2 Data Sources SEQUEL Data Warehouse on IBM i Oracle, MS SQL, MySQL, Sybase XML FilesText Files (fixed length or delimited) MS Excel Salesforce Apache Hive, Impala
- 22. Share Information with SEQUEL Data Warehouse Key to formatting and distribution of data
- 23. Browser Access! Share information with SEQUEL Web Interface
- 24. Data Warehouse A solution to your business needs
- 26. SEQUEL is offering exciting spring promotions. We’d love to tell you about them! Interested? Give your sales rep a call at 800-328-1000. Special Promotions—Just for You
- 27. Heath Kath Sr. Data Access Consultant heath.kath@helpsystems.com Mike Stegeman Sr. Data Access Consultant mike.stegeman@helpsystems.com www.helpsystems.com/sequel We are ready for your questions!
- 28. Thank You for Joining Us Today! Website: www.helpsystems.com/sequel Phone: 800-328-1000 or +1 952-933-0609 Email: mike.stegeman@helpsystems.com heath.kath@helpsystems.com support.sequel@helpsystems.com
Notas do Editor
- ============ (on-line….notes) -- PPT 15 -18 minutes! Less time is better, long demo Mike -- Hello everyone. Welcome to today’s presentation on 7 Signs You Need a data warehouse As most of you know, the amount of data that we continue to collect and manage grows significantly each and every day. I am talking in the millions… The challenge today is to find the right tool, or combination of tools that will let you be successful in analyzing all that data and conforming it into meaningful information allowing you to do your business. Before we go any further, (introductions …. Next slide) NOTE: This has a Polling Question – see slide -- maybe try the new Polling tool MIke -- Hello everyone and welcome to today’s webinar – 10 Ways Your Query Tool is Failing You : During today’s presentation, we will address common frustrations why basic query tools could be failing you, discuss major data access and distribution trends, and help you understand what to look for in a more advanced query tool. …………………. Notes: --notes…. Goal: 30 minute or less webinar (PPT – Less than 20 minutes + Live Demo 10 minutes) Setup Polling: OPEN Poll #18 – Break Free from Basic QUERY – to be used Just Before Demo Abstract: 10 Ways Your Query Tool is Failing You Your query tool has been around for a while—a long while. It used to be state-of-the-art, but now you’ve started to realize that it’s failing you. Badly. Your management team and business users deserve better. In this webinar, we’ll address more than ten common frustrations with the IBM i basic query tool and show you how a modern, advanced data access tool gives you quick access to your data and helps you present it in a way that is useful to your users. Join us to learn how a data access tool like SEQUEL makes it possible to: • Empower your users with executive dashboards • Access data from other servers (including SQL, DB2, and so on) • Save your results in Microsoft Excel, Adobe PDF, etc. • And more!
- Mike: My name is Mike Stegeman. I will be your host for today’s webinar. I am a Senior Data Access Consultant with Sequel Software working in the Schaumburg, IL office outside of Chicago. I have been with Sequel for xx years. I started working on the IBM System 36; And then moved to the AS/400, I-Series, Power machines. Prior to working with Sequel, I spent many years in a variety of areas including public accounting, ERP software support, CASE tool support, support manager, IT manager. I believe my varied background gives me the ability to understand business needs while knowing about the technical aspects: The software, the database and the hardware. Today, we have another DB expert that has joined us, Heath how are you doing? Heath: I am doing great, thank you. My Name is Heath Kath. I am also a Sr. Data Access Consultant with Help Systems. I have been working on the IBM i, around data access, for over 24 years. I actually worked IBM in Rochester MN from programming to supporting Query/400, SQL, and other DB tools. Today I am excited to talk to you about an IBM i Solution for a Data Warehosue. Thanks Mike. Mike: Exellent, About today’s session, if you have questions during the session, please use the CHAT feature and send a message to All Panelists, and we will address those questions during our Q&A at the conclusion of the webinar. I am also recording the webinar and we will include a link to the recording in our follow-up email usually within a day. Now, let’s go ahead and get started. (next slide)
- Heath – perfect… Today we will be talking about A quick overview of What is Data Warehousing (DW) or What is Business Intelligence (BI) – they go hand in hand Why do YOU need a Data Warehouse…and 7 common pitfalls from: Incompatible Sources , Operational Databases initial design is a mess, Performance, and more And what is SEQUEL Data Warehouse (SDW)
- Heath – What is a Data Warehouse… or What is Business Intelligence (BI) Business intelligence (BI) is a broad category of applications and technologies for gathering, storing, analyzing, and providing access to data to help enterprise users make better business decisions. BI applications include data warehouses, data marts, decision support systems, query and reporting, online analytical processing (OLAP), statistical analysis, forecasting, and data mining. <user pointer or outline > In other words, a Data Warehouse is a is a set of BI tools and technologies that allow you to gather the data, from multiple data files and multiple databases to get from here, the raw data, to here, an informative dashboards or spreadsheets that business users need. A Data Warehouse can be an integral solution to an enterprise's day to day operations, or, occasionally when needed to meet a special requirements. They could be Enterprise-wide or local to one division, department, or project
- Heath – You may have a query tool already, but you could or have a need to do more but just are limit. Why…. Well, …
- Heath – Now, lets focus on some of the common pitfalls that could be surrounding you and your basic query tool. Sign 1 You have many different environments, one could be for your sales data, that is stored locally on the IBM i. Then you have … Financials / POS – Point of Sales data / Purchasing information… What you may find is that the data is on different structures – but the information is related. It’s very difficult, if not impossible to join tables across databases. Maybe you have run into different levels of security and availability – all of which will add significant complexity, And then you have the challenge of trying to get all that information in a timely matter--- Are you running into poor performance that could be costing you money?
- Heath – You could also be Working with multiple files or tables, they are very similar, however, we find this to be very true: Duplicate key values…. CUSTNO 10001 is John Smith in the US file, but in Canada, Harry Potter is customer # 10001 Incompatible data types…. CUSTNO is numeric in one file but the CUSTID character in another.
- Heath – A third common issue that we are faced with is PERFORMANCE, and I find this to be a very common issue. The files, the amount of data being collected is only growing.. You may have a Large transaction table and Many related tables You have Several different reports, most could be summary level information but it could be collecting data over one or several of these tables. When running the Reports and queries, you may have many existing customers accessing the information, and maybe you as well,, are seeing longer run times, and the jobs could be consuming considerable amount of system resources. So are others also being impacted?
- Heath - The Fourth sign, Changing Dimensions e.g. customer attributes such as group, territory are periodically changed What happens if…. I need to re-run a report from 2012 Many tools do not have the capability to reproduce the same report since the original groupings (customer group, territory) are no longer available in the operational database Maybe you need to compare This Year sales against Last Year for all sales reps Same problem… do you have the data, are the files still consistent?
- Heath – Another common issue that we find is that Files and fields are different (field length, field types, numeric date vs character type date fields and more) Maybe some files have a 6 character field names (RPG III legacy formatting where) First 2 characters of the file is the prefix – so only 4 characters left for the actual field name The databases that we work on, were designed to support transactions, not query access! The Use of 3rd normal form to avoid redundancy resulted in many tables. Date fields; One file may have a numeric or character type date as YYMMDD, another MMDDYYYY, or the date might be in a Julian format To many fields… some files could have hundreds of fields, but yet most of the time, only 10% of the available fields are useful for BI reporting Or simply, bad data. Invalid data stored in the record
- Heath – What we fond in various studies Sign # 6 – then you have Independent ‘stovepipe’ data marts Built independently without any coherent plan – basically they Do not agree with each other. A lot of Duplicated effort by the users. Usually undocumented data structure, and it’s a major maintenance headache that doesn’t go away. <user pointer> Which leads to: reports and spread sheets that are not trusted, reports that do not add up, a lot of wasted time verifying and correcting the data… hoping it is correct, you also have: Joe who spends hours each day manually copying and pasting the data into an excel spreadsheet, Mary who has a very unique…her own..process that she runs - but she is out of the office, maybe on a vacation or is no longer with the company – now who is supposed to run it let alone even know where to start since it’s Mary’s process, and the last example,…is your data loaded correctly and do you have a good trail of all your transactions for the auditors? Notes:------------------------- Poor Data Quality Very little if any validation and error management in the load process Inconsistent calculations and business rules Slow, takes too long Inflexible Very difficult to modify to cater for emerging business needs Leading to… Bad decisions, based on incorrect or incomplete data Eventual lack of trust, leading to disuse A Disillusionment with Business Intelligence
- Heath – Prime example, back in 2005 Valparaiso, Indiana - somehow a property assessment value for the home shown below was incorrectly changed to $400M in the property tax database The expected property tax revenue was included in the county budget - but the $8M property tax bill on the house was (of course) not paid Thus, the county had a huge revenue shortfall since they were budgetign for the $8m, resulting in lots of cuts The school district was forced to return $2.7M All extracurricular activities and sports were cancelled that year… once big mess that could have and should have been easily prevented.
- If we had more time, I would have a few more questions, but today, we our curious: Do you have complex or to many files finding them hard to manage across the company? Are you struggling with inconsistent data? Do you have files (data) on multiple databases? Location of polling question: \\nt5master\training\5 - E-Training Classes\_WebEx\WebEx Polling questions\SEQUEL Webinars Poll [2015 - 01] Why SEQUEL Data Warehouse
- Heath – from a few various studies ,,, 96% of data marts require change in the first year (usually requiring them to be totally rebuilt) 75% of independent data marts do not survive past first two years and 60% of companies without a data warehouse architecture abandon their BI investment within 5 years, citing maintenance complexity and cost as the prime reasons
- Heath – A solution - SEQUEL Data Warehouse, software that was previously known as RODIN as been around since mid 80’s that runs on the natively on the IBM i. SEQUEL Data Warehouse is an advanced, fully integrated, visual development environment primarily designed to define, help you build, load and manage data warehouses, data marts, and operational data stores (ODS) for use in Business Intelligence applications. Similar tools are often referred to as Extract,Transform, Load (ETL) tools. As shown, these tools can help across your organization from Inventory, payroll, production….. Providing profitability from supplier and strategic performance and productivity. SEQUEL Data Warehouse also includes an extensive Management layer that will monitor and control the development and execution of these ETL processes; really ensuring a very high degree of data integrity. Hence SEQUEL Data Warehouse is a fully integrated development and deployment environment for ETLM applications or as we call it -- "Data Engineering". Notes..omit: ------------- It is also important to realize that SEQUEL Data Warehouse is not just suitable for data warehousing applications, although it was specifically designed for and excels at that task. Because it can basically do anything in relation to selecting, manipulating, validating and moving data of all types, it has many other uses. For example, our customers use SEQUEL Data Warehouse to undertake data conversion projects when migrating from one software package to another. They also use it to move and integrate data between applications that aren't currently integrated (e.g. to update last payment date onto the customer master file whenever a payment is made) or to provide data such as price lists or inventory updates to suppliers and customers (and vice versa). There are ever increasing requirements to supply data and information about your business to government and regulatory agencies. This is an area that often re-directs valuable development resources away from core business applications. These and many other requirements can be easily handled by SEQUEL Data Warehouse, often within minutes, hours or days as opposed to days, weeks or months.
- Heath – SDW – It is a self-contained native IBM i run-time environment It is a data Extraction, Transformation and Load tool (ETL tool). It is a Data Management tool to define and maintain user-defined Data Warehouses, Data Marts and Operation Data Stores. It is based around an active metadata repository And, It includes a fully managed development environment, used to define and maintain the Tables, ETL Definitions and Metadata It allows you to retrieve and consolidate data periodically from source systems into a dimensional or normalized data store. Notes: a data warehouse is a system that retrieves and consolidates data periodically from the source systems into a dimensional or normalised data store. It usually keeps years of history and is queried for business intelligence or other analytical activities. It is typically updated in batches, not every time the transaction happens in the source systems.
- Heath – Visual representations used where appropriate to help in understanding and building the environment. We have Wizards that guide you through step by step to create an object or definition
- Heath – Powerful, easy to use visual development environment --- allow you to simply Drag and drop, double click, or do right click for options… Also, SDW includes an Extensive set of audit reports, error reports and metadata reports - simply, every step you make is being tracked
- Heath – One more time - Without SDW, users are forced to pull in data from multiple unstructured databases… where users are up against: Many different files Different field types Poor performance Files that are constantly changing Data on multiple databases (or platforms) Bad data Or A data environment that’s to complex to understand
- Heath – With SDW, you have a structured environment, data is being managed and accessed in a controlled manner.
- Heath – AND With SDW, you can also included non iSeries data… data from an Oracle, MS SQL , and other databases.
- Heath – With SEQUEL software, you can take the information that was gathered by SEQUEL Data Warehouse and distribute several different ways. From customized dashboards Customized reports Let the users export on demand, or from a scheduled job the data that they need and how they need it – maybe as an excel, csv, pdf, or other PC document types. --------------------- Notes: One is though reporting. This is an area where SEQUEL really excels. Reporting and distribution are very important because not all information is displayed on a screen. SEQUEL’s report formatting tools let you change fonts, colors, insert pictures and graphs, and save the results. The result—each time you access your information, it looks great! Reports can be initiated by a user, called from a program, or from a scheduled job. You can send customized reports to many users using a list based process. Reports can be displayed; printed; saved to a folder (PC, network, IFS); delivered using FTP; and sent via e-mail, including attachments. We all have different needs and preferences and SEQUEL is designed to meet them. Some people will want results in text format, such as text files (.txt) and rich text files (.rtf). Others may want XLS (Excel), PDF, XML, HTML, or CSV formats. SEQUEL handles all these plus other standard and custom types.
- Heath – Another option, access the data from a browser or mobile device! Even from the web, you can get your business data, and you have the options to display, export as an PC type document, and email the information on demand.
- Heath – A BI tool like SEQUEL Data Warehouse, can help your business manage, verify, control, convert data, and distribute meaningful information to the users that need it, how they need it, and when they need it. No more waiting around, no more worrying if the data is correct. The information will be correct and the data will be displayed in a modern way with SEQUEL.
- Mike - A couple of quick announcements: 1) HelpSystems is a diverse company with solutions for system automation, document management, security and business intelligence. This June in Minneapolis, we are holding our user conference. This will give you the opportunity to meet several Help Systems employees and network with other customers. You can gain more product knowledge through a large selection of sessions.
- Mike - AND today and for short time we are offering some special deals… give us a call if intereted.
- Mike: Heath, let’s take a look at the questions that came in during todays session. And if you still have a question or two, please send them to all panelists. Please keep in mind that we only just scratched the surface in showing you some of SEQUEL’s capabilities. Lets take a look…. Heath: Mike as you are looking for questions…I am going to bring up another quick poll at the same time, (bring up another POLL on top of this slide ): The results will be kept to us, we will not share them. We are curious… How helpful was today session and is there another topic or area on BI solutions, data access, dashboards, or even general topics on the IBM I that you would like additional information…. Maybe we some information that could help or we could possibly build a presentation to. Once you are set, press the submit button to complete the survey. You response will not be shared back, this is to help us. Poll [2016 - 02] IBM i 101 ( Do Not Share!) Did you find today’s overall content helpful (1 to 5, 5 being very beneficial) (select 1 thru 5) What other areas or subjects would you like to learn more about? Questions - fillers: Sample QUESTIONS: Do you need SEQUEL to run SEQUEL Data Warehouse? No…. Do you offer assistance with installation and implementation? Yes, we offer several different levels of training and assistance to help you get started. Does SEQUEL include the ability to create Dashboards? Yes, with SEQUEL you will be able to create customized views, reports, and dashboards through a nice easy to use graphical interface.
- Mike: I want to thank everyone for attending today’s presentation. We hope that you now have a good idea if you could benefit from a data warehouse solution. If your users are struggling in accessing, managing, and distributing the data because of incompatible field types, complex structure, or just because it’s too slow. You may need to take a closer look at SEQUEL Data Warehouse. If you have any further questions, or would like to see more of SDW or SEQUEL, give us a call, or email Heath or I – we would be glad to schedule a demonstration with you or your team. We appreciate your time and look forward to seeing you in futures Webinars. This concludes our session on: Seven Signs you need a data warehouse. Thank you, and have a great day! ** STOP RECORDING