Mais conteúdo relacionado

Apresentações para você(20)

Similar a Keeping the Pulse of Your Data – Why You Need Data Observability to Improve Data Quality(20)




Keeping the Pulse of Your Data – Why You Need Data Observability to Improve Data Quality

  1. Keeping the Pulse of Your Da ta : Why You Need Data Observa bility to Improve Da ta Qua lity
  2. Spea kers Julie Skeen Sr. Product Marketing Manager Micha el Sisola k Principa l Sa les Engineer
  3. Agenda • Introduction to data observability • How data observability works • Use case examples • Q&A 3
  4. 47% of newly crea ted da ta records ha ve a t lea st one critica l error 68% of orga niza tions sa y dispa ra te da ta nega tively impa cts their orga niza tion 84% of CEOs sa y tha t they a re concerned a bout the integrity of the da ta they a re ma king decisions on Precisely Da ta Trends Survey Forbes Ha rva rd Business Review Da ta integrity is a business impera tive
  5. Introduction to Da ta Observa bility Business Challenges • Data downtime disrupts critical data pipelines and processes that power downstream analytics and operations • Lack of visibility around health of data reduces confidence in business decisions • Traditional manual methods do not scale, are error-prone, and are resource intensive 5
  6. Everything old is new a ga in • “W. Edwards Deming The Father of Quality Management” started the observability concept 100 years ago • Observability is a key foundational concept of SPC, Lean, Six Sigma and any process dependent on building quality into repetitive tasks Applying the same principles to data = data observability • Using statistical methods to control complex processes to ensure quality data products over time Wha t is Da ta Observa bility? 6 IDC; Phil Goodwin a nd Stewa rt Bond, “IDC Ma rket Gla nce: Da ta Ops, 2Q21”(June 2021) Ga rtner, Hype Cycle for Da ta Ma na gement, 2022, Melody Chien, Ankush Ja in, Robert Tha na ra j, June 30, 2022
  7. Why Now? 7 • Businesses a re more da ta -driven tha n ever • Problema tic events a re infrequent but ca n be ca ta strophic • User’s da ta expertise ha s evolved a long with expecta tions to do more with it • Da ta prolifera tion a nd technology diversifica tion • AI ha s evolved to support the complexity of the problem
  8. Da ta Observa bility is proa ctive, not rea ctive 8
  9. Da ta Integrity a nd Qua lity QA is done at the time of development Ra ndom issues a re surfa ced Users find a nd report defects 9 9 Typica l Da ta Products a nd Pipelines Tra ditiona lly, the qua lity of a da ta product or pipeline is ensured during the development process a nd not throughout the opera tiona l lifecycle. Da ta Product(s) X Da ta Source #1 ? Da ta Source #2 ? Da ta Source #3 ? Da ta Source #4 ? Crea te a nd/ or Source The Da ta Tra nsform Da ta Enrich / Blend / Merge Da ta Publish a n Expose Da ta P r o c e s s
  10. 10 10 Da ta Pipelines with Observa bility Da ta Observa bility tools observe the performa nce of da ta products a nd processes in order to detect significa nt va ria tions before they result in the crea tion of erroneous work product in reports, a na lytics, insights a nd outcomes. Da ta Source #1 Da ta Source #2 Da ta Source #3 ! Da ta Source #4 Crea te a nd/ or Source The Da ta Tra nsform Da ta Enrich / Blend / Merge Da ta Publish a n Expose Da ta P r o c e s s Observing ea ch sta ge in the pipeline Issues identified a nd resolved prior to fina l product O b s e r v e Da ta Product(s)
  11. 11 Da ta Observa bility Impa ct of Unexpected Da ta Da ta a noma lies ha ve downstrea m impa cts, but not every issue impa cts the process in the sa me wa y. The sooner you ca n detect a noma lies, the sooner you ca n a ssess the impa cts a nd effectively remedia te. EXAMPLE
  12. How Da ta Observa bility Works Discovery Ana lysis Action
  13. Intelligent Ana lysis Identifies Anoma lies 13 AI identifies trends tha t tra ditiona l methods ca nnot ea sily find Ra ndom Noise Upwa rd Trend Downwa rd Trend Step Cha nge 2 Step Cha nge 1 Sudden Jump Up
  14. Da ta Observa bility a nd Qua lity 14 Rules Metadata Time Data Quality Management Da ta Observa bility Focused Ca pa bilities • Alerts a nd da shboa rds for overa ll da ta hea lth trending a nd threshold a na lysis • Anoma ly detection ba sed on volume, freshness, distribution a nd schema meta da ta • Predictive a na lysis simula ting huma n intelligence to identify potentia l a dverse da ta integrity events “Observa bility is the missing piece toda y to give our da ta stewa rds a ccess to da ta discovery insights without ha ving to go to IT for queries or reports” - Jea n-Pa ul Otte, CDO, Degroof Peterca m
  15. Alerts a nd Impa cts 15 Volume Alert Impacts
  16. Use Ca se Exa mples
  17. 17 Da ta Observa bility Impa ct of Unexpected Va lues An incorrect currency type in the order crea ted a n infla ted revenue a mount which would ha ve resulted in the incorrect tota l revenue a mount. The error wa s ca used beca use the currency conversion ta ble wa s not upda ted. The Da ta Observa bility solution would notify the Da ta Ops tea m of the da ta drift so tha t they could quickly resolve the issue a nd prevent it from impa cting downstrea m a na lytics a nd rela ted decisions. EXAMPLE
  18. 18 Da ta Observa bility Unexpected da ta volumes impa ct opera tions A single-da y spike of 500% in the dolla r a mount of orders ca used beca use the compa ny expa nded into a new geogra phy without notifying a ll a ffected a rea s within the compa ny. Da ta stewa rd would receive a volume a lert which a llows them to quickly investiga te the issue before it impa cts downstrea m a na lytics a nd rela ted decisions. EXAMPLE
  19. Use Ca se Reca p 19 • Da ta a noma ly impa cted downstrea m processes • Impa ct of Unexpected Va lues ca used by a n inva lid currency type • Unexpected data values ca used by la ck of communica tion interna lly
  20. Understa nd the hea lth of your data with continuous measuring and monitoring Obta in visibility into your da ta la ndsca pe a nd dependencies with intuitive self-discovery ca pa bilities Receive a lerts when outliers a nd a noma lies a re identified using a rtificia l intelligence Resolve da ta drift a nd shift when identified by intelligent a na lysis 1 2 3 4 Enable quick remediation when issues occur by understanding the cause of the issue 5 Da ta Observa bility benefits 20
  21. Da ta Observa bility Proactively uncover data a noma lies a nd ta ke a ction before they become costly downstrea m issues
  22. For trusted da ta , you need da ta integrity Data integrity is data with maximum a ccura cy, consistency, a nd context for confident business decision-ma king Da ta Integrity
  23. The modular, interoperable Precisely Data Integrity Suite conta ins everything you need to deliver a ccura te, consistent, contextua l da ta to your business - wherever a nd whenever it’s needed. 23
  24. 7 strong modules deliver exceptiona l va lue Da ta Integra tion Da ta Observa bility Da ta Governa nce Da ta Qua lity Geo Addressing Spa tia l Ana lytics Da ta Enrichment Break down da ta silos by quickly building modern da ta pipelines tha t drive innova tion Proa ctively uncover da ta a noma lies a nd ta ke a ction before they become costly downstrea m issues Ma na ge da ta policy a nd processes with grea ter insight into your da ta ’s mea ning, linea ge, a nd impa ct Deliver da ta tha t’s a ccura te, consistent, a nd fit for purpose a cross opera tiona l a nd a na lytica l systems Verify, sta nda rdize, clea nse, a nd geocode a ddresses to unlock va lua ble context for more informed decision ma king Derive a nd visua lize spa tia l rela tionships hidden in your da ta to revea l critica l context for better decisions Enrich your business da ta with expertly cura ted da ta sets conta ining thousa nds of a ttributes for fa ster, confident decisions
  25. Questions?
  26. Tha nk you Lea rn more a bout Da ta Observa bility -integrity/ precisely-da ta -integrity-suite/ da ta -observa bility