O slideshow foi denunciado.
Seu SlideShare está sendo baixado. ×

The Modern Data Platform - How to Conquer a New World with Old Problems

Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Anúncio
Carregando em…3
×

Confira estes a seguir

1 de 22 Anúncio

The Modern Data Platform - How to Conquer a New World with Old Problems

Baixar para ler offline

As Hadoop and Big Data technologies take root in the enterprise, they bring new challenges for data orchestration and interaction with traditional environments. The journey of transitioning to a modern data platform involves not only ingesting and offloading data from multiple sources, but also transforming the data, making it instantly available to the business, for on demand analytics. Manual efforts and scripting are no longer practical and projects seem to come to a grinding halt due to failures and delays.
Join this session by Cloudwick and BMC and learn through real-world examples of a proven approach that meet the required standards of ease of use, scalability, security, data governance and service level management.

As Hadoop and Big Data technologies take root in the enterprise, they bring new challenges for data orchestration and interaction with traditional environments. The journey of transitioning to a modern data platform involves not only ingesting and offloading data from multiple sources, but also transforming the data, making it instantly available to the business, for on demand analytics. Manual efforts and scripting are no longer practical and projects seem to come to a grinding halt due to failures and delays.
Join this session by Cloudwick and BMC and learn through real-world examples of a proven approach that meet the required standards of ease of use, scalability, security, data governance and service level management.

Anúncio
Anúncio

Mais Conteúdo rRelacionado

Diapositivos para si (20)

Semelhante a The Modern Data Platform - How to Conquer a New World with Old Problems (20)

Anúncio

Mais de DataWorks Summit/Hadoop Summit (20)

Mais recentes (20)

Anúncio

The Modern Data Platform - How to Conquer a New World with Old Problems

  1. 1. © Copyright 2014 BMC Software, Inc. 1 — The Modern Data Platform How to Conquer a New World with Old Problems
  2. 2. © Copyright 2014 BMC Software, Inc. 2 BMC & Control-M Leader in Digital Enterprise Management 98 of Forbes Global 100 & 80%+ of Fortune 500 Workload Automation – #1 Market share – #1 Analyst Recognition – Big Data leadership – Customer satisfaction
  3. 3. © Copyright 2014 BMC Software, Inc. 3
  4. 4. © Copyright 2014 BMC Software, Inc. 4 DIGITAL WORLD WE LIVE IN A
  5. 5. © Copyright 2014 BMC Software, Inc. 5 of S&P 500 will be replaced by 2027 75%25%of businesses will lose competitive ranking due to digital incompetence by 2017 The future is here. Go digital or go extinct. © Copyright 2016 BMC Software, Inc.
  6. 6. © Copyright 2014 BMC Software, Inc. 6 Progress means building upon … not leaving behind
  7. 7. © Copyright 2014 BMC Software, Inc. 7 How will Enterprise IT manage Big Data?
  8. 8. © Copyright 2014 BMC Software, Inc. 8 © 2015 Cloudwick. All rights reserved. Confidential and Proprietary. Build, Operate & Manage Focused on Three Key Phases Assess & Plan Architect & Engineer Optimise & Scale Size, Environment & Use Cases Secure, Govern & Data Management Applications, Analytics & Visualisation
  9. 9. © Copyright 2014 BMC Software, Inc. 9 © 2015 Cloudwick. All rights reserved. Confidential and Proprietary. 7 Founded in 2010 200+ Data Engineers, Developers & Data Scientist 50k Hadoop Clusters Engineered & Operated On-Prem & Cloud 3 Global Regions Americas | EMEA | APAC 300+ Big Data & Security Enterprise Engagements 6 Strategic Partnerships Cloudwick Open Source, Big Data, Advanced Analytics & Cloud
  10. 10. © Copyright 2014 BMC Software, Inc. 10 Data Accessibility Typical Enterprise Challenges Unaffordable storage at Scale Data Ingestion / ETL Security Data Governance & Management Data Quality & Performance
  11. 11. © Copyright 2014 BMC Software, Inc. 11
  12. 12. © Copyright 2014 BMC Software, Inc. 12 Building on each stage of the Foundation Data Science ML, IoT, AI BI/Analytics Predictive & Prescriptive Reporting Transactional, Operational, Ad-hoc & Self Service Management Strategy, Govern, Quality, Platform, Multi-tenancy & Operations Consolidation Ingest, Cleanse/ETL, Store, Secure
  13. 13. © Copyright 2014 BMC Software, Inc. 13 Who holds the Key?
  14. 14. © Copyright 2014 BMC Software, Inc. 14 IT • Enabling the Answers • Reporting: Batch to Streaming • Operational foundation & management • People: Resourcing and developing • Optimise: Savings Focus • Design: Govern, Access & Secure Business • Driving the Questions • Demanding: Mining & Exploring • Immediate & Scalable • People: training & accessing • Generating: Revenue Focus • Enable: Govern, Access and secure Aligning Gaps
  15. 15. © Copyright 2014 BMC Software, Inc. 15 Prioritising Use Cases
  16. 16. © Copyright 2014 BMC Software, Inc. 16 Batch to Streaming & Beyond
  17. 17. © Copyright 2014 BMC Software, Inc. 17 STABILITY COMPLIANCE COST EFFICIENCY
  18. 18. © Copyright 2014 BMC Software, Inc. 18 Workflow Construction • Drag and Drop makes building workflows simple • Native out-of-the-box integration for traditional and big data applications and technologies • Same Look & Feel HDFS Java MapReduce Pig, Hive, Sqoop Spark Distcp Tajo, Streaming File Transfer Informatica DataStage Business Objects Cognos Oracle Sybase SQL Server SSIS PostgreSQL z/OS Linux/Unix/Windows Amazon EC2 / VMware NetBackup / TSM SAP / OEBS / Peoplesoft
  19. 19. © Copyright 2014 BMC Software, Inc. 19 AUTOMATION AS A COMPETITIVE WEAPON
  20. 20. © Copyright 2014 BMC Software, Inc. 20 Sky Q and Control-M © Copyright 2016 BMC Software, Inc.
  21. 21. © Copyright 2014 BMC Software, Inc. 21 21 Navistar’s OnCommand™ service reduces vehicle downtime by 30%  Reduced unplanned repairs by up to 30%  20M+ data records/day collected and transformed from more than 160,000 trucks  20,000 Control-M jobs daily  20% improved time savings for Hadoop engineers  Actionable data sets delivered 5x faster “We’ve just begun to tap the power of Control-M to help us use big data to enhance remote diagnostics, improve vehicle quality, and protect critical resources from unauthorized access, among other initiatives.”
  22. 22. © Copyright 2014 BMC Software, Inc. 22

Notas do Editor

  • Introduce myself and a few words about BMC
    Big Data project is a journey
    This is not a simple process -  right tools, the right skills and even the right culture
    Enterprise-grade implementations =  complex (Data ingestion, Processing, Analytics
    In this session : Cloudwick and BMC sharing experience
  • WE have been in this business of Digital Enterprise Management for over 30 years now
     Workload Automation or Scheduling – providing end to end automation of business process across the enterprise
    MF, Unix, Linux, ERP, File Transfers, Java, Informatica, Cognos, Hadoop and so on and so on
    Analysts recognition
  • A personal story – Vietnam
    Digital Transformation
    Technology adoption
    Customer expectations
  • It may be  cliche, we are truly living in a digital world
    Every speaker nowadays talk about the use of the mobile (flights, check-in, hotel, etc)
    Online doctor appointment via video on a mobile app
    Oh, and yes occasionally I will use it also for phone calls
  • Going digital is not an option – it’s a must to maintain and grow the business
    You can see the numbers
    Davos World Economic Forum  – The chairman of BBVA Talking about the Digital disruption …
    20,000 banks in the world and many will disappear – those that will not adopt new technologies as he called it – the Uber moment
  • So, let’s drop all the technologies we have been using for decades and go digital? Yeah? Really? Hmm… not quite
    Anyone recognizes the city in the photo? New York, 1904 , what happened in 1904? The first subway was operated ..
    But the original subway cars have been replaced, other things like plumbing, lighting, air conditioning, security have been improved as well
    Today, the subway is still the heartbeat of the city, moving over 4 million people every day, and tying buildings together
    New York City is a great example of building upon layers, rather than leaving them behind
    Modernizing the legacy subway infrastructure and at the same time adding the newest modern architectures - like an enterprise IT environment
    When going digital or specifically implementing Big Data and IOT initiatives,  companies cannot drop existing technology and replace with new
    Like the subway in New York, the strong foundation, is what really allows the modernization to success and flourish
  • So, when we talked to companies implementing Big Data we realized that the challenges they are facing …
    How do I schedule my Big Data workflow? how can I identify something went wrong and fix it on time?
    How do I assure my SLAs are met? ready for changes? Integrate? secure? scale?
    I will talk more about it and about our scheduling solution that helps you address all of these challenges, in a couple of minutes but first, let me turn over to our partner Tina Diamond from Cloudwick who will share from her experience and of how they have been helping customers in these areas
  • How will Enterprises manage big data -

    It is about Building a Data Platform for efficiently, optimised for scale
    Enterprises Require: Security, Access and Control with optimum Performance
    Scalable architecture in the platform and software
    Built on a flexible infrastructure
    Advanced analytics and Apps to grow the business

    *I would like to introduce C
  • I am the MD, I joined Cloudwick, a US based OS PS in Jan 2016

    Focused on OS, Big Data& Advanced Analytics, delivering On prem, hybrid and cloud environments to Enterprises

    *Share old world challenges with new world solutions:

    What is our Differentiator: Cloudwick was founded on BD, it is our DNA
    Core/Foundation: Network & Consultants
  • These are not new challenges
    New world challenges with Old World Problems still includes:
    Building a platform in order to access their data
    Security
    Governance: with access and control
    Predictive / Prescriptive Analytics

    Example of a UK customer that Transformed these challenges : UKDA
  • UKDA – They used old existing way to base the new security… what happened, completely locked down, couldn’t get data in or out

    UKDA – Largest Government Research Agency for Social Sciences
    Security – 0 Data Loss – DR strategy took time
    Hybrid environment accessing data – security –
    Synergy between platform and analytics
    Strategy: share their architecture and data with other research compn
    Going a bit deeper, sharing where The most time and focus was spent understanding and building this solution
  • Just like in Maslow’s model, each lower stage is a necessary foundation for the stages above it.
    If any foundation is weak, the higher stages may crumble and leak even if they initially work. Resources for building each stage involve a combination of human talent and the right set of tools for your needs.

    Collect: For a robust data pipeline, you want access to clean, reliable data then manage for - Govern, Quality, Platform, Multi-tenancy & Operations
    Manage: Once you have a consistent stream of clean data, you need to have a clear understanding of the data’s composition. Descriptive statistics is where you look into the past and ask questions that have concrete answers to them.

    TR: The other question I’m always interested in ,… who holds the key to building?
  • Both IT & Business hold the key to unlock the value – the goal is the same, to secure, govern allow the individuals that have signing authority to access – once that is there, the individual can access as often as they like –rely on their valuable assets being in the safest and most secure location for them, with back up strategies and insurance in case of a robbery

    Business understands what data they want to get to
    IT unlocks and makes sure it’s secure and accessible to the business users

    Transition: Once the keys are established, how do you build alignment?
  • IT builds from the bottom up to enable business to drive from the top down
    IT Builds for growth, Business demands growth
    Automated technology is allowing the Gap to close
    Though still about understand the old problems with new solutions
    Business is becoming more savvy with Tech
    IT are becoming aware of the business

    AZ…. Pharma that has both
    Once the synergy and conversations unfold, use cases start to emerge
  • How to you…. Determine the Business value
    I see it by prioritsing Use Cases, and calculating 4 areas:
    Metrics to improve: application, activation, retention, revenue, referral, operation
    Expected improvement: 10x or more, 2x or more, 1-2x, 1x, <1x
    % of users affected: 100%, 80-99%, 50-79%, 20-49%, 10-19%, <10%
    Frequency of Use Case: Daily, few times/week, weekly, monthly, few times a year, once a year or less
    Calculate:
    As the previous slide showed, it’s a staged process, building upon each other, going from….
  • Once you implement batch use cases, streaming will follow and accelerate the value for your compeletive edge

    Global Copyright infringment: Use Case / Batch – got them down from 4 hours / match to 1 second
    Next implementing Kafka use case

    Disrupting revenue streaming –
    Streaming is 50 days?
    TR: At this time, I’d like to hand over to Alon to share the BMC offering he helps his customers with
  • We talked the challenges organizations implementing Big Data have and the many of these challenges have been around for a long time
    business keeps throwing things at us and it boils down to the needs of the business day
    With Control-M we are uniquely positioned to help our customers in reconciling these challenges
    Being consistent across all the technologies
    EDW offload job or Hadoop Pig or Hive or Scoop or Impala or any other tool to load and manipulate data
    Beyond Hadoop
    Self Service
    Cloud
  • The process for building the workflows - Simple drag and drop 
    For example - transfer data, running a Hadoop process like scoop or MapReduce, a few in parallel, creating data and then perhaps another file transfer + apply SLA
    That’s it - simple as that to put a workflow together 
    No need of scripting or the headache to integrate all apps 
  • Automation is a must  in order to support today’s modern application delivery
    Scripting is just not good enough
    The more I talk to orgs implementing Big Data, the more I hear about time wasted on scripting and failures - it doesn’t need 2 b this way
    What we offer - automate each in every step of your big data journey, from the data ingestion process, through data processing and analytics
  • Sky, a British telecommunications – using Big Data  to cost-effectively deliver their media services and content
    Converted from Jenkins to Control-M (management, speed, reliability)
    Uses Control-M for Hadoop to run customer analytics (make realistic estimations of what people are watching)
    Estimates a 40% reduction in time from Dev to Prod by replacing Jenkins with Control-M in Development phase
  • Navistar, a $10B company who is on the latest frontier with big data and the Internet of Things (IoT)
    OnCommand Connection system is used to monitor the performance of more than 160,000 trucks
    It captures millions of data points from vehicles, including fuel economy, geo location, idle time, and potential failures
    They can diagnose issues as they develop and provide guidance to fleet owners—helping them route trucks to dealer service centers
    This means less time in the garage and more time on the road, and that translates into a higher return on the investment in the vehicle

×