How Experian increased insights with Hadoop
•Transferir como PPTX, PDF•
1 gostou•925 visualizações
This document provides an overview of MapR Technologies and their products. It discusses how MapR helps companies harness big data by providing an enterprise-grade distribution of Apache Hadoop that includes data protection, security, and high performance capabilities. It also highlights MapR partnerships with companies like Syncsort to provide data integration, migration, and analytics solutions that help customers derive more value from their data.
Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a How Experian increased insights with Hadoop
Semelhante a How Experian increased insights with Hadoop (20)
Mais de Precisely
Mais de Precisely (20)
Último
Último (20)
How Experian increased insights with Hadoop
- 1. © 2015 MapR Technologies 1
- 2. © 2015 MapR Technologies 2 Today’s Presenters Bill Peterson Director - Product Marketing @thebillp Jorge A. Lopez Director - Product Marketing @zanilli Tom Thomas Sr. Director – IT, Consumer Information Services
- 3. © 2015 MapR Technologies 3 Industry Leaders Compete and Win with Data1TREND More Data Beats Better Algorithms Collecting interaction data from ecommerce, social media, offline, and call centers enables a “customer 360 view” and consumer intimacy Competitive Advantage is Decided by 0.5% Consumer financial services: 1% improvement in fraud detection means hundreds of millions of dollars Advertising and retail: 0.5% improvement in lift means millions of dollars increase in profitability
- 4. © 2015 MapR Technologies 4 Big Data is Overwhelming Traditional Systems • Mission-critical reliability • Transaction guarantees • Deep security • Real-time performance • Backup and recovery • Interactive SQL • Rich analytics • Workload management • Data governance • Backup and recovery Enterprise Data Architecture 2TREND ENTERPRISE USERS OPERATIONAL SYSTEMS ANALYTICAL SYSTEMS PRODUCTION REQUIREMENTS PRODUCTION REQUIREMENTS OUTSIDE SOURCES
- 5. © 2015 MapR Technologies 5 OPERATIONAL SYSTEMS ANALYTICAL SYSTEMS ENTERPRISE USERS 1REALITY • Data staging • Archive • Data transformation • Data exploration • Streaming, interactions Hadoop Relieves the Pressure from Enterprise Systems 2 Interoperability 1 Reliability and DR 4 Supports operations and analytics 3 High performance Keys for Production Success
- 6. © 2015 MapR Technologies 6 Architecture Matters for Success2REALITY FOUNDATION
- 7. © 2015 MapR Technologies 7 FOUNDATION Architecture Matters for Success2REALITY Data protection & security High performance Multi-tenancy Real-time operational & analytical apps Open standards for integration NEW APPLICATIONS SLAs TRUSTEDINFORMATION LOWERTCO
- 8. © 2015 MapR Technologies 8 The Power of the Open Source Community APACHE HADOOP AND OSS ECOSYSTEM Security YARN Spark Streaming Storm StreamingNoSQL & Search Juju Provisioning & Coordination Sahara ML, Graph Mahout MLLib GraphX EXECUTION ENGINES DATA GOVERNANCE AND OPERATIONS Workflow & Data Governance Pig Cascading Spark Batch MapReduce v1 & v2 Tez HBase Solr Hive Impala Spark SQL Drill SQL Sentry Oozie ZooKeeperSqoop Flume Data Integration & Access HttpFS Hue Data PlatformMapR-FS MapR-DB Management
- 9. © 2015 MapR Technologies 9 The MapR Distribution including Apache Hadoop APACHE HADOOP AND OSS ECOSYSTEM Security YARN Spark Streaming Storm StreamingNoSQL & Search Juju Provisioning & Coordination Sahara ML, Graph Mahout MLLib GraphX EXECUTION ENGINES DATA GOVERNANCE AND OPERATIONS Workflow & Data Governance Pig Cascading Spark Batch MapReduce v1 & v2 Tez HBase Solr Hive Impala Spark SQL Drill SQL Sentry Oozie ZooKeeperSqoop Flume Data Integration & Access HttpFS Hue Data PlatformMapR-FS MapR-DB Management Data HubEnterprise Grade Operational
- 10. © 2015 MapR Technologies 10 MapR: Best Solution for Customer Success Premier Investors High Growth 2X Growth In Direct Customers 90% Subscription Licenses Software Margins 140% Dollar-based Net Expansion 700+ Customers 2X Growth In Annual Subscriptions ( ACV) Best Product Apache Open Source
- 11. © 2015 MapR Technologies 11 MapR and Syncsort Reference Architecture Sources RELATIONAL, SAAS, MAINFRAME DOCUMENTS, EMAILS LOG FILES, CLICKSTREAMS BLOGS, TWEETS, LINK DATA DATA MARTS DATA WAREHOUSE MapR Data Platform Business Intelligence / Visualization MapR-DB MapR-FS Batch (MR, Spark, Hive, Pig, …) Interactive (Impala, Drill, …) Streaming (Spark Streaming, Storm…) MAPR DISTRIBUTION FOR HADOOP
- 12. © 2015 MapR Technologies 12 Achieving Operational Efficiencies with Hadoop 61% The most popular workloads being shifted are large-scale data transformations Practitioners who have shifted one or more workloads from legacy data warehouses or mainframes to Hadoop!
- 13. © 2015 MapR Technologies 13 The Hadoop Adoption Challenge > hadoop fs -put
- 14. © 2015 MapR Technologies 14 A Complete Solution to Harness the Power of Hadoop
- 15. © 2015 MapR Technologies 15 Break Free from Hadoop Complexity Design Once, Deploy Anywhere! • Visually design data transformations once, and run anywhere • No changes or tuning required • Combine new and legacy sources for bigger insights • Intelligent Execution Layer dynamically optimizes the job for each platform: Hadoop, Windows, Unix, Linux or Cloud • Future-proof your applications! Intelligent ExecutionLayer Windows, Linux, Unix Hadoop Cloud
- 16. © 2015 MapR Technologies 16 One-step Access to All Your Data Build Your Enterprise Data Hub Hadoop + DMX-h Avro Parquet Cassandra MongoDB Mainframe Vertica Oracle Teradata Netezza JSON HBaseFiles Cloud • Collect virtually any data from mainframe to Big Data and NoSQL sources • Load data directly into Avro & Parquet. No staging required • Access & translate mainframe data using Sqoop and Spark • Let DMX-h dynamically split the data and load it to HDFS in parallel
- 17. © 2015 MapR Technologies 17 Make Data Available to Business Analysts Achieve the Fastest Path from Raw Data to Insight • Create Tableau & Qlikview files with one click • Achieve the fastest data loads without tuning hassles: • Fastest parallel loads to Greenplum, Netezza, Teradata & Vertica • High-performance connectivity to Big Data & NoSQL databases such as Cassandra, Hbase & MongoDB Hadoop + DMX-h NoSQL
- 18. © 2015 MapR Technologies 18 Accelerate EDW Offload Initiatives with SILQ Up to 20x shorter development time! • Web-based utility • Takes SQL as an input • Provides visual analysis of SQL ELT jobs • Generates metadata and data migration with DMX jobs • Supports ANSI-SQL 2011, BTEQ, Netezza, Oracle PL/SQL
- 19. © 2015 MapR Technologies 19 MapR + Syncsort Solutions Data Warehouse Optimization Click-stream Analysis Mainframe Offload Shift ELT Workloads to Hadoop Access, Translate & Analyze Mainframe Data with Hadoop Collect, Process & Analyze More Data from Your Website
- 20. © 2015 MapR Technologies 20 Experience More! 1. Listen to this webcast on demand: http://bit.ly/1y1z0Ex 2. Download the MapR Sandbox for Hadoop: www.mapr.com/sandbox 3. Sign up for a free DMX-h test drive: www.syncsort.com/mapr
Notas do Editor
- The first trend is that the industry leaders have shown how to use big data to compete and win in their markets. It’s no longer a nice to have – you need big data to compete Google pioneered MapReduce processing on commodity hardware and used that to catapult themselves to into the leading search engine even though they were 19th in the market Yahoo! Leveraged these ideas to create Hadoop to keep up with Google and many mainstream companies have followed with new data-driven applications such as “people you may know” (started by LinkedIn and now used by Facebook, Twitter, and every social application), product recommendation engines, contextual and personalized music services (beats), measuring digital media effectiveness (comScore), serving more relevant/targeted ads(Comcast, rubicon project), fraud and risk detection, healthcare efficacy, and more What makes the difference? A lot of attention is given to data science and developing sophisticated new algorithms, but in many cases just having more data beats better algorithms. (make point on collecting more consumer interaction as well as transaction data, as an example). In addition, competitive advantage is decided by very small percentages. Just 1% improvement in fraud can mean hundreds $millions in savings. A ½% lift in advertising effectiveness means millions in new product sales and profitability. The same can be applied to customer churn, disease diagnosis, and more.
- A second trend in enterprise architecture has been big data overwhelming the existing workload-specific systems which are in production. (list of requirements for each of these on the side in text) People started with mainframes or operational systems which run ERP, finance, CRM and other mission-critical applications. They require… (pick out attributes you want to stress on the left) You also have data warehouses, marts, data mining, and other analytical systems which pull data from these operational and other systems for providing insights to the business for decision making The amount/variety of data has been overloading these systems. You reach a certain point as you try to ingest new types of data when these systems are not cost-effective to scale to terabytes or petabytes of data
- The first reality is that as people put Hadoop into production, to relieve the pressure from other systems in their enterprise architecture it needs to reliable . Hadoop needs to be held to the same enterprise standards as your Oracle, SAP, Teradata, NetApp storage, or any other enterprise system. Many organizations are putting Hadoop into their data center to provide (list of use cases underneath) … it can do all of this and more, but For Hadoop to act as a system of record , it must provide the same guarantees for SLA’s, performance, data protection, and more Most importantly, Hadoop has the potential for both analytics AND operations. It can be used to optimize the data warehouse provide batch data refining or storage. But Hadoop can provide many operational analytics or database operations/jobs when done right.
- Choosing the right big data architecture is critical for success with your Hadoop projects and business applications One analogy is building a sky scraper. Before you can start building up, you have to lay a rock-solid foundation. This building is the new Wilshire Grand project in Los Angeles. In Feb of this year they set a Guinness World Record for pouring a 21,000 cubic yard (16,000 cubic meters) foundation over 26 hours (http://www.theguardian.com/cities/2014/feb/14/world-largest-concrete-pour-la-trucks-los-angeles) When completed in 2017, the building will be the tallest in the US outside of NY and Chicago.
- This analogy applies as well to building a data platform – you have to architect for the future. This allows you to build higher, stronger, and faster, without retrofitting later down the road (anyone who has added a second story to their house can attest to the additional cost and construction delays if you have to reinforce a foundation which wasn’t designed to hold the stress) For business-critical applications you must have data protection and security (availability, data protection, and recovery), high performance (with random read-write system), multi-tenancy (to support multiple business units, isolate applications or user data,…), provide good resource and workload management to support multiple applications, and open standards to integrate with the rest of the enterprise data architecture This data foundation allows you to support new data-driven applications (both operational and analytical) , maintain service level agreements with the business, provide information you can trust and count on being there when you need it, and ultimately being the best TCO for the long-run. Supporting enterprise systems without retrofits or multiple clusters to work around platform deficiencies (e.g., to support operational/online applications in Hadoop today, you need a separate HBase cluster – separate from the rest of your Hadoop cluster/investment)
- The power of MapR begins with the power of open source innovation and community participation. In some cases MapR leads the community in projects like Apache Mahout (machine learning) or Apache Drill (SQL on Hadoop) In other areas, MapR contributes, integrates Apache and other open source software (OSS) projects into the MapR distribution, delivering a more reliable and performant system with lower overall TCO and easier system management. MapR releases a new version with the latest OSS innovations on a monthly basis. We add 2-4 new Apache projects annually as new projects become production ready and based on customer demand.
- The power of MapR begins with the power of open source innovation and community participation. In some cases MapR leads the community in projects like Apache Mahout (machine learning) or Apache Drill (SQL on Hadoop) In other areas, MapR contributes, integrates Apache and other open source software (OSS) projects into the MapR distribution, delivering a more reliable and performant system with lower overall TCO and easier system management. MapR releases a new version with the latest OSS innovations on a monthly basis. We add 2-4 new Apache projects annually as new projects become production ready and based on customer demand.
- The MapR distribution for Hadoop is globally recognized as the technology leader Forrester published a Wave for Big Data Hadoop Solutions where it placed MapR as the highest ranking product based on current offering as well as roadmap. Cloud: MapR has been selected by two of the companies most experienced with MapReduce technology which is a testament to the technology advantages of MapR’s distribution. Amazon through its Elastic MapReduce service (EMR) hosted over 2 million clusters in the past year. Amazon selected MapR to complement EMR as the only commercial Hadoop distribution being offered, sold and supported as a service by Amazon to its customers. MapR was also selected by Google – the pioneer of MapReduce and the company whose white paper on MapReduce inspired the creation of Hadoop – has also selected MapR to make our distribution available on Google Compute Engine.