基調講演：「ビッグデータのセキュリティとガバナンス要件」 #cwt2015
•Transferir como PPTX, PDF•
2 gostaram•1,244 visualizações
Cloudera, Inc. プロダクトマーケティングディレクター、Clarke PattersonによるCloudera World Tokyo 2015の講演資料です
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Destaque
Destaque (20)
Semelhante a 基調講演：「ビッグデータのセキュリティとガバナンス要件」 #cwt2015
Semelhante a 基調講演：「ビッグデータのセキュリティとガバナンス要件」 #cwt2015 (20)
Mais de Cloudera Japan
Mais de Cloudera Japan (10)
Último
Último (11)
基調講演：「ビッグデータのセキュリティとガバナンス要件」 #cwt2015
- 1. 1© Cloudera, Inc. All rights reserved. ビッグデータの セキュリティとガバナンス要件 Clarke Patterson | 製品、ソリューションおよび パートナーマーケティング担当 シニアディレクター
- 2. 2© Cloudera, Inc. All rights reserved. これらの企業が共通に持っているものは何でしょう？
- 3. 3© Cloudera, Inc. All rights reserved. 組織を守るには、新しい情報戦略が不可欠です
- 4. 4© Cloudera, Inc. All rights reserved. Cloudera の情報セキュリティのための エンタープライズデータハブ 1つのプラットフォームで セキュリティをより多くの情報に あらゆる環境を調査 未知の脅威を検知
- 5. 5© Cloudera, Inc. All rights reserved. セキュリティをより多くの情報に Clouderaが違うのは … コモディティハードウェアや クラウドに導入、１TBあたりの コストを低減できます *security optimized for Intel chips 手頃なストレージ Apache Hadoop は、データに アプリケーションを提供します 最小限のデータ移動 暗号化とキー管理、バックアップ ディザスタリカバリ、ユーザー認証 … エンタープライズ向け セキュリティ
- 6. 6© Cloudera, Inc. All rights reserved. あらゆる環境を調査 どんな規模の 非構造化および 構造化情報でも調査 あらゆるデータ 保存されているどんな情報も、 単純な検索やSQL、高度な テクニックを使って分析 さまざまなアクセス フレームワーク ストリーミングデータを投入し処理、 即座にアップデート リアルタイム モニタリング Discover 分析 データベース Impala 検索 Solr Model 機械 学習 SAS, R, Spark, Mahout Clouderaが違うのは …
- 7. 7© Cloudera, Inc. All rights reserved. 未知の脅威を検知 Clouderaが違うのは … 処理対象となるデータセットは、 直ちに分析ができるよう、 1つのシステムに配置 シングルデータソース インメモリおよび 並列処理により、曖昧な 問い合わせにも迅速に回答 処理速度 数年にわたるデータ全体から、 異常なビヘイビア（行動）を検知 大規模に高度な分析 生データ データの改善 信頼性の高いデータ 0 50 100 150 200 250 300 350 時間(秒)
- 8. 8© Cloudera, Inc. All rights reserved. 既存のテクノロジーとスキルに対する投資を補完 Data Systems Applications System Integration 豊富な情報セキュリティ エコシステムを含む、 1,900社以上の Cloudera 認定パートナーOperational Tools Infrastructure Enterprise Data Hub Security and Administration Unlimited Storage Process Discover Model Serve 情報セキュリティ
- 9. 9© Cloudera, Inc. All rights reserved. Cloudera は次世代の情報セキュリティを 推進しています
- 10. 10© Cloudera, Inc. All rights reserved. Dell SecureWorks は、 1,000 のセキュリティ インシデントを調査し、 20,000 のマルウェアを 解析しました
- 11. 11© Cloudera, Inc. All rights reserved. FINRA は日々300億の マーケットイベントを 監視し、米国市場の 総合的な動きを把握する と同時に、年間1,000万 ~2,000万ドルの節減を 果たしています
- 12. 12© Cloudera, Inc. All rights reserved. 世界で最初に Hadoop で PCI 認定を受けました
- 13. 13© Cloudera, Inc. All rights reserved. CounterTack は Cloudera のビッグデータ分析機能を 活用し、組織をサイバー 攻撃から守っています
- 14. 14© Cloudera, Inc. All rights reserved. 組織を守るには、新しい情報戦略が不可欠です
- 15. 15© Cloudera, Inc. All rights reserved. 容易な道のりでは ありません
- 16. 16© Cloudera, Inc. All rights reserved. Cloudera の Hadoop は、包括的でコンプライアンスレディな セキュリティ機能を提供 アクセス データにアクセスできる ユーザーや アプリケーションを定義 テクニカルコンセプト： 認可 ネットワークの分離 データ クラスター内のデータを 不正アクセスから保護 テクニカルコンセプト： 暗号化、トークン化 データのマスキング 可視化 データの出どころと どうのように 利用されているか報告 テクニカルコンセプト： 監査 リネージ Cloudera Manager Apache Sentry Cloudera Navigator Navigator Encrypt & Key Trustee | Partners 境界セキュリティ クラスタ本体に対する アクセスを防御 テクニカルコンセプト： 認証 ネットワークの分離
- 17. 17© Cloudera, Inc. All rights reserved. さあ始めよう 運用効率 新しいビジネス価値 OPERATIONS DATAMANAGEMENT UNIFIED SERVICES PROCESS,ANALYZE, SERVE STORE INTEGRATE SQL を超える アジャイルな データソース 調査 ETL/Batch の高速化 安価な ストレージ EDW の最適化 パーベイシブ 分析
- 18. 18© Cloudera, Inc. All rights reserved. Cloudera の情報セキュリティのための エンタープライズデータハブ 1つのプラットフォームで セキュリティをより多くの情報に あらゆる環境を調査 未知の脅威を検知
- 19. 19© Cloudera, Inc. All rights reserved. ありがとうございました Clarke Patterson | @clarkepatterson
Notas do Editor
- IBM Research: Single User (solid bar) vs 10 User Response Time (striped bar) (Lower bars = better)
- Cloudera partners more broadly and deeply across the Hadoop ecosystem than any other vendor. With over 1200 partners and counting, our partnerships offer: Compatibility with your existing tools and skills 160+ certified on Cloudera 5, including all 12 of the 12 Gartner Business Intelligence Magic Quadrant leaders Flexible deployment options On-premises Public, private, or hybrid cloud Appliances and engineered systems Partnerships you can trust Deep engineering relationships Comprehensive certification program
- Link to account record in SFDC (valid for Cloudera employees only): https://na6.salesforce.com/0018000000rBJSQ?srPos=0&srKp=001 Dell SecureWorks designs better defenses through complex analytics on historical security breaches. Background: Dell SecureWorks is on deck 24 hours a day, 365 days a year, to help protect the security of its customers’ assets in real time. To meet its enormous data processing challenges, Dell SecureWorks requires leading-edge technologies for cost-effectively storing, scaling, and analyzing massive amounts of information in real time. SecureWorks processes ~20 billion events per day in real time. An event is a single transaction for a security device, such as a firewall or an intrusion prevention system (IPS), or a log event from an application or operating system. Challenge: Each day, the SecureWorks organization investigates ~1,000 potential security incidents and analyzes ~20,000 pieces of malicious software. Leveraging this global intelligence enables SecureWorks to offer improved threat protection on behalf of its customers. To deal with the massive amounts of information it collects, SecureWorks requires highly scalable solutions for processing, storing, and analyzing big data in a cost-conscious manner. Historically, the SecureWorks organization met this need with proprietary technologies. But a few years ago, it started looking for next-generation solutions that would deliver greater scalability at lower cost, including those based on open-source software and industry standard hardware. Solution: This search culminated in the deployment of the Dell | Cloudera Solution, comprised of Apache® Hadoop® open source software, Dell PowerEdge C servers, Force10™ switches, and services from Cloudera and Dell. Hadoop offered a highly scalable system with built-in availability, and the capability to run very-high-analytics types of activities. Results: By migrating to the Dell | Cloudera solution, SecureWorks reduced storage costs from $17 per GB to only 21 cents per GB. The Hadoop environment paid for itself in one year. And the cost savings delivered by this solution will continue delivering substantial savings as SecureWorks doubles its business and associated data volumes every 18 months.
- Link to account record in SFDC (valid for Cloudera employees only): https://na6.salesforce.com/0018000000pvwoy?srPos=0&srKp=001 A financial regulatory body is building a holistic picture of US market activity by looking at 30 billion events per day. Background: The Financial Industry Regulatory Authority, or FINRA, is an independent regulator authorized by U.S. Congress to promote investor protection and market integrity. FINRA oversees every brokerage firm and broker doing business with the U.S. public and monitors trading on the U.S. stock markets. Challenge: Protecting market integrity while keeping pace with the billions of events that take place every day presents unique challenges: Market volumes are volatile and steadily increasing. Exchanges are dynamically evolving. Regulatory rules are continuously being created and enhanced. New securities products are regularly introduced. Market manipulators are constantly innovating. Solution: Cloudera was selected as the Hadoop distribution for the critical process of building the market event graph database described above, and for providing rapid access to that data for regulatory analysts, based on a few key factors: - Cloudera’s software platform delivers a pervasive, enterprise-ready Apache Hadoop distribution. - Cloudera Manager provides a sophisticated and user-friendly management console, simplifying and streamlining Hadoop cluster administration. - Cloudera’s deep expertise, available through Professional Services and Support, would be critical to solving technical challenges inherent in a Hadoop-based architecture of this scale. Using Hadoop to build the market event graph database was the first major challenge to the redeployment program. Every day, FINRA receives data feeds containing orders, quotes, and trades from securities firms and various exchanges. Since transactions may span several data providers over days or weeks, understanding order lifecycles requires that huge volumes of data be linked across these different feeds and updated as new information is received. This requires very rapid data access, which was provided by implementing Apache HBase as part of FINRA’s CDH deployment. Results: The new system must be able to handle increases in market volumes. The Cloudera Hadoop platform will assist FINRA in keeping pace with this growth in the following ways: - By enabling horizontal rather than vertical scaling, FINRA is no longer obliged to buy ever larger, more expensive data appliances to support increasing volumes. - Similarly, the Hadoop architecture doesn’t impose “largest available appliance” limitations. The use of industry standard hardware allows for more flexibility in selecting suppliers and architectures to cope with future growth. FINRA's Big Data platform supports its mission of investor protection and market integrity by helping the organization increase performance and accelerate innovations in the following ways: - The use of HBase to access the order lifecycle graph database has reduced response times for certain complex queries by orders of magnitude. Queries that took hours now take seconds. One complex query in particular that took ninety minutes to run was reduced to ten seconds. This creates a far more interactive experience, allowing analysts to rapidly iterate and quickly converge on answers that would have been prohibitive in the prior system. - With its open source core for data processing, FINRA is able to leverage the rapid innovation cycles in the Hadoop and Big Data community. FINRA's program to implement market regulation applications using Big Data and cloud technologies including Cloudera is expected to provide a net annual benefit of USD $10 to $20 million. These savings are derived from reducing costs of the operational infrastructure in a few key ways. - The specialized nature of its legacy data appliances resulted in significant hardware costs and ongoing support requirements. FINRA’s Hadoop-based platform allows the use of less expensive, industry standard hardware. - Unlike proprietary data appliances, a Hadoop-based architecture lends itself to deployment on a public cloud such as Amazon Web Services. This allows FINRA to benefit from the operational economies of scale of such a vender, as well as providing the elasticity to avoid overprovisioning to handle peak loads. - The legacy environment was also challenged with capacity limitations, causing FINRA to spend much effort and analysis determining where and when available capacity could be used for various types of analytics. By removing the capacity limitations, the Hadoop/cloud combination virtually eliminates this effort. - Cloudera Manager automatically manages extremely complex operational tasks within Hadoop, allowing FINRA to focus resources and expertise on areas of strategic regulatory importance rather than fundamental IT tasks. This saves costs while increasing FINRA's ability to accomplish core objectives.
- KEY MESSAGE ON THIS SLIDE: Our relationship first began with MasterCard. They chose Cloudera Enterprise for fraud detection and to optimize their DW infrastructure. And then expanded to form a partnership with MC Advisors, the consulting arm of MasterCard. MasterCard requires that any technology handling its applications or payment card data files must have full PCI certification. Receiving this important certification allows MasterCard the opportunity to integrate Hadoop datasets with other environments that are already PCI-certified. "Data privacy and protection is a top priority for MasterCard," said Gary VonderHaar, Chief Technology Officer, Architecture at MasterCard. "As we maximize the most advanced technologies from partners and vendors, they must meet the rigorous security standards we’ve set. With Cloudera’s commitment to the same standards, we now have additional options in how we manage our data center.” ADDITIONAL TALKING POINTS (USE AS REQUIRED) As customer and transaction data continues to grow at Internet scale, a variety of new security, risk, transparency, and technology standards are forcing banks, payment processors, and retailers to contend with major compliance challenges. Industry regulations have introduced new complexity and costs related balancing big data analytics with customer privacy. Striking the right balance, security becomes the foundational capability on which valuable customer insights and competitive advantage are built. In the credit card industry, the primary goal of information management is ensuring that cardholder data is properly secured and protected and that merchants and third-party solution providers meet minimum privacy levels for any application, database, or file system that plays a role in storing, processing, or transmitting account-related data. The Payment Card Industry Data Security Standard (PCI DSS) was formalized as an industrywide standard in 2004, originating as separate data security standards established by the five major credit card companies: MasterCard, VISA, Discover, American Express, and the Japan Credit Bureau (some of whom are Cloudera Enterprise customers). The most straightforward way to comply with PCI DSS is to encrypt all data-at-rest and data-in-motion and manage the encryption keys away from the protected data. However, the security standards have historically required specialized software, which, at times, has failed to prevent infiltration by nefarious actors—technology alone is insufficient to stop attacks. In the event of a security breach resulting from non-compliance, the breached organization could be subject to stiff penalties, huge legal fees, chargeback of all fraudulent transactions, and increased government scrutiny. In recent months, credit card data security breaches have not only led to fines, but also to consumer mistrust, massive brand damage, and sharp declines in revenues for the merchants managing the point of sale. MasterCard and Cloudera: Safeguarding Expertise Because of the stringency of complete data security, it has historically taken up to 24 months to build a fully PCI-compliant big data infrastructure. Even then, a faulty privacy configuration could lead to a devastating data breach on a presumed compliant system. In response, Cloudera has partnered with MasterCard Advisors to simplify and certify the deployment and compliance process for an EDH. MasterCard was the first company to achieve PCI certification on a Hadoop system and brings decades of technology, data, and process expertise to help organizations of all sizes cut the time to production and the total cost of passing an annual PCI audit by more than half. MasterCard Advisors and Cloudera can provide education and technical guidance for your team and can be longterm partners as standards change and data needs evolve. By developing a certification plan for your big data infrastructure that leverages Cloudera’s leading Hadoop platform and MasterCard Advisors’ proven PCI architecture, you can remove the cost and uncertainty of achieving and maintaining compliance and focus on realizing the business value of all your data in an EDH.
- SLIDE 6,7,8: case study individual slides Key Point 3: Customers choose Cloudera for IoT success · Sub point 1: Opower · Sub point 2: Monstanto (?) · Sub point 3: Vivint (need logo & entity surveillance & monitoring — nest-esque, but not nest) Vivint story: http://www.cloudera.com/content/cloudera/en/about/press-center/press-releases/2014/07/15/vivint-standardizes-on-a-cloudera-enterprise-data-hub-for-securi.html
- With Cloudera, we offer a comprehensive set of security controls that balances the flexibility of Hadoop against the concerns of stakeholders - for the most secure Hadoop distribution in the market. We comprehensively address all the traditional security concerns around authentication, authorization, audit, and compliance – for a full compliance-ready stack. In fact, we were the first and only distribution to achieve PCI compliance. Credentialize Differentiate Cloudera contributed to Hadoop community, partnership with Intel, Gazzang acquisition
- Data Engineering Use Cases: Active Archive ETL Offload EDW Optimization Data Discovery & Analytics Use Cases: Self-service BI Advanced Analytics Data Science Data Applications Use Cases: Recommendation Engine Event Detection Real-time Analytics