O slideshow foi denunciado.
Utilizamos seu perfil e dados de atividades no LinkedIn para personalizar e exibir anúncios mais relevantes. Altere suas preferências de anúncios quando desejar.
Cloud Computing @Yahoo! Dekel Tankel Director, Product Management Yahoo!  Cloud Computing [email_address] IGT, June 2009
What we’ll cover today… <ul><li>Why Cloud?  </li></ul><ul><ul><li>Scale and Abstraction; Quality and Agility </li></ul></u...
Why Cloud? Benefits for Yahoo! <ul><li>Higher Agility & Stability while maintaining Scale </li></ul><ul><li>Abstraction  <...
Yahoo!’s Unique Cloud:  Unprecedented Scale <ul><li>Massive user base and engagement </li></ul><ul><ul><li>500M+ unique us...
Yahoo! Cloud Services ROI & Innovation Y!OS, BOSS, YQL, APT, Analytics, … Storage, Batch,  Edge Serving,… Users Applicatio...
Yahoo! Cloud Services:  Focus on PaaS offerings ROI & Innovation IaaS PaaS SaaS Users Applications Functional Cloud Servic...
From Infrastructure to  Shareholders benefit <ul><li>Horizontal Cloud </li></ul><ul><ul><li>Focus on open source and colla...
Yahoo! Cloud Strategy in Action: The Front Page Case Study <ul><li>Horizontal Cloud – Storage & Hadoop </li></ul><ul><ul><...
Yahoo! Cloud Strategy in Action: The Inquisitor Case Study <ul><li>Horizontal Cloud – Hadoop </li></ul><ul><ul><li>Analyze...
Horizontal Cloud Services ROI & Innovation Users Applications Functional Cloud Services Horizontal Cloud  Services  Physic...
Horizontal Cloud Services <ul><li>Optimized for Yahoo!-scale </li></ul><ul><ul><li>Yahoo!-internal focus </li></ul></ul><u...
Horizontal Cloud Services:  Conceptual View Common Approaches to QA, Production Engineering, Performance Engineering, Data...
Horizontal Cloud Services: Use Cases Ads Optimization Content Optimization Search Index Image/Video Storage & Delivery Mac...
Yahoo! Distribution of Hadoop <ul><li>Hadoop in a nutshell </li></ul><ul><ul><li>Open source distributed file system & par...
Yahoo! runs the largest  Hadoop Clusters in the World <ul><li>25,000+ nodes  </li></ul><ul><ul><li>Clusters of up to 4,000...
Case Study  -  Search Assist™ <ul><li>Database for  Search Assist™  is built using Hadoop.  </li></ul><ul><li>3 years of l...
Functional Cloud Services ROI & Innovation Users Applications Functional Cloud Services Horizontal Cloud  Services  Physic...
Functional Cloud Services <ul><li>Provides functional capabilities for applications </li></ul><ul><ul><li>Help developers ...
Functional Cloud Services:  YQL & BOSS <ul><li>  </li></ul>A single endpoint service that enables developers to query, fil...
Build your Own Search Service (BOSS) <ul><li>Yahoo!'s open search web services platform </li></ul><ul><ul><li>Serving hund...
Yahoo! Query Language (YQL) <ul><li>Single endpoint service to query, filter and combine data across Yahoo! and beyond </l...
Y!OS and Cloud
Yahoo! Open Stagey (Y!OS): Goals
Y!OS and Cloud Strategy CLOUD SERVICES
Open Collaborations  around the globe <ul><li>M45  - Yahoo!’s supercomputing cluster </li></ul><ul><ul><li>4,000 cores, 3 ...
Questions? Dekel Tankel Director, Product Management Yahoo! Cloud Computing [email_address]
Próximos SlideShares
Carregando em…5
×

Cloud Computing @Yahoo!

1.644 visualizações

Publicada em

Publicada em: Negócios, Tecnologia
  • Seja o primeiro a comentar

Cloud Computing @Yahoo!

  1. 1. Cloud Computing @Yahoo! Dekel Tankel Director, Product Management Yahoo! Cloud Computing [email_address] IGT, June 2009
  2. 2. What we’ll cover today… <ul><li>Why Cloud? </li></ul><ul><ul><li>Scale and Abstraction; Quality and Agility </li></ul></ul><ul><ul><li>Yahoo!’s unique footprint </li></ul></ul><ul><li>Yahoo!’s Cloud Strategy </li></ul><ul><ul><li>Overview of the Yahoo! Cloud vision and portfolio </li></ul></ul><ul><ul><li>Deep dive on Horizontal & Functional Cloud Services </li></ul></ul><ul><li>The Yahoo! Open Strategy </li></ul><ul><ul><li>Marrying Yahoo!’s “Open Strategy”, its platforms and ethic with external Cloud services </li></ul></ul>
  3. 3. Why Cloud? Benefits for Yahoo! <ul><li>Higher Agility & Stability while maintaining Scale </li></ul><ul><li>Abstraction </li></ul><ul><ul><li>Enable developers to focus on their applications, not infrastructure </li></ul></ul><ul><li>Accelerating innovation </li></ul><ul><ul><li>Adding new features and products at an ever faster rate </li></ul></ul><ul><li>Increasing Scale & Availability </li></ul><ul><ul><li>More robustly, more globally, more completely, for a given budget </li></ul></ul>Cloud is pushing up the Operation Excellence Curve Agility & Innovation Quality & Stability
  4. 4. Yahoo!’s Unique Cloud: Unprecedented Scale <ul><li>Massive user base and engagement </li></ul><ul><ul><li>500M+ unique users per month </li></ul></ul><ul><ul><li>Hundreds of petabyte of storage </li></ul></ul><ul><ul><li>Hundreds of billions of objects </li></ul></ul><ul><ul><li>Hundred of thousands of requests/sec </li></ul></ul><ul><li>Global </li></ul><ul><ul><li>Tens of globally distributed data centers </li></ul></ul><ul><ul><li>Serving each region at low latencies </li></ul></ul><ul><li>Challenging Users </li></ul><ul><ul><li>Rapidly extracting value from voluminous data </li></ul></ul><ul><ul><li>Downtime is not an option (outages cost $millions) </li></ul></ul><ul><ul><li>Variable usage patterns </li></ul></ul>
  5. 5. Yahoo! Cloud Services ROI & Innovation Y!OS, BOSS, YQL, APT, Analytics, … Storage, Batch, Edge Serving,… Users Applications Functional Cloud Services Horizontal Cloud Services Physical Layer
  6. 6. Yahoo! Cloud Services: Focus on PaaS offerings ROI & Innovation IaaS PaaS SaaS Users Applications Functional Cloud Services Horizontal Cloud Services Physical Layer
  7. 7. From Infrastructure to Shareholders benefit <ul><li>Horizontal Cloud </li></ul><ul><ul><li>Focus on open source and collaborative R&D with industry, academia and government </li></ul></ul><ul><li>Functional Cloud </li></ul><ul><ul><li>Focus on developing &quot;open strategy&quot; frameworks, tools and services for developers (at Yahoo! and beyond) </li></ul></ul><ul><li>Combined Together </li></ul><ul><ul><li>Leverage our unique scale, assets and data to drive disruptive innovations in the market and expand Yahoo!’s competitive differentiation </li></ul></ul>
  8. 8. Yahoo! Cloud Strategy in Action: The Front Page Case Study <ul><li>Horizontal Cloud – Storage & Hadoop </li></ul><ul><ul><li>Analyze extremely large content data sets </li></ul></ul><ul><li>Functional Cloud – Content Optimization </li></ul><ul><ul><li>Rate content items based on various parameters </li></ul></ul><ul><li>Applications – Yahoo’s Front Page </li></ul><ul><ul><li>Display “high rating” items to the right users </li></ul></ul><ul><ul><li>Benefit consumers and advertisers and grow Yahoo!’s revenue </li></ul></ul>
  9. 9. Yahoo! Cloud Strategy in Action: The Inquisitor Case Study <ul><li>Horizontal Cloud – Hadoop </li></ul><ul><ul><li>Analyze large search-index data sets </li></ul></ul><ul><li>Functional Cloud - BOSS </li></ul><ul><ul><li>Expose the data in a structured, open, flexible and “cloud like” way </li></ul></ul><ul><li>Applications - iPhone TM Inquisitor </li></ul><ul><ul><li>Leverage BOSS to provide innovative consumer experience </li></ul></ul><ul><ul><li>Benefit consumers and grow Yahoo!’s revenue </li></ul></ul>
  10. 10. Horizontal Cloud Services ROI & Innovation Users Applications Functional Cloud Services Horizontal Cloud Services Physical Layer
  11. 11. Horizontal Cloud Services <ul><li>Optimized for Yahoo!-scale </li></ul><ul><ul><li>Yahoo!-internal focus </li></ul></ul><ul><ul><li>Data processing and serving environments </li></ul></ul><ul><li>Drive faster innovation and agility </li></ul><ul><ul><li>Shorter product development cycles </li></ul></ul><ul><ul><li>Reduce labor and costs for infrastructure </li></ul></ul><ul><li>Multi-year effort </li></ul><ul><ul><li>Strategic investment across the company </li></ul></ul>
  12. 12. Horizontal Cloud Services: Conceptual View Common Approaches to QA, Production Engineering, Performance Engineering, Datacenter Management, and Optimization Shared Infrastructure Simple API’s ID & Account Management Provisioning & Virtualization (Xen) Operational Storage Structured, unstructured Batch Storage & Processing Hadoop, PIG Edge Content Services Caching, Proxies Online Serving Web, Data Security and Authentication Metering, Billing Monitoring & QoS
  13. 13. Horizontal Cloud Services: Use Cases Ads Optimization Content Optimization Search Index Image/Video Storage & Delivery Machine Learning (e.g. Spam filters) Attachment Storage
  14. 14. Yahoo! Distribution of Hadoop <ul><li>Hadoop in a nutshell </li></ul><ul><ul><li>Open source distributed file system & parallel execution environment to process massive amounts of data </li></ul></ul><ul><ul><li>Started in 2005, became top-level Apache project in 2008 </li></ul></ul><ul><ul><li>Simple Design for Horizontal Scaling on commodity HW </li></ul></ul><ul><li>Yahoo! Distribution of Hadoop </li></ul><ul><ul><li>Source distribution of Yahoo!’s implementation of Hadoop (Based entirely on code found in the Apache Hadoop) </li></ul></ul><ul><ul><li>Tested and deployed at Yahoo!’s massive scale </li></ul></ul><ul><ul><li>Benefit the larger ecosystem , Increase pace of innovation </li></ul></ul><ul><ul><li>http://developer.yahoo.com/hadoop </li></ul></ul>
  15. 15. Yahoo! runs the largest Hadoop Clusters in the World <ul><li>25,000+ nodes </li></ul><ul><ul><li>Clusters of up to 4,000 nodes </li></ul></ul><ul><li>4 Tiers of clusters </li></ul><ul><ul><li>Development & Testing, POCs, Science & Research, Production </li></ul></ul><ul><li>Terasort Benchmarks </li></ul><ul><ul><li>62 seconds to sort One Terabyte (run on 1,500 nodes) </li></ul></ul><ul><ul><li>16.25 hours to sort One Petabyte (run on 3,700 nodes) </li></ul></ul><ul><li>Webmap application </li></ul><ul><ul><li>~490 TB shuffling </li></ul></ul><ul><ul><li>~280 TB output </li></ul></ul>
  16. 16. Case Study - Search Assist™ <ul><li>Database for Search Assist™ is built using Hadoop. </li></ul><ul><li>3 years of log-data, 20-steps of map-reduce </li></ul><ul><ul><li>Leverage Hadoop’s scalability, load balancing and resiliency </li></ul></ul><ul><ul><li>Simplified access, flexibility for rapid innovation (from C++ to Python) </li></ul></ul>Before Hadoop After Hadoop Time 26 days 20 minutes Development Time 2-3 weeks 2-3 days
  17. 17. Functional Cloud Services ROI & Innovation Users Applications Functional Cloud Services Horizontal Cloud Services Physical Layer
  18. 18. Functional Cloud Services <ul><li>Provides functional capabilities for applications </li></ul><ul><ul><li>Help developers to accomplish integrated web experiences in a faster and easier way </li></ul></ul><ul><ul><li>Provides common set of functional “building blocks” </li></ul></ul><ul><li>“ Powered by” the horizontal cloud services </li></ul><ul><ul><li>Abstracts infrastructure services from the Application </li></ul></ul><ul><ul><ul><li>E.g. Storage, Compute, Serving, Robustness and Scalability </li></ul></ul></ul><ul><ul><li>Self-Served, Global, Managed, Elastic and Metered </li></ul></ul>
  19. 19. Functional Cloud Services: YQL & BOSS <ul><li> </li></ul>A single endpoint service that enables developers to query, filter and combine data across Yahoo! and beyond http://developer.yahoo.com/yql/console/ Providing Yahoo! Search infrastructure and technology to developers and companies to help them build their own search experiences Build your Own Search Service http://developer.yahoo.com/search/boss / Yahoo! Query Language
  20. 20. Build your Own Search Service (BOSS) <ul><li>Yahoo!'s open search web services platform </li></ul><ul><ul><li>Serving hundreds of millions of users across the Web. </li></ul></ul><ul><li>Goal: foster innovation in the search industry </li></ul><ul><ul><li>Build and launch web-scale search products that utilize the entire Yahoo! Search index. </li></ul></ul><ul><ul><li>Access to Yahoo!'s investments in crawling and indexing, ranking and relevancy algorithms </li></ul></ul>
  21. 21. Yahoo! Query Language (YQL) <ul><li>Single endpoint service to query, filter and combine data across Yahoo! and beyond </li></ul><ul><ul><li>The “Internet API” </li></ul></ul><ul><li>SQL-like SELECT syntax for getting the right data </li></ul><ul><ul><li>Quickly discover available data sources and structure </li></ul></ul><ul><ul><li>Combined data from a single web browser </li></ul></ul><ul><li>Easy-to-use Consol </li></ul><ul><ul><li>http://developer.yahoo.com/yql/console/ </li></ul></ul>
  22. 22. Y!OS and Cloud
  23. 23. Yahoo! Open Stagey (Y!OS): Goals
  24. 24. Y!OS and Cloud Strategy CLOUD SERVICES
  25. 25. Open Collaborations around the globe <ul><li>M45 - Yahoo!’s supercomputing cluster </li></ul><ul><ul><li>4,000 cores, 3 TB RAM, 1.5 PB disks, 27 teraflops! </li></ul></ul><ul><ul><li>Operational since November 2007, 4 major Universities </li></ul></ul><ul><ul><li>Focus on highly parallel computing </li></ul></ul><ul><li>Open Cirrus™ with HP & Intel </li></ul><ul><ul><li>A global, multi-data center, open source test bed </li></ul></ul><ul><ul><li>Target to advance cloud computing research & education </li></ul></ul><ul><ul><li>Simulates a real-life, Internet-scale environment </li></ul></ul><ul><ul><li>9 Global sites, more than 50 research projects </li></ul></ul>
  26. 26. Questions? Dekel Tankel Director, Product Management Yahoo! Cloud Computing [email_address]

×