The Missing Piece of On-Demand Clusters

•

0 gostou•1,617 visualizações

The Missing Piece of On-Demand Clusters Presented by Calvin Jia, Alluxio Introduction to Alluxio Meetup at Princeton http://www.meetup.com/futureofdata-princeton/events/232927731/

Software

The Missing Piece of On-Demand Clusters
October 2016
Calvin Jia

Outline
• Cloud Architectures
• Alluxio in Cloud Architectures
• Takeaways
3

Cloud Architectures – Overview
• Mostly service based, from providers
– Amazon Web Services
– Google Cloud Platform
• Separate compute and storage clusters
• Compute clusters are ephemeral
4

Cloud Architectures – Pros & Cons
Pros
• Low maintenance
• Pay as you go
• Elastic and scalable
• Cost effective storage
Cons
• Lower Performance
5

Outline
• Cloud Architectures
• Alluxio in Cloud Architectures
• Takeaways
6

Alluxio in Cloud Architectures – Overview
• Deployed in compute clusters
– Memory speed data access
– Transparent data access to any storage
• Simple to deploy
– Mount storage systems similar to local disks
7

Alluxio in Cloud Architectures – Example
8

Alluxio in Cloud Architectures – Benefits
9
• Remedies the performance drawback
• Acceleration due to memory-speed I/O
• Designed to improve the affinity of compute
and storage

Alluxio in Cloud Architectures – Data Path
10
FAST
104 - 105 MB/s
MODERATE 103 - 104 MB/s
SLOW 102 - 103 MB/s
Only when necessary
Limited
Often
SSD
HDD
Mem

Takeaways – Experiment Results
11
0
100
200
300
400
500
600
Initial
Read Subsequent
Read Read
from

Separate
Job
Runtime
(seconds)
Spark
-‐ No
Persist
Spark
-‐ Persist
Alluxio

Takeaways – Alluxio & Cloud Architectures
12
• Cloud architectures have significant upsides
• Alluxio alleviates the major downsides

Try out Alluxio 1.2.0
http://www.alluxio.org/releases
13

Read the blog athttp://www.alluxio.com/2016/08/accelerating-on-demand-data-analytics-with-alluxio
Read the whitepaper at www.alluxio.com.
Check out www.alluxio.org.
Thank you!
14

Mais conteúdo relacionado

Mais procurados

Effective Spark with Alluxio at Strata+Hadoop World San Jose 2017

Alluxio, Inc.

Alluxio (formerly Tachyon): Open Source Memory Speed Virtual Distributed Storage

Alluxio, Inc.

Alluxio (formerly Tachyon): The Journey thus far and the Road Ahead

Alluxio, Inc.

Alluxio-FUSE as a data access layer for Dask

Alluxio, Inc.

ALLUXIO (formerly Tachyon): Unify Data at Memory Speed - Effective using Spar...

Alluxio, Inc.

Rise of Intermediate APIs - Beam and Alluxio at Alluxio Meetup 2016

Alluxio, Inc.

Introduction to Alluxio (formerly Tachyon) and how it brings up to 300x perfo...

Alluxio, Inc.

Alluxio Use Cases at Strata+Hadoop World Beijing 2016

Alluxio, Inc.

Alluxio: Unify Data at Memory Speed; 2016-11-18

Alluxio, Inc.

Open Source Memory Speed Virtual Distributed Storage

Alluxio, Inc.

Best Practices for Using Alluxio with Spark

Alluxio, Inc.

Alluxio Mesos Meetup - SMACK to SMAACK

Alluxio, Inc.

Best Practices for Using Alluxio with Spark

Alluxio, Inc.

Alluxio Keynote at Strata+Hadoop World Beijing 2016

Alluxio, Inc.

Securely Enhancing Data Access in Hybrid Cloud with Alluxio

Alluxio, Inc.

Deploying Alluxio in the Cloud for Machine Learning

Alluxio, Inc.

Alluxio Presentation at Strata San Jose 2016

Jiří Šimša

Building a high-performance data lake analytics engine at Alibaba Cloud with ...

Alluxio, Inc.

Getting Started with Alluxio + Spark + S3

Alluxio, Inc.

Presto on Alluxio Hands-On Lab

Alluxio, Inc.

Mais procurados (20)

Effective Spark with Alluxio at Strata+Hadoop World San Jose 2017

Alluxio (formerly Tachyon): Open Source Memory Speed Virtual Distributed Storage

Alluxio (formerly Tachyon): The Journey thus far and the Road Ahead

Alluxio-FUSE as a data access layer for Dask

ALLUXIO (formerly Tachyon): Unify Data at Memory Speed - Effective using Spar...

Rise of Intermediate APIs - Beam and Alluxio at Alluxio Meetup 2016

Introduction to Alluxio (formerly Tachyon) and how it brings up to 300x perfo...

Alluxio Use Cases at Strata+Hadoop World Beijing 2016

Alluxio: Unify Data at Memory Speed; 2016-11-18

Open Source Memory Speed Virtual Distributed Storage

Best Practices for Using Alluxio with Spark

Alluxio Mesos Meetup - SMACK to SMAACK

Best Practices for Using Alluxio with Spark

Alluxio Keynote at Strata+Hadoop World Beijing 2016

Securely Enhancing Data Access in Hybrid Cloud with Alluxio

Deploying Alluxio in the Cloud for Machine Learning

Alluxio Presentation at Strata San Jose 2016

Building a high-performance data lake analytics engine at Alibaba Cloud with ...

Getting Started with Alluxio + Spark + S3

Presto on Alluxio Hands-On Lab

Destaque

Tachyon workshop 2015-07-19

Tachyon Nexus, Inc.

Accelerating Machine Learning Pipelines with Alluxio at Alluxio Meetup 2016

Alluxio, Inc.

Tachyon: An Open Source Memory-Centric Distributed Storage System

Tachyon Nexus, Inc.

This talk was first presented at the 2015 Strata+Hadoop World NYC (http://strataconf.com/big-data-conference-ny-2015/public/schedule/detail/43484) Deep learning algorithms have been widely used in many real-world applications, including computer vision, machine translation, and fraud detection. Unfortunately, deep learning only works best when the model is big and trained on large-scale datasets. Meanwhile, distributed computing platforms like Spark are designed to handle big data, and have been used extensively. By having deep learning available on Spark, businesses can fully take advantage of deep learning capabilities on their datasets using their existing Spark infrastructure. In this talk, we present a scalable implementation of predictive deep learning algorithms on Spark, including feedforward neural networks, convolutional neural networks (CNNs), and recurrent neural networks (RNNs). This, to our best knowledge, is the first successful implementation of CNNs and RNNs on Spark. To support big model training, we use Tachyon as common storage layers between the Spark workers. With its in-memory distributed execution model, Tachyon provides a scalable approach even when the model is too big to be handled on a single machine. Our solution also exploits graphical processing units (GPUs) for matrix computation whenever they are available on worker nodes, further improving execution time. The attendees will learn about deep learning models, the architecture of the system, and how to train and run deep learning models on Spark with Tachyon.

First-ever scalable, distributed deep learning architecture using Spark & Tac...

Arimo, Inc.

Presentation by TachyonNexus & Intel at Strata Singapore 2015

Tachyon Nexus, Inc.

Presentation by TachyonNexus & Baidu at Strata Singapore 2015

Tachyon Nexus, Inc.

Tachyon Presentation at AMPCamp 6 (November, 2015)

Tachyon Nexus, Inc.

Spark Summit EU talk by Jiri Simsa

Alluxio, Inc.

FCSarch 09 Gothic

jdankoff

La spectroscopie Raman appliquée au contrôle de qualité analytique des médicaments injectables : évaluation analytique et économique comparative versus CLHP et UV/visible-IRTF. La centralisation hospitalière, sous couvert pharmaceutique, du façonnage des solutions injectables des médicaments anticancéreux est devenue obligation légale en France. Nous avons établi de longue date que les Objets Thérapeutiques (OTs) ainsi fabriqués devraient tous bénéficier d’un Contrôle de Qualité Analytique (CQA) idéalement Libératoire (CQAL) pour les paramètres clés d’identité, de pureté et de concentration nominale en tel ou tel principe actif. Au cours de travaux récents, nous avons établi dans plusieurs situations, la supériorité technique de la Spectroscopie Raman (SR) non intrusive sur toute autre solution analytique et notamment sur les méthodes séparatives chromatographiques CLHP ainsi que sur l’analyse vibrationnelle par UV/visible-IRTF. Une évaluation à la fois qualitative, économique et environnementale croisée vient enrichir ces travaux. Les 3 modèles analytiques ont a été comparés en situation opérationnelle au moyen : a) d’une grille de critères qualitatifs, b) de tableaux d’amortissement des équipements, c) des coûts en biens consommables, d) du poids éventuel des équipements de support et des locaux, e) de l’Unité d’Œuvre (UO) dont le coût direct composite (€) varie en fonction de l’option technique retenue, de la charge catégorielle en ressources humaines (RHs) opérateurs (diverses combinaisons sont possibles) et, de l’éventuelle fraction de biens consommables. L’UO peut ainsi prendre 12 valeurs possibles comprises entre les bornes extrêmes de 1 à 5,5 €. Une grille d’évaluation qualitative et de performance positionne la technologie SR non intrusive comme constamment supérieure ou équivalente aux 2 autres techniques éprouvées. Les données rapportées confirment le caractère prometteur de la SR pour le CQA non intrusif et y compris dans des situations inaccessibles aux autres techniques. Nos résultats confirment la place centrale que pourra dans l’avenir occuper cette solution d’exploration contextuelle d’objets de géométrie quelconque e.g. diffuseurs portables. La SR apparaît comme un fort contributeur à la sécurisation du circuit (ou de la filière) du médicament injectable ainsi qu’à la protection des acteurs de soins et de leur environnement de travail.

SFPC 2014 - Communication orale - SR & CQAL - A. Amin et col.

pharmex

Combined Spinal Epidural Anesthesia

Bilal Baig

Destaque (11)

Tachyon workshop 2015-07-19

Accelerating Machine Learning Pipelines with Alluxio at Alluxio Meetup 2016

Tachyon: An Open Source Memory-Centric Distributed Storage System

First-ever scalable, distributed deep learning architecture using Spark & Tac...

Presentation by TachyonNexus & Intel at Strata Singapore 2015

Presentation by TachyonNexus & Baidu at Strata Singapore 2015

Tachyon Presentation at AMPCamp 6 (November, 2015)

Spark Summit EU talk by Jiri Simsa

FCSarch 09 Gothic

SFPC 2014 - Communication orale - SR & CQAL - A. Amin et col.

Combined Spinal Epidural Anesthesia

Semelhante a The Missing Piece of On-Demand Clusters

Boomi Molecule Migration to the Cloud: Top 5 Strategies Revealed

Kellton Tech Solutions Ltd

Distributed Query Service Powered By Presto & Alluxio Across Clouds @Walmart...

Ashish Tadose

Cloud Computing Basics

DataNext Solutions

Enterprise Distributed Query Service powered by Presto & Alluxio across cloud...

Alluxio, Inc.

When small problems become big problems

Adrian Cole

Today everyone is talking about clouds, and a few are building them, but far fewer are operating successful clouds. In this session we'll examine a variety of paradigm shifts IT makes when moving from a traditional virtualization and management mindset to operating a successful cloud. For most organizations, without careful planning the hype of a cloud solution can quickly overcome its capabilities and pre-existing best practices can combine to create the worst possible cloud scenario -- a cloud which isn't economical to operate, and which is more cumbersome to manage than a traditional virtualization farm. Key topics covered include: - Successful transition of operational and management paradigm - How the VM density of clouds change Ops - What it means to monitor the network in a cloud environment, at hyper-dense virtualization levels - Preventing storage costs from outpacing delivery costs

Taming the cost of your first cloud - CCCEU 2014

Tim Mackey

Migrate to platform of your choice

Ashnikbiz

Cloud Computing

Yong Heui Cho

Enterprise Distributed Query Service powered by Presto & Alluxio across cloud...

Shubham Tagra

Building a Hybrid Cloud Solution

Cloudian

Apps Associates - Insmed INNOVATE16- EBS on the AWSCloudREV

Christopher Colucci

Transforming Enterprises through Next-generation Cloud Applications

Tata Consultancy Services

Architecting on The Cloud

Amazon Web Services

What's new CloudBerry Backup 5.7

Evgeny Rudinsky

Learn why Snowflake analytic data warehouse makes sense for BI including data loading flexibility and scalability, consumption-based storage and compute costs, Time Travel and data sharing features, support across a range of BI tools like Power BI and Tableau and ability to allocate compute costs. View this on-demand webinar: https://senturus.com/resources/10-reasons-snowflake-is-great-for-analytics/. Senturus offers a full spectrum of services in business intelligence and training on Cognos, Tableau and Power BI. Our resource library has hundreds of free live and recorded webinars, blog posts, demos and unbiased product reviews available on our website at: http://www.senturus.com/senturus-resources/.

10 Reasons Snowflake Is Great for Analytics

Senturus

Venture capitalist Matt Ocko’s 20-year track record of success in the startup world has given him unique insight into how AWS has changed the venture financing process. In this session, you’ll learn about industries susceptible to disruption by AWS-based startups, and where VCs are willing to take new risks on those startups, including the heavily-regulated medical, government, financial, and industrial sectors. Matt will talk about how new, supercomputing startups are now possible because of AWS technologies. Hear about how using AWS technologies can actually reduce risk – and reduce time to customer penetration – from a VC perspective, and how to go from ‘AWS to Series A’ in 5 easy pieces.

From AWS to Series A in 5 Easy Pieces

Amazon Web Services

Alluxio Tech Talk Mar 12, 2019 Speaker: Bin Fan, Alluxio Matt Fuller, Starburst As data analytic needs have increased with the explosion of data, the importance of the speed of analytics and the interactivity of queries has increased dramatically In this tech talk, we will introduce the Starburst Presto, Alluxio, and cloud object store stack for building a highly-concurrent and low-latency analytics platform. This stack provides a strong solution to run fast SQL across multiple storage systems including HDFS, S3, and others in public cloud, hybrid cloud, and multi-cloud environments. You’ll learn about: - The architecture of Presto, an open source distributed SQL engine, as well as innovations by Starburst like as it’s cost-based optimizer - How Presto can query data from cloud object storage like S3 at high performance and cost-effectively with Alluxio - How to achieve data locality and cross-job caching with Alluxio no matter where the data is persisted and reduce egress costs In addition, we’ll present some real world architectures & use cases from internet companies like JD.com and NetEase.com running the Presto and Alluxio stack at the scale of hundreds of nodes.

Interactive Analytics with the Starburst Presto + Alluxio stack for the Cloud

Alluxio, Inc.

Microsoft applications like SharePoint and Exchange are excellent candidates to move to the cloud, because they readily benefit from the scalability and low storage costs that Amazon Web Services offers. Review this webinar, led by the AWS Premier Consulting Partner Smartronix, to learn best practices in designing, building, and managing Microsoft workloads on AWS. Webinar topics include: • Cloud deployment options best suited to Microsoft SharePoint and Exchange environments • Governance, security, and compliance considerations when migrating Microsoft workloads • Proven methods for enabling enterprise-class Microsoft solutions on AWS.

AWS Partner Webcast - Rapid and Reliable Implementation of Microsoft Workload...

Amazon Web Services

Unified Big Data Analytics: Any Stack, Any Cloud

Alluxio, Inc.

Automating Infrastructure as a Service Deployments and monitoring – TEC213

Chris Kernaghan

Semelhante a The Missing Piece of On-Demand Clusters (20)

Boomi Molecule Migration to the Cloud: Top 5 Strategies Revealed

Distributed Query Service Powered By Presto & Alluxio Across Clouds @Walmart...

Cloud Computing Basics

Enterprise Distributed Query Service powered by Presto & Alluxio across cloud...

When small problems become big problems

Taming the cost of your first cloud - CCCEU 2014

Migrate to platform of your choice

Cloud Computing

Enterprise Distributed Query Service powered by Presto & Alluxio across cloud...

Building a Hybrid Cloud Solution

Apps Associates - Insmed INNOVATE16- EBS on the AWSCloudREV

Transforming Enterprises through Next-generation Cloud Applications

Architecting on The Cloud

What's new CloudBerry Backup 5.7

10 Reasons Snowflake Is Great for Analytics

From AWS to Series A in 5 Easy Pieces

Interactive Analytics with the Starburst Presto + Alluxio stack for the Cloud

AWS Partner Webcast - Rapid and Reliable Implementation of Microsoft Workload...

Unified Big Data Analytics: Any Stack, Any Cloud

Automating Infrastructure as a Service Deployments and monitoring – TEC213

Mais de Alluxio, Inc.

As GenAI and AI continue to transform businesses, scaling these workloads requires optimized underlying infrastructure. A multi-cloud architecture allows organizations to leverage different cloud services to meet diverse workload demands while maximizing efficiency, reducing costs, and avoiding vendor lock-in. However, achieving a multi-cloud vision can be challenging. In this webinar, Tarik will share how an agonistic data layer, like Alluxio, allows you to embrace the separation of storage from compute and simplify the adoption of multi-cloud for AI. - Learn why leveraging multiple cloud providers is critical for balancing performance, scalability, and cost of your AI platform - Discover how an agnostic data layer like Alluxio provides seamless data access in multi-cloud that bridges storage and compute without data replication - Gain insights into real-world examples and best practices for deploying AI across on-prem, hybrid, and multi-cloud environments

Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data

Alluxio, Inc.

Optimizing Data Access for Analytics And AI with Alluxio

Alluxio, Inc.

Speed Up Presto at Uber with Alluxio Caching

Alluxio, Inc.

Alluxio x Tobiko - ETL Happy Hour April 16, 2024 For more Alluxio events: https://alluxio.io/events/ Speaker: Toby Mao (CTO @ Tobiko Data) Writing efficient and correct incremental pipelines is challenging. Data practitioners who take on this challenge are viewed as performing an "advanced" function, which discourages broader teams from adopting incremental loads. In this lightning talk, CTO of Tobiko Data, Toby Mao, will demystify incremental loading data at scale.

Correctly Loading Incremental Data at Scale

Alluxio, Inc.

Big Data Bellevue Meetup March 21, 2024 For more Alluxio events: https://alluxio.io/events/ Speakers: Bin Fan (VP of Open Source, Alluxio) In this presentation, Bin Fan (VP of Open Source @ Alluxio) will address a critical challenge of optimizing data loading for distributed Python applications within AI/ML workloads in the cloud, focusing on popular frameworks like Ray and Hugging Face. Integration of Alluxio’s distributed caching for Python applications is accomplished using the fsspec interface, thus greatly improving data access speeds. This is particularly useful in machine learning workflows, where repeated data reloading across slow, unstable or congested networks can severely affect GPU efficiency and escalate operational costs. Attendees can look forward to practical, hands-on demonstrations showcasing the tangible benefits of Alluxio’s caching mechanism across various real-world scenarios. These demos will highlight the enhancements in data efficiency and overall performance of data-intensive Python applications. This presentation is tailored for developers and data scientists eager to optimize their AI/ML workloads. Discover strategies to accelerate your data processing tasks, making them not only faster but also more cost-efficient.

Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML

Alluxio, Inc.

Alluxio Monthly Webinar Feb. 27, 2024 For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Tarik Bennett (Senior Solutions Engineer, Alluxio) As GenAI and AI continue to transform businesses, scaling these workloads requires optimized underlying infrastructure. A multi-cloud architecture allows organizations to leverage different cloud services to meet diverse workload demands while maximizing efficiency, reducing costs, and avoiding vendor lock-in. However, achieving a multi-cloud vision can be challenging. In this webinar, Tarik will share how an agonistic data layer, like Alluxio, allows you to embrace the separation of storage from compute and simplify the adoption of multi-cloud for AI. - Learn why leveraging multiple cloud providers is critical for balancing performance, scalability, and cost of your AI platform - Discover how an agnostic data layer like Alluxio provides seamless data access in multi-cloud that bridges storage and compute without data replication - Gain insights into real-world examples and best practices for deploying AI across on-prem, hybrid, and multi-cloud environments

Alluxio Monthly Webinar | Why a Multi-Cloud Strategy Matters for Your AI Plat...

Alluxio, Inc.

Alluxio Monthly Webinar Jan. 30, 2024 For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Kevin Petrie (VP of Research, Eckerson Group) - Omid Razavi (SVP of Customer Success, Alluxio) 2024 is gearing up to be an impactful year for AI and analytics. Join us on January 30, as Kevin Petrie (VP of Research at Eckerson Group) and Omid Razavi (SVP of Customer Success at Alluxio) share key trends that data and AI leaders should know. This event will efficiently guide you with market data and expert insights to drive successful business outcomes. - Assess current and future trends in data and AI with industry experts - Discover valuable insights and practical recommendations - Learn best practices to make your enterprise data more accessible for both analytics and AI applications

Alluxio Monthly Webinar | Five Disruptive Trends that Every Data & AI Leader...

Alluxio, Inc.

Data Infra Meetup Jan. 25, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Juncheng Yang(Ph.D Candidate, @CMU) As a cache eviction algorithm, FIFO has a lot of attractive properties, such as simplicity, speed, scalability, and flash-friendliness. The most prominent criticism of FIFO is its low efficiency (high miss ratio). In this talk, I will describe a simple, scalable FIFO-based algorithm with three static queues (S3-FIFO). Evaluated on 6594 cache traces from 14 datasets, we show that S3- FIFO has lower miss ratios than state-of-the-art algorithms across traces. Moreover, S3-FIFO’s efficiency is robust — it has the lowest mean miss ratio on 10 of the 14 datasets. FIFO queues enable S3-FIFO to achieve good scalability with 6× higher throughput compared to optimized LRU at 16 threads. Our insight is that most objects in skewed workloads will only be accessed once in a short window, so it is critical to evict them early (also called quick demotion). The key of S3-FIFO is a small FIFO queue that filters out most objects from entering the main cache, which provides a guaranteed demotion speed and high demotion precision.

Data Infra Meetup | FIFO Queues are All You Need for Cache Eviction

Alluxio, Inc.

Data Infra Meetup Jan. 25, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Jingwen Ouyang (Product Manager, @Alluxio) In this session, Jingwen presents an overview of using Alluxio Edge caching to accelerate Trino or Presto queries. She offers practical best practices for using distributed caching with compute engines. In addition, this session also features insights from real-world examples.

Data Infra Meetup | Accelerate Your Trino/Presto Queries - Gain the Alluxio Edge

Alluxio, Inc.

Data Infra Meetup Jan. 25, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Siyuan Sheng (Senior Software Engineer, @Alluxio) - Chunxu Tang (Research Scientist, @Alluxio) In this session, cloud optimization specialists Chunxu and Siyuan break down the challenges and present a fresh architecture designed to optimize I/O across the data pipeline, ensuring GPUs function at peak performance. The integrated solution of PyTorch/Ray + Alluxio + S3 offers a promising way forward, and the speakers delve deep into its practical applications. Attendees will not only gain theoretical insights but will also be treated to hands-on instructions and demonstrations of deploying this cutting-edge architecture in Kubernetes, specifically tailored for Tensorflow/PyTorch/Ray workloads in the public cloud.

Data Infra Meetup | Accelerate Distributed PyTorch/Ray Workloads in the Cloud

Alluxio, Inc.

Data Infra Meetup | ByteDance's Native Parquet Reader

Alluxio, Inc.

Data Infra Meetup Jan. 25, 2024 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Jing Zhao (Principal Engineer, @Uber) Uber builds one of the biggest data lakes in the industry, which stores exabytes of data. In this talk, we will introduce the evolution of our data storage architecture, and delve into multiple key initiatives during the past several years. Specifically, we will introduce: - Our on-prem HDFS cluster scalability challenges and how we solved them - Our efficiency optimizations that significantly reduced the storage overhead and unit cost without compromising reliability and performance - The challenges we are facing during the ongoing Cloud migration and our solutions

Data Infra Meetup | Uber's Data Storage Evolution

Alluxio, Inc.

Alluxio Monthly Webinar Nov. 15, 2023 For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Tarik Bennett (Senior Solutions Engineer) - Beinan Wang (Senior Staff Engineer & Architect) Many companies are working with development architectures for AI platforms but have concerns about efficiency at scale as data volumes increase. They use centralized cloud data lakes, like S3, to store training data for AI platforms. However, GPU shortages add more complications. Storage and compute can be separate, or even remote, making data loading slow and expensive: 1) Optimizing a developmental setup can include manual copies, which are slow and error-prone 2) Directly transferring data across regions or from cloud to on-premises can incur expensive egress fees This webinar covers solutions to improve data loading for model training. You will learn: - The data loading challenges with distributed infrastructure - Typical solutions, including NFS/NAS on object storage, and why they are not the best options - Common architectures that can improve data loading and cost efficiency - Using Alluxio to accelerate model training and reduce costs

Alluxio Monthly Webinar | Why NFS/NAS on Object Storage May Not Solve Your AI...

Alluxio, Inc.

AI Infra Day Oct. 25, 2023 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Adit Madan (Director of Product Management, @Alluxio) In this session, Adit Madan, Director of Product Management at Alluxio, presents an overview of using distributed caching to accelerate model training and serving. He explores the requirements of data access patterns in the ML pipeline and offers practical best practices for using distributed caching in the cloud. This session features insights from real-world examples, such as AliPay, Zhihu, and more.

AI Infra Day | Accelerate Your Model Training and Serving with Distributed Ca...

Alluxio, Inc.

AI Infra Day Oct. 25, 2023 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Bin Fan (Cheif Architect, VP of Open Source, @Alluxio) As the AI landscape rapidly evolves, the advancements in generative AI technologies, such as ChatGPT, are driving a need for a robust AI infra stack. This opening keynote will explore the key trends of the AI infra stack in the generative AI era.

AI Infra Day | The AI Infra in the Generative AI Era

Alluxio, Inc.

AI Infra Day Oct. 25, 2023 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Lu Qiu (Machine Learning Engineer, @Alluxio) - Shawn Sun (Software Engineer, @Alluxio) This hands-on session will discuss best practices for using PyTorch and Alluxio during model training on AWS. Chunxu and Lu will provide a step-by-step demonstration of how to use Alluxio on EKS as a distributed cache to accelerate computer vision model training jobs that read datasets from S3. This architecture significantly improves the utilization of GPUs from 30% to 90%+, archives ~5x faster training, and lower cloud storage costs.

AI Infra Day | Hands-on Lab: CV Model Training with PyTorch & Alluxio on Kube...

Alluxio, Inc.

AI Infra Day Oct. 25, 2023 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Jordan Plawner (Global Director of Artificial intelligence Product Management and Strategy, @Intel) ChatGPT and other massive models represents an amazing step forward in AI, yet they do not solve real-world business problems. We will survey how the AI ecosystem has worked non-stop over this last year to take these all-purpose multi-task models and optimize them to they can be used by organizations to address domain specific problems. We will explain these new AI-for-the-real world techniques and methods such as fine tuning and how can be applied to deliver results which are highly performant with state-of-the-art accuracy while also being economical to build and deploy everywhere to enhance products and services.

AI Infra Day | The Generative AI Market And Intel AI Strategy and Product Up...

Alluxio, Inc.

AI Infra Day | Composable PyTorch Distributed with PT2 @ Meta

Alluxio, Inc.

AI Infra Day Oct. 25, 2023 Organized by Alluxio For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Sally (Mihyoung) Lee (Senior Staff Engineer, TLM, @Uber) Machine learning models power Uber’s everyday business. However, developing and deploying a model is not a one-time event but a continuous process that requires careful planning, execution, and monitoring. This session will highlight Uber’s practice on the machine learning lifecycle to ensure high model quality.

AI Infra Day | Model Lifecycle Management Quality Assurance at Uber Scale

Alluxio, Inc.

Alluxio Monthly Webinar Oct. 3, 2023 For more Alluxio Events: https://www.alluxio.io/events/ Speaker: - Greg Palmer (Lead Solutions Engineer) Model training requires extensive computational and GPU resources. When training models on AWS, loading data from S3 often becomes a major bottleneck, wasting valuable GPU cycles. Optimizing data loading can greatly reduce GPU idle time and increase GPU utilization. In this webinar, Greg Palmer will discuss best practices for efficient data loading during model training on AWS. He will demonstrate how to use Alluxio on EKS as a distributed cache to accelerate PyTorch training jobs that read datasets from S3. This architecture significantly improves the utilization of GPUs from 30% to 90%+, archives ~5x faster training, and lower cloud storage costs. What you will learn: - The challenges of feeding data-hungry GPUs in the cloud - How to accelerate model training by optimizing data loading on AWS - The reference architecture for running PyTorch jobs with Alluxio cache on EKS while reading data from S3, with benchmark results of training ResNet50 and BERT - How to use TensorBoard to identify bottlenecks in GPU utilization

Alluxio Monthly Webinar | Efficient Data Loading for Model Training on AWS

Alluxio, Inc.

Mais de Alluxio, Inc. (20)

Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data

Optimizing Data Access for Analytics And AI with Alluxio

Speed Up Presto at Uber with Alluxio Caching

Correctly Loading Incremental Data at Scale

Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML

Alluxio Monthly Webinar | Why a Multi-Cloud Strategy Matters for Your AI Plat...

Alluxio Monthly Webinar | Five Disruptive Trends that Every Data & AI Leader...

Data Infra Meetup | FIFO Queues are All You Need for Cache Eviction

Data Infra Meetup | Accelerate Your Trino/Presto Queries - Gain the Alluxio Edge

Data Infra Meetup | Accelerate Distributed PyTorch/Ray Workloads in the Cloud

Data Infra Meetup | ByteDance's Native Parquet Reader

Data Infra Meetup | Uber's Data Storage Evolution

Alluxio Monthly Webinar | Why NFS/NAS on Object Storage May Not Solve Your AI...

AI Infra Day | Accelerate Your Model Training and Serving with Distributed Ca...

AI Infra Day | The AI Infra in the Generative AI Era

AI Infra Day | Hands-on Lab: CV Model Training with PyTorch & Alluxio on Kube...

AI Infra Day | The Generative AI Market And Intel AI Strategy and Product Up...

AI Infra Day | Composable PyTorch Distributed with PT2 @ Meta

AI Infra Day | Model Lifecycle Management Quality Assurance at Uber Scale

Alluxio Monthly Webinar | Efficient Data Loading for Model Training on AWS

Último

+971565801893 Mtp-Kit (500MG) Prices » Dubai [(+971565801893**)] Abortion Pills For Sale In Dubai, UAE, Mifepristone and Misoprostol Tablets Available In Dubai, UAE CONTACT DR.Leen Whatsapp +971565801893 We Have Abortion Pills / Cytotec Tablets /Mifegest Kit Available in Dubai, Sharjah, Abudhabi, Ajman, Alain, Fujairah, Ras Al Khaimah, Umm Al Quwain, UAE, Buy cytotec in Dubai +971565801893''''Abortion Pills near me DUBAI | ABU DHABI|UAE. Price of Misoprostol, Cytotec” +971565801893' Dr.DEEM ''BUY ABORTION PILLS MIFEGEST KIT, MISOPROTONE, CYTOTEC PILLS IN DUBAI, ABU DHABI,UAE'' Contact me now via What's App…… abortion Pills Cytotec also available Oman Qatar Doha Saudi Arabia Bahrain Above all, Cytotec Abortion Pills are Available In Dubai / UAE, you will be very happy to do abortion in Dubai we are providing cytotec 200mg abortion pill in Dubai, UAE. Medication abortion offers an alternative to Surgical Abortion for women in the early weeks of pregnancy. We only offer abortion pills from 1 week-6 Months. We then advise you to use surgery if its beyond 6 months. Our Abu Dhabi, Ajman, Al Ain, Dubai, Fujairah, Ras Al Khaimah (RAK), Sharjah, Umm Al Quwain (UAQ) United Arab Emirates Abortion Clinic provides the safest and most advanced techniques for providing non-surgical, medical and surgical abortion methods for early through late second trimester, including the Abortion By Pill Procedure (RU 486, Mifeprex, Mifepristone, early options French Abortion Pill), Tamoxifen, Methotrexate and Cytotec (Misoprostol). The Abu Dhabi, United Arab Emirates Abortion Clinic performs Same Day Abortion Procedure using medications that are taken on the first day of the office visit and will cause the abortion to occur generally within 4 to 6 hours (as early as 30 minutes) for patients who are 3 to 12 weeks pregnant. When Mifepristone and Misoprostol are used, 50% of patients complete in 4 to 6 hours; 75% to 80% in 12 hours; and 90% in 24 hours. We use a regimen that allows for completion without the need for surgery 99% of the time. All advanced second trimester and late term pregnancies at our Tampa clinic (17 to 24 weeks or greater) can be completed within 24 hours or less 99% of the time without the need surgery. The procedure is completed with minimal to no complications. Our Women's Health Center located in Abu Dhabi, United Arab Emirates, uses the latest medications for medical abortions (RU-486, Mifeprex, Mifegyne, Mifepristone, early options French abortion pill), Methotrexate and Cytotec (Misoprostol). The safety standards of our Abu Dhabi, United Arab Emirates Abortion Doctors remain unparalleled. They consistently maintain the lowest complication rates throughout the nation. Our Physicians and staff are always available to answer questions and care for women in one of the most difficult times in their lives. The decision to have an abortion at the Abortion Clinic in Abu Dhabi, United Arab Emirates.+971565801893

+971565801893>>SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHAB...

Health

(Vivek)Call Us, 8448380779,Call girls in Delhi NCr – We Offer best in class call girls. escort Service At Affordable Price At low Rate with Space Night 8000 We Are One Of The Oldest Escort and Call girls Agencies in Delhi. You Will Find That Our Female Escorts Are Full Of Fun, Sexy And They Would Love Enjoy Your Company. We Have A Fantastic Selection Of Escort Ladies Available For In-Calls As Well As Out-Calls. Our Escorts Are Not Only Beautiful But All Have Great Personalities Making Them The Perfect Companion For Any Occasion. In-Call:- You Can Come At Our Place in Delhi Our place Which Is Very Clean Hygienic 100% safe Accommodation. Out-Call:- You have To Come Pick The Girl From My Place We Are Also Provide Door Step Services (Delhi Ncr, Noida, Gurgaon, Faridabad, Ghaziabad Note:- Pic Collectors Time Passers Bargainers Stay Away As We Respect The Value For Your Money Time And Expect The Same From You Hygienic:- Full Ac room And Clean Rooms Available In Hotel 24 * 7 Hourly In Delhi NCR More Details, With WhatsApp Number, +91-8448380779

call girls in Vaishali (Ghaziabad) 🔝 >༒8448380779 🔝 genuine Escort Service 🔝✔️✔️

Delhi Call girls

Foundation models are machine learning models which are easily capable of performing variable tasks on large and huge datasets. FMs have managed to get a lot of attention due to this feature of handling large datasets. It can do text generation, video editing to protein folding and robotics. In case we believe that FMs can help the hospitals and patients in any way, we need to perform some important evaluations, tests to test these assumptions. In this review, we take a walk through Fms and their evaluation regimes assumed clinical value. To clarify on this topic, we reviewed no less than 80 clinical FMs built from the EMR data. We added all the models trained on structured and unstructured data. We are referring to this combination of structured and unstructured EMR data or clinical data.

Reassessing the Bedrock of Clinical Function Models: An Examination of Large ...

harshavardhanraghave

Unlocking the Future of AI Agents with Large Language Models

aagamshah0812

Document contains steps to getting ups and running quickly with MyTimeClock Employee Scheduling and Time Keeping Cloud Software as a Service Solution, Web version. Try MyTimeClock or any of our other software packages risk-free by registering for a FREE ACCOUNT at https://register.myintellisource.com/. If you would like more information about our company or its software, follow us on Facebook, Instagram, LinkedIn, Twitter, or YouTube, visit our home page at https://www.myintellisource.com/, or send us an email at cs@myintellisource.com. Take care and have a great day.

Steps To Getting Up And Running Quickly With MyTimeClock Employee Scheduling ...

MyIntelliSource, Inc.

In an era where security concerns are paramount, the integration of artificial intelligence (AI) into CCTV cameras has revolutionized surveillance capabilities. One of the most significant advancements is the ability to achieve real-time threat detection, enabling immediate responses to potential security breaches. This blog explores how AI is reshaping surveillance through real-time threat detection and the implications of this technology.

Optimizing AI for immediate response in Smart CCTV

shikhaohhpro

At TECUNIQUE, we're a stable and steadily growing Indian software services company with over 14 years of industry experience. Specializing in offshore software development and quality assurance services, we've built a reputation for delivering unique and effective solutions to start-ups, software development companies, enterprises, and digital agencies. We pride ourselves on our commitment to excellence and innovation. By blending insightful business domain knowledge with exceptional technical prowess, we craft tailor-made solutions that meet the unique needs of our clients. Our dedicated teams are adept in specific technologies, ensuring seamless integration of skills and delivering reliable, scalable, and high-quality software solutions aligned with our clients' preferences. Bespoke Dedicated Teams: Crafted to meet your specific needs and technology preferences, our dedicated teams are committed to delivering top-notch software solutions. Offshore Software Development: Accelerate your software development and scale up quickly with our 12+ years of expertise in offshore development. Quality Assurance Services: Ensure the quality of your software products with our dedicated teams of experienced QA professionals. IT Staff Augmentation: Overcome skill gaps with our client-centric software team, offering staff augmentation services. Expert Software Services: Unlock our capabilities in custom software development, product development, and quality assurance. Mission and Vision: Our mission at TECUNIQUE is to be the catalyst for our clients' success in the dynamic domain of software development. Rooted in our core values of respect, authenticity, and responsibility, we strive to ease the software outsourcing experience, reducing both time and cost to market for our clients. We envision ourselves as the leading Indian software services company, renowned for our unwavering commitment to excellence and innovation. www.tecunique.com

TECUNIQUE: Success Stories: IT Service provider

mohitmore19

5 Signs You Need a Fashion PLM Software.pdf

Wave PLM

Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live Booking Contact Details :- WhatsApp Chat :- [+91-9999965857 ] The Best Call Girls Delhi At Your Service Russian Call Girls Delhi Doing anything intimate with can be a wonderful way to unwind from life's stresses, while having some fun. These girls specialize in providing sexual pleasure that will satisfy your fetishes; from tease and seduce their clients to keeping it all confidential - these services are also available both install and outcall, making them great additions for parties or business events alike. Their expert sex skills include deep penetration, oral sex, cum eating and cum eating - always respecting your wishes as part of the experience (29-April-2024(PSS)

Vip Call Girls Noida ➡️ Delhi ➡️ 9999965857 No Advance 24HRS Live

Call Girls In Delhi Whatsup 9873940964 Enjoy Unlimited Pleasure

Conference: Engage2024 in Antwerp Type: Workshop Speakers: Florian Vogler, Henning Kunz, Christoph Adler Title: Navigating the Future with The Hitchhiker's Guide to Notes and Domino 14 Abstract: Embark on an exhilarating journey with industry trailblazers Florian Vogler, Henning Kunz, and Christoph Adler in this not-to-be-missed workshop at the forefront of the tech universe. Get ready for a thrilling kick-off as we navigate the current state of the HCL universe, setting the stage for an exploration of the groundbreaking Notes and Domino 14. Discover the latest enhancements and revolutionary features that will redefine your experience. In this interactive session, unlock a treasure trove of tips and tricks to elevate your utilization of version 14, both with and without the game-changing panagenda MarvelClient. Brace yourself for also diving into Nomad, Nomad Web, and VoltMX, expanding your horizons in the expansive HCL landscape. Be a part of this exclusive opportunity to stay ahead in the ever-evolving world of HCL technologies. Your journey to mastering Notes and Domino 14 begins here. And remember, in the spirit of intergalactic exploration, don't forget to bring your towel!

W01_panagenda_Navigating-the-Future-with-The-Hitchhikers-Guide-to-Notes-and-D...

panagenda

👉 𝑾𝑰𝑳𝑳 𝑷𝑹𝑶𝑽𝑰𝑫𝑬 𝒀𝑶𝑼 𝑾𝑰𝑻𝑯 𝑺𝑬𝑿𝒀 𝑴𝑶𝑫𝑬𝑳𝑺 𝑾𝑯𝑶 𝑾𝑰𝑳𝑳 𝑫𝑨𝑵𝑪𝑬 & 𝑫𝑹𝑰𝑵𝑲 𝑾𝑰𝑻𝑯 𝒀𝑶𝑼 𝑨𝑵𝑫 𝑨𝑳𝑺𝑶 𝑷𝑹𝑶𝑽𝑰𝑫𝑬 𝒀𝑶𝑼 𝑺𝑬𝑿𝑼𝑨𝑳 𝑩𝑶𝑫𝒀 𝑻𝑶 𝑩𝑶𝑫𝒀 𝑴𝑨𝑺𝑺𝑨𝑮𝑬 𝑾𝑰𝑻𝑯 𝑺𝑬𝑿. 👉𝒀𝑶𝑼 𝑴𝑨𝒀 𝑻𝑨𝑲𝑬 𝑻𝑯𝑬𝑴 𝑶𝑼𝑻 𝑭𝑶𝑹 𝑨 𝑷𝑨𝑹𝑻𝒀 𝑶𝑹 𝑨𝑳𝑺𝑶 𝑭𝑶𝑹 𝑨𝑵𝒀 𝑷𝑹𝑰𝑽𝑨𝑻𝑬 𝑷𝑨𝑹𝑻𝑰𝑬𝑺. 👉𝑻𝑯𝑬𝑺𝑬 𝑮𝑰𝑹𝑳𝑺 𝑨𝑹𝑬 𝑰𝑵𝑻𝑬𝑹𝑬𝑺𝑻𝑬𝑫 𝑰𝑵 𝑯𝑨𝑽𝑰𝑵𝑮 𝑺𝑶𝑴𝑬 𝑭𝑼𝑵 𝑾𝑰𝑻𝑯 𝒀𝑶𝑼 𝑨𝑵𝑫 𝑾𝑰𝑳𝑳 𝑬𝑵𝑺𝑼𝑹𝑬 𝑻𝑯𝑨𝑻 𝒀𝑶𝑼 𝑯𝑨𝑽𝑬 𝑪𝑶𝑴𝑷𝑳𝑬𝑻𝑬 𝑭𝑼𝑵. 28/April/2024 {{🎗️SCH🎗️}} 8923113531 How to book call girls The Booking process is particularly time-saving and more comfortable for us. Below mentioned are the steps you need to follow to hire our call girl: Step 1 - Visit our Call Girls Service website Step 2 - check for the portfolios of our sexy and hot call girls Step 3 - check on the services provided by the specific call girls Step 4 - Once you have selected the call girls and respective services, go to the contact us page. Step 5 - on the contact page, you will get our phone number, WhatsApp number, and email address. You can choose any one of them to connect with us. Call Us – 8923113531 ✣ ✤ ✥ ✦ 𝑻𝑰𝑴𝑬 𝑾𝑨𝑺𝑻𝑬𝑹𝑺 𝑨𝑵𝑫 𝑩𝑨𝑹𝑮𝑨𝑰𝑵𝑬𝑹𝑺 𝑨𝑹𝑬 𝑷𝑳𝑬𝑨𝑺𝑬 𝑬𝑿𝑪𝑼𝑺𝑬, 𝑾𝑬 𝑹𝑬𝑺𝑷𝑬𝑪𝑻 𝒀𝑶𝑼𝑹 𝑺𝑨𝑭𝑬𝑻𝒀 𝑨𝑵𝑫 𝑷𝑹𝑰𝑽𝑨𝑪𝒀 𝑨𝑵𝑫 𝑬𝑿𝑷𝑬𝑪𝑻 𝑻𝑯𝑬 𝑺𝑨𝑴𝑬 𝑭𝑹𝑶𝑴 𝒀𝑶𝑼.✣ ✤ ✥ ✦ Duration Normal Girls Housewives High-Profile Models (Celebrities) 1 Shot ₹ 8,000 ₹ 10,000 ₹ 15,000 2 Shots ₹ 15,000 ₹ 15,000 ₹ 25,000 Full Day ₹ 35,000 ₹ 35,000 ₹ 40,000 Full Night ₹ 40,000 ₹ 40,000 ₹ 50,000 Weekend ₹ 45,000 ₹ 45,000 ₹ 50,000 Party Nights ₹ 50,000 ₹ 50,000 ₹ 60,000

CALL ON ➥8923113531 🔝Call Girls Kakori Lucknow best sexual service Online ☂️

anilsa9823

How To Use Server-Side Rendering with Nuxt.js

Andolasoft Inc

Investing in AI transformation today The modern business advantage: Uncovering deep insights with AI Organizations around the world have come to recognize AI as the transformative technology that enables them to gain real business advantage. AI’s ability to organize vast quantities of data allows those who implement it to uncover deep business insights, augment human expertise, drive operational efficiency, transform their products, and better serve their customers

Microsoft AI Transformation Partner Playbook.pdf

Willy Marroquin (WillyDevNET)

Model Call Girl Services in Delhi reach out to us at 🔝 9953056974 🔝✔️✔️ Our agency presents a selection of young, charming call girls available for bookings at Oyo Hotels. Experience high-class escort services at pocket-friendly rates, with our female escorts exuding both beauty and a delightful personality, ready to meet your desires. Whether it's Housewives, College girls, Russian girls, Muslim girls, or any other preference, we offer a diverse range of options to cater to your tastes. We provide both in-call and out-call services for your convenience. Our in-call location in Delhi ensures cleanliness, hygiene, and 100% safety, while our out-call services offer doorstep delivery for added ease. We value your time and money, hence we kindly request pic collectors, time-passers, and bargain hunters to refrain from contacting us. Our services feature various packages at competitive rates: One shot: ₹2000/in-call, ₹5000/out-call Two shots with one girl: ₹3500/in-call, ₹6000/out-call Body to body massage with sex: ₹3000/in-call Full night for one person: ₹7000/in-call, ₹10000/out-call Full night for more than 1 person: Contact us at 🔝 9953056974 🔝. for details Operating 24/7, we serve various locations in Delhi, including Green Park, Lajpat Nagar, Saket, and Hauz Khas near metro stations. For premium call girl services in Delhi 🔝 9953056974 🔝. Thank you for considering us!

CHEAP Call Girls in Pushp Vihar (-DELHI )🔝 9953056974🔝(=)/CALL GIRLS SERVICE

9953056974 Low Rate Call Girls In Saket, Delhi NCR

HR Software Buyers Guide in 2024 - HRSoftware.com

Fatema Valibhai

A Secure and Reliable Document Management System is Essential.docx

ComplianceQuest1

Learn the Fundamentals of XCUITest Framework_ A Beginner's Guide.pdf

kalichargn70th171

Introducing MyIntelliAccount™ Cloud Accounting Software as a Service (SaaS) , the complete system for simplifying business accounting needs. MyIntelliAccount Cloud Accounting SaaS is an easy to understand and easy to use application and system designed for the Web with supporting applications for iOS and Android devices. Designed to work like a natural extension of your web browser, the user interface for MyIntelliAccount Cloud Accounting SaaS is intuitive and thoughtfully organized to help you easily navigate and access your business accounting information. Because our company takes the time to research, study, and understand the applications we develop, we are confident that once you use MyIntelliAccount Cloud Accounting SaaS , you will be asking yourself how you ever got along without it.

Try MyIntelliAccount Cloud Accounting Software As A Service Solution Risk Fre...

MyIntelliSource, Inc.

Test automation is a cornerstone of software development and quality assurance in today's rapidly evolving digital landscape. Its significance cannot be overstated. Businesses can enhance efficiency, productivity, and accelerate software delivery to market through automation, streamlining testing processes effectively. This comprehensive guide addresses the best practices for test automation in 2024. It offers a detailed checklist to empower you to optimize your automation efforts and maintain a competitive edge.

The Ultimate Test Automation Guide_ Best Practices and Tips.pdf

kalichargn70th171

How To Troubleshoot Collaboration Apps for the Modern Connected Worker

ThousandEyes

The Missing Piece of On-Demand Clusters

1. The Missing Piece of On-Demand Clusters October 2016 Calvin Jia

2. About Me • Calvin Jia • Software Engineer @ Alluxio, Inc. • Alluxio PMC • #1 Alluxio Contributor • Twitter: @JiaCalvin 2

3. Outline • Cloud Architectures • Alluxio in Cloud Architectures • Takeaways 3

4. Cloud Architectures – Overview • Mostly service based, from providers – Amazon Web Services – Google Cloud Platform • Separate compute and storage clusters • Compute clusters are ephemeral 4

5. Cloud Architectures – Pros & Cons Pros • Low maintenance • Pay as you go • Elastic and scalable • Cost effective storage Cons • Lower Performance 5

6. Outline • Cloud Architectures • Alluxio in Cloud Architectures • Takeaways 6

7. Alluxio in Cloud Architectures – Overview • Deployed in compute clusters – Memory speed data access – Transparent data access to any storage • Simple to deploy – Mount storage systems similar to local disks 7

8. Alluxio in Cloud Architectures – Example 8

9. Alluxio in Cloud Architectures – Benefits 9 • Remedies the performance drawback • Acceleration due to memory-speed I/O • Designed to improve the affinity of compute and storage

10. Alluxio in Cloud Architectures – Data Path 10 FAST 104 - 105 MB/s MODERATE 103 - 104 MB/s SLOW 102 - 103 MB/s Only when necessary Limited Often SSD HDD Mem

11. Takeaways – Experiment Results 11 0 100 200 300 400 500 600 Initial Read Subsequent Read Read from Separate Job Runtime (seconds) Spark -‐ No Persist Spark -‐ Persist Alluxio

12. Takeaways – Alluxio & Cloud Architectures 12 • Cloud architectures have significant upsides • Alluxio alleviates the major downsides

13. Try out Alluxio 1.2.0 http://www.alluxio.org/releases 13

14. Read the blog athttp://www.alluxio.com/2016/08/accelerating-on-demand-data-analytics-with-alluxio Read the whitepaper at www.alluxio.com. Check out www.alluxio.org. Thank you! 14

The Missing Piece of On-Demand Clusters

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Destaque

Destaque (11)

Semelhante a The Missing Piece of On-Demand Clusters

Semelhante a The Missing Piece of On-Demand Clusters (20)

Mais de Alluxio, Inc.

Mais de Alluxio, Inc. (20)

Último

Último (20)

The Missing Piece of On-Demand Clusters