SlideShare uma empresa Scribd logo
1 de 14
Baixar para ler offline
Get Started with Alluxio on
Kubernetes
Shawn Sun - Alluxio
Shawn Sun
● Software Engineer at Alluxio
● Responsible for containerization
of Alluxio, integration with
Docker and Kubernetes
● Resource provision: network, system setup, etc.
● Multi-tenancy: different Alluxio clusters for different teams
● Easier deployment: Helm chart / Operator
● Security: containers provide isolation
Why Alluxio on Kubernetes?
Alluxio on Kubernetes - Master StatefulSet
Alluxio Master pod 0
Master
Contai
ner
Job
Master
Contai
ner
Alluxio Master pod 2
Master
Contai
ner
Job
Master
Contai
ner
Alluxio Master pod 1
Master
Contai
ner
Job
Master
Contai
ner
Master
Service 0
Master
Service 1
Master
Service 2
Alluxio on Kubernetes - Worker DaemonSet
Alluxio Worker pod
Worker
Container
Job
Worker
Container
Alluxio Worker pod
Worker
Container
Job
Worker
Container
…
K8s host machine K8s host machine
Alluxio on Kubernetes - Basic Cluster
Alluxio Master StatefulSet
Master
pod
Master
pod
Master
pod
Alluxio Worker DaemonSet
Worker
pod
Worker
pod
Worker
pod
Worker
pod
…
Alluxio clients
Deploy Alluxio on Kubernetes - Prerequisites
1. A running Kubernetes Cluster (e.g. eksctl with AWS)
2. Helm installed on the control plane of the
kubernetes cluster
a. Packages & publishes k8s resource
definitions
b. Enables templating of k8s YAMLs
c. Manages installation lifecycle in k8s
Download Alluxio helm chart
$ helm repo add alluxio-charts
https://alluxio-charts.storage.googleapis.com/openSource/2.9.3
$ helm repo list
NAME URL
alluxio-charts https://alluxio-charts.storage.googleapis.com/openSource/2.9.3
Construct configuration file - basic
properties:
alluxio.master.mount.table.root.ufs: <your ufs>
<credentials for your ufs>
shortCircuit:
enabled: false
journal:
type: EMBEDDED
volumeType: emptyDir
! Not for production, only for trying Alluxio out
Save the file as config.yaml
$ helm install alluxio -f config.yaml alluxio-charts/alluxio
NAME: alluxio
LAST DEPLOYED: Mon Apr 24 10:09:57 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
$ kubectl get po
NAME READY STATUS RESTARTS AGE
alluxio-master-0 2/2 Running 0 61s
alluxio-worker-85b8c 2/2 Running 0 61s
alluxio-worker-vmtpl 2/2 Running 0 61s
Installation
Construct configuration file - next level
# 1. Set resources (cpu, memory) for master, job master, worker, and job worker.
# 2. Set “-Xmx” and “-Xms” for these components too. Specify the allocated heap size.
journal:
type: EMBEDDED
volumeType: persistentVolumeClaim # persistent storage
storageClass: <storage for the pvc>
master:
count: 3
tieredStore:
# configure this for Alluxio to use designated storage space. NVMe is recommended.
Construct configuration file - next level
fuse:
Enabled: # Set to true if you want to use POSIX API to access data in Alluxio.
# See https://docs.alluxio.io/os/user/2.9.3/en/api/POSIX-API.html
Proxy:
Enabled: # Set to true if you want to use S3 API to access data in Alluxio.
# See https://docs.alluxio.io/os/user/2.9.3/en/api/S3-API.html
Further tuning guide can be found at
● https://docs.alluxio.io/os/user/2.9.3/en/administration/Performance-Tuning.html
● https://docs.alluxio.io/os/user/2.9.3/en/administration/Scalability-Tuning.html
Stay tuned
For next generation of Alluxio on k8s, we move the open source code to
https://github.com/Alluxio/k8s-operator. Helm chart and operator are currently
under development. Feel free to check it out and give feedbacks!
Enterprise version will come out soon too!
Thank you for attending Alluxio
k8s product school session!
QA
Complete the Alluxio Community
survey for a chance to win an
Amazon Gift Card ($50)

Mais conteúdo relacionado

Semelhante a Alluxio Product School Webinar - Get Started with Alluxio on Kubernetes

How to become cloud backup provider with Cloudian HyperStore and CloudBerry L...
How to become cloud backup provider with Cloudian HyperStore and CloudBerry L...How to become cloud backup provider with Cloudian HyperStore and CloudBerry L...
How to become cloud backup provider with Cloudian HyperStore and CloudBerry L...
Cloudian
 
kubernetes - minikube - getting started
kubernetes - minikube - getting startedkubernetes - minikube - getting started
kubernetes - minikube - getting started
Munish Mehta
 

Semelhante a Alluxio Product School Webinar - Get Started with Alluxio on Kubernetes (20)

ProxySQL on Kubernetes
ProxySQL on KubernetesProxySQL on Kubernetes
ProxySQL on Kubernetes
 
Kubernetes
KubernetesKubernetes
Kubernetes
 
Learn kubernetes in 90 minutes
Learn kubernetes in 90 minutesLearn kubernetes in 90 minutes
Learn kubernetes in 90 minutes
 
Run K8s on Local Environment
Run K8s on Local EnvironmentRun K8s on Local Environment
Run K8s on Local Environment
 
JupyterHub + kubernetes
JupyterHub + kubernetesJupyterHub + kubernetes
JupyterHub + kubernetes
 
CERN OpenStack Cloud Control Plane - From VMs to K8s
CERN OpenStack Cloud Control Plane - From VMs to K8sCERN OpenStack Cloud Control Plane - From VMs to K8s
CERN OpenStack Cloud Control Plane - From VMs to K8s
 
Build Your Own CaaS (Container as a Service)
Build Your Own CaaS (Container as a Service)Build Your Own CaaS (Container as a Service)
Build Your Own CaaS (Container as a Service)
 
[Devconf.cz][2017] Understanding OpenShift Security Context Constraints
[Devconf.cz][2017] Understanding OpenShift Security Context Constraints[Devconf.cz][2017] Understanding OpenShift Security Context Constraints
[Devconf.cz][2017] Understanding OpenShift Security Context Constraints
 
Kubernetes 101 for_penetration_testers_-_null_mumbai
Kubernetes 101 for_penetration_testers_-_null_mumbaiKubernetes 101 for_penetration_testers_-_null_mumbai
Kubernetes 101 for_penetration_testers_-_null_mumbai
 
Building an HPC Cluster in 10 Minutes
Building an HPC Cluster in 10 MinutesBuilding an HPC Cluster in 10 Minutes
Building an HPC Cluster in 10 Minutes
 
Artem Zhurbila - docker clusters (solit 2015)
Artem Zhurbila - docker clusters (solit 2015)Artem Zhurbila - docker clusters (solit 2015)
Artem Zhurbila - docker clusters (solit 2015)
 
Data weekender deploying prod grade sql 2019 big data clusters
Data weekender deploying prod grade sql 2019 big data clustersData weekender deploying prod grade sql 2019 big data clusters
Data weekender deploying prod grade sql 2019 big data clusters
 
Getting Started with Kubernetes on AWS
Getting Started with Kubernetes on AWSGetting Started with Kubernetes on AWS
Getting Started with Kubernetes on AWS
 
Hands-On Introduction to Kubernetes at LISA17
Hands-On Introduction to Kubernetes at LISA17Hands-On Introduction to Kubernetes at LISA17
Hands-On Introduction to Kubernetes at LISA17
 
How to become cloud backup provider with Cloudian HyperStore and CloudBerry L...
How to become cloud backup provider with Cloudian HyperStore and CloudBerry L...How to become cloud backup provider with Cloudian HyperStore and CloudBerry L...
How to become cloud backup provider with Cloudian HyperStore and CloudBerry L...
 
Install a micro k8s single node cluster of kubernetes on windows 10
Install a micro k8s single node cluster of kubernetes on windows 10Install a micro k8s single node cluster of kubernetes on windows 10
Install a micro k8s single node cluster of kubernetes on windows 10
 
kubernetes - minikube - getting started
kubernetes - minikube - getting startedkubernetes - minikube - getting started
kubernetes - minikube - getting started
 
Kubernetes for the PHP developer
Kubernetes for the PHP developerKubernetes for the PHP developer
Kubernetes for the PHP developer
 
Deploy the blockchain network using kubernetes ap is on google cloud
Deploy the blockchain network using kubernetes ap is on google cloudDeploy the blockchain network using kubernetes ap is on google cloud
Deploy the blockchain network using kubernetes ap is on google cloud
 
ARCHITECTING TENANT BASED QOS IN MULTI-TENANT CLOUD PLATFORMS
ARCHITECTING TENANT BASED QOS IN MULTI-TENANT CLOUD PLATFORMSARCHITECTING TENANT BASED QOS IN MULTI-TENANT CLOUD PLATFORMS
ARCHITECTING TENANT BASED QOS IN MULTI-TENANT CLOUD PLATFORMS
 

Mais de Alluxio, Inc.

Mais de Alluxio, Inc. (20)

Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed DataAlluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
Alluxio Monthly Webinar | Cloud-Native Model Training on Distributed Data
 
Optimizing Data Access for Analytics And AI with Alluxio
Optimizing Data Access for Analytics And AI with AlluxioOptimizing Data Access for Analytics And AI with Alluxio
Optimizing Data Access for Analytics And AI with Alluxio
 
Speed Up Presto at Uber with Alluxio Caching
Speed Up Presto at Uber with Alluxio CachingSpeed Up Presto at Uber with Alluxio Caching
Speed Up Presto at Uber with Alluxio Caching
 
Correctly Loading Incremental Data at Scale
Correctly Loading Incremental Data at ScaleCorrectly Loading Incremental Data at Scale
Correctly Loading Incremental Data at Scale
 
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/MLBig Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
Big Data Bellevue Meetup | Enhancing Python Data Loading in the Cloud for AI/ML
 
Alluxio Monthly Webinar | Why a Multi-Cloud Strategy Matters for Your AI Plat...
Alluxio Monthly Webinar | Why a Multi-Cloud Strategy Matters for Your AI Plat...Alluxio Monthly Webinar | Why a Multi-Cloud Strategy Matters for Your AI Plat...
Alluxio Monthly Webinar | Why a Multi-Cloud Strategy Matters for Your AI Plat...
 
Alluxio Monthly Webinar | Five Disruptive Trends that Every Data & AI Leader...
Alluxio Monthly Webinar | Five Disruptive Trends that Every  Data & AI Leader...Alluxio Monthly Webinar | Five Disruptive Trends that Every  Data & AI Leader...
Alluxio Monthly Webinar | Five Disruptive Trends that Every Data & AI Leader...
 
Data Infra Meetup | FIFO Queues are All You Need for Cache Eviction
Data Infra Meetup | FIFO Queues are All You Need for Cache EvictionData Infra Meetup | FIFO Queues are All You Need for Cache Eviction
Data Infra Meetup | FIFO Queues are All You Need for Cache Eviction
 
Data Infra Meetup | Accelerate Your Trino/Presto Queries - Gain the Alluxio Edge
Data Infra Meetup | Accelerate Your Trino/Presto Queries - Gain the Alluxio EdgeData Infra Meetup | Accelerate Your Trino/Presto Queries - Gain the Alluxio Edge
Data Infra Meetup | Accelerate Your Trino/Presto Queries - Gain the Alluxio Edge
 
Data Infra Meetup | Accelerate Distributed PyTorch/Ray Workloads in the Cloud
Data Infra Meetup | Accelerate Distributed PyTorch/Ray Workloads in the CloudData Infra Meetup | Accelerate Distributed PyTorch/Ray Workloads in the Cloud
Data Infra Meetup | Accelerate Distributed PyTorch/Ray Workloads in the Cloud
 
Data Infra Meetup | ByteDance's Native Parquet Reader
Data Infra Meetup | ByteDance's Native Parquet ReaderData Infra Meetup | ByteDance's Native Parquet Reader
Data Infra Meetup | ByteDance's Native Parquet Reader
 
Data Infra Meetup | Uber's Data Storage Evolution
Data Infra Meetup | Uber's Data Storage EvolutionData Infra Meetup | Uber's Data Storage Evolution
Data Infra Meetup | Uber's Data Storage Evolution
 
Alluxio Monthly Webinar | Why NFS/NAS on Object Storage May Not Solve Your AI...
Alluxio Monthly Webinar | Why NFS/NAS on Object Storage May Not Solve Your AI...Alluxio Monthly Webinar | Why NFS/NAS on Object Storage May Not Solve Your AI...
Alluxio Monthly Webinar | Why NFS/NAS on Object Storage May Not Solve Your AI...
 
AI Infra Day | Accelerate Your Model Training and Serving with Distributed Ca...
AI Infra Day | Accelerate Your Model Training and Serving with Distributed Ca...AI Infra Day | Accelerate Your Model Training and Serving with Distributed Ca...
AI Infra Day | Accelerate Your Model Training and Serving with Distributed Ca...
 
AI Infra Day | The AI Infra in the Generative AI Era
AI Infra Day | The AI Infra in the Generative AI EraAI Infra Day | The AI Infra in the Generative AI Era
AI Infra Day | The AI Infra in the Generative AI Era
 
AI Infra Day | Hands-on Lab: CV Model Training with PyTorch & Alluxio on Kube...
AI Infra Day | Hands-on Lab: CV Model Training with PyTorch & Alluxio on Kube...AI Infra Day | Hands-on Lab: CV Model Training with PyTorch & Alluxio on Kube...
AI Infra Day | Hands-on Lab: CV Model Training with PyTorch & Alluxio on Kube...
 
AI Infra Day | The Generative AI Market And Intel AI Strategy and Product Up...
AI Infra Day | The Generative AI Market  And Intel AI Strategy and Product Up...AI Infra Day | The Generative AI Market  And Intel AI Strategy and Product Up...
AI Infra Day | The Generative AI Market And Intel AI Strategy and Product Up...
 
AI Infra Day | Composable PyTorch Distributed with PT2 @ Meta
AI Infra Day | Composable PyTorch Distributed with PT2 @ MetaAI Infra Day | Composable PyTorch Distributed with PT2 @ Meta
AI Infra Day | Composable PyTorch Distributed with PT2 @ Meta
 
AI Infra Day | Model Lifecycle Management Quality Assurance at Uber Scale
AI Infra Day | Model Lifecycle Management Quality Assurance at Uber ScaleAI Infra Day | Model Lifecycle Management Quality Assurance at Uber Scale
AI Infra Day | Model Lifecycle Management Quality Assurance at Uber Scale
 
Alluxio Monthly Webinar | Efficient Data Loading for Model Training on AWS
Alluxio Monthly Webinar | Efficient Data Loading for Model Training on AWSAlluxio Monthly Webinar | Efficient Data Loading for Model Training on AWS
Alluxio Monthly Webinar | Efficient Data Loading for Model Training on AWS
 

Último

%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
masabamasaba
 
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
masabamasaba
 
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Medical / Health Care (+971588192166) Mifepristone and Misoprostol tablets 200mg
 
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
VictoriaMetrics
 
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
masabamasaba
 

Último (20)

%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
%+27788225528 love spells in Huntington Beach Psychic Readings, Attraction sp...
 
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
%in Stilfontein+277-882-255-28 abortion pills for sale in Stilfontein
 
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
Direct Style Effect Systems -The Print[A] Example- A Comprehension AidDirect Style Effect Systems -The Print[A] Example- A Comprehension Aid
Direct Style Effect Systems - The Print[A] Example - A Comprehension Aid
 
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
%+27788225528 love spells in Boston Psychic Readings, Attraction spells,Bring...
 
VTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learnVTU technical seminar 8Th Sem on Scikit-learn
VTU technical seminar 8Th Sem on Scikit-learn
 
Announcing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK SoftwareAnnouncing Codolex 2.0 from GDK Software
Announcing Codolex 2.0 from GDK Software
 
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
Crypto Cloud Review - How To Earn Up To $500 Per DAY Of Bitcoin 100% On AutoP...
 
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
%in Hazyview+277-882-255-28 abortion pills for sale in Hazyview
 
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa%in tembisa+277-882-255-28 abortion pills for sale in tembisa
%in tembisa+277-882-255-28 abortion pills for sale in tembisa
 
MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...
MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...
MarTech Trend 2024 Book : Marketing Technology Trends (2024 Edition) How Data...
 
%in Soweto+277-882-255-28 abortion pills for sale in soweto
%in Soweto+277-882-255-28 abortion pills for sale in soweto%in Soweto+277-882-255-28 abortion pills for sale in soweto
%in Soweto+277-882-255-28 abortion pills for sale in soweto
 
Artyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptxArtyushina_Guest lecture_YorkU CS May 2024.pptx
Artyushina_Guest lecture_YorkU CS May 2024.pptx
 
tonesoftg
tonesoftgtonesoftg
tonesoftg
 
WSO2Con204 - Hard Rock Presentation - Keynote
WSO2Con204 - Hard Rock Presentation - KeynoteWSO2Con204 - Hard Rock Presentation - Keynote
WSO2Con204 - Hard Rock Presentation - Keynote
 
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
Abortion Pill Prices Tembisa [(+27832195400*)] 🏥 Women's Abortion Clinic in T...
 
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
Large-scale Logging Made Easy: Meetup at Deutsche Bank 2024
 
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
WSO2CON 2024 - Navigating API Complexity: REST, GraphQL, gRPC, Websocket, Web...
 
WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?WSO2CON 2024 - Does Open Source Still Matter?
WSO2CON 2024 - Does Open Source Still Matter?
 
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
%+27788225528 love spells in Colorado Springs Psychic Readings, Attraction sp...
 
WSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go PlatformlessWSO2CON2024 - It's time to go Platformless
WSO2CON2024 - It's time to go Platformless
 

Alluxio Product School Webinar - Get Started with Alluxio on Kubernetes

  • 1. Get Started with Alluxio on Kubernetes Shawn Sun - Alluxio
  • 2. Shawn Sun ● Software Engineer at Alluxio ● Responsible for containerization of Alluxio, integration with Docker and Kubernetes
  • 3. ● Resource provision: network, system setup, etc. ● Multi-tenancy: different Alluxio clusters for different teams ● Easier deployment: Helm chart / Operator ● Security: containers provide isolation Why Alluxio on Kubernetes?
  • 4. Alluxio on Kubernetes - Master StatefulSet Alluxio Master pod 0 Master Contai ner Job Master Contai ner Alluxio Master pod 2 Master Contai ner Job Master Contai ner Alluxio Master pod 1 Master Contai ner Job Master Contai ner Master Service 0 Master Service 1 Master Service 2
  • 5. Alluxio on Kubernetes - Worker DaemonSet Alluxio Worker pod Worker Container Job Worker Container Alluxio Worker pod Worker Container Job Worker Container … K8s host machine K8s host machine
  • 6. Alluxio on Kubernetes - Basic Cluster Alluxio Master StatefulSet Master pod Master pod Master pod Alluxio Worker DaemonSet Worker pod Worker pod Worker pod Worker pod … Alluxio clients
  • 7. Deploy Alluxio on Kubernetes - Prerequisites 1. A running Kubernetes Cluster (e.g. eksctl with AWS) 2. Helm installed on the control plane of the kubernetes cluster a. Packages & publishes k8s resource definitions b. Enables templating of k8s YAMLs c. Manages installation lifecycle in k8s
  • 8. Download Alluxio helm chart $ helm repo add alluxio-charts https://alluxio-charts.storage.googleapis.com/openSource/2.9.3 $ helm repo list NAME URL alluxio-charts https://alluxio-charts.storage.googleapis.com/openSource/2.9.3
  • 9. Construct configuration file - basic properties: alluxio.master.mount.table.root.ufs: <your ufs> <credentials for your ufs> shortCircuit: enabled: false journal: type: EMBEDDED volumeType: emptyDir ! Not for production, only for trying Alluxio out Save the file as config.yaml
  • 10. $ helm install alluxio -f config.yaml alluxio-charts/alluxio NAME: alluxio LAST DEPLOYED: Mon Apr 24 10:09:57 2023 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None $ kubectl get po NAME READY STATUS RESTARTS AGE alluxio-master-0 2/2 Running 0 61s alluxio-worker-85b8c 2/2 Running 0 61s alluxio-worker-vmtpl 2/2 Running 0 61s Installation
  • 11. Construct configuration file - next level # 1. Set resources (cpu, memory) for master, job master, worker, and job worker. # 2. Set “-Xmx” and “-Xms” for these components too. Specify the allocated heap size. journal: type: EMBEDDED volumeType: persistentVolumeClaim # persistent storage storageClass: <storage for the pvc> master: count: 3 tieredStore: # configure this for Alluxio to use designated storage space. NVMe is recommended.
  • 12. Construct configuration file - next level fuse: Enabled: # Set to true if you want to use POSIX API to access data in Alluxio. # See https://docs.alluxio.io/os/user/2.9.3/en/api/POSIX-API.html Proxy: Enabled: # Set to true if you want to use S3 API to access data in Alluxio. # See https://docs.alluxio.io/os/user/2.9.3/en/api/S3-API.html Further tuning guide can be found at ● https://docs.alluxio.io/os/user/2.9.3/en/administration/Performance-Tuning.html ● https://docs.alluxio.io/os/user/2.9.3/en/administration/Scalability-Tuning.html
  • 13. Stay tuned For next generation of Alluxio on k8s, we move the open source code to https://github.com/Alluxio/k8s-operator. Helm chart and operator are currently under development. Feel free to check it out and give feedbacks! Enterprise version will come out soon too!
  • 14. Thank you for attending Alluxio k8s product school session! QA Complete the Alluxio Community survey for a chance to win an Amazon Gift Card ($50)