Vertex AI - Unified ML Platform for the entire AI workflow on Google Cloud

Vertex AI
Unified ML Platform for the entire
AI workflow on Google Cloud
DevFest Season, November 2021
Márton Kodok
Google Developer Expert at REEA.net

● Among the Top3 romanians on Stackoverflow 195k reputation
● Google Developer Expert on Cloud technologies
● Crafting Web/Mobile backends at REEA.net
● BigQuery + Redis database engine expert
Slideshare: martonkodok
Articles: martonkodok.medium.com
Twitter: @martonkodok
StackOverflow: pentium10
GitHub: pentium10
Vertex AI - Unified ML Platform @martonkodok
About me

1. What is Vertex AI
2. Gather, Import & label datasets
3. Build, train and deploy ML solutions
4. Manage your models with conﬁdence
5. Using Pipelines throughout your ML workﬂow
6. Adapting to changes of data
7. Conclusions
Agenda

“VertexAI is a managed ML platform for practitioners
to accelerate experiments and deploy AI models.

Where does VertexAI ﬁt in?
Application Servers
Vertex AI
Desktop client
Mobile client
End-to-end platform for
ML model development
and deployment
Backend
Application Logic

VertexAI is a uniﬁed MLOps platform
Operational
Model
Programming
Model
No Infra Management Managed Security Pay only for usage
Model-as-a-service
oriented
Streamlined model
development
Open SDKs,
integrates with ML frameworks

What’s included in VertexAI?
Data Labeling
AutoML
DL Environment (DL VM + DL Container)
Prediction
Feature Store Training
Experiments
Data Readiness
Feature
Engineering
Training/
HP-Tuning
Model
Monitoring
Model serving
Understanding/
Tuning
Edge
Model
Management
Notebooks
Pipelines (Orchestration)
Explainable AI
Hybrid AI
Model
Monitoring
Metadata
Vision Translation Tables
Language
Video
AI Accelerators
Models
Datasets
Custom Models
Containers
Python
Endpoints
Vizier Optimization

VertexAI supports...
UI based model development
# Define job
job = aiplatform.AutoMLTabularTrainingJob(
display_name='price-predict-training',
optimization_prediction_type='regression'
)
# Run job
model = job.run(
dataset=ds,
target_column='median_house_value',
model_display_name='house-value-prediction',
)
Code-based model development

Using Vertex AI throughout your ML workﬂow
Gather data Train model
Scalably
deploy
model
Evaluate,
monitor,
retrain

@martonkodok
Gather,
Import & label
datasets at
scale
Part #1

VertexAI: Gather, Import & label datasets at scale
Data Labeling
AutoML
Prediction
Experiments
Data Readiness
Feature
Engineering
Training/
HP-Tuning
Model
Monitoring
Model serving
Understanding/
Tuning
Edge
Model
Management
Notebooks
Explainable AI
Hybrid AI
Model
Monitoring
Metadata
Language
Video
AI Accelerators
Models
Datasets
Custom Models
Containers
Python
Endpoints
Vizier Optimization

“ Datasets

Datasets
Vertex AI datasets (fully managed)
• Fully serverless
• Region based
• Free to store
• In tandem with AutoML managed models
Custom Datasets
Cloud Storage, BigQuery or on Internet
Accessing managed dataset from your app:
- JSONL (default)
- CSV or BigQuery stream

“ VertexAI Managed Datasets + Objectives
(AutoML*)
* legacy name, previous generation naming from AI Platform

- Regression/classification
- Forecasting
- Single-label classification
- Multi-label classification
- Text entity extraction
- Text sentiment analysis
- Video action recognition
- Video classifications
for entire video, shots, frames
- Video object tracking
Vertex AI: Managed datasets + objectives
Image Tabular Text Video
- Single-label classification
- Multi-label classification
- Image object detection
- Image segmentation

Vertex AI: Managed dataset + objectives
Image

Tabular

Text

Video

Data labeling + Feature Store
Data labeling (fully managed)
• Create data labeling, annotation tasks
• Use human labelers
• Use Google’s labeler workforce
• Use your own workforce
Feature Store (fully managed)
• Centralized repository for organizing, storing,
and serving ML features
• Organization can eﬃciently share, discover,
re-use features
• Data Model: Entity Type -> Feature
• Ingest data from BigQuery or Cloud Storage
Pro: point-in-time lookup from time series

@martonkodok
Build,
train & deploy
models at
scale
Part #2

2. Train models
Scalably
deploy
model
Evaluate,
monitor,
retrain

VertexAI: Build, train & deploy models at scale
Data Labeling
AutoML
Prediction
Experiments
Data Readiness
Feature
Engineering
Training/
HP-Tuning
Model
Monitoring
Model serving
Understanding/
Tuning
Edge
Model
Management
Notebooks
Explainable AI
Hybrid AI
Model
Monitoring
Metadata
Language
Video
AI Accelerators
Models
Datasets
Custom Models
Containers
Python
Endpoints
Vizier Optimization

1. AutoML out-of-the box training integration
No-code solution. You must target one of the AutoML’s predeﬁned objectives.
2. Custom Training - run your own training applications in the cloud
Train with one of the Google’s pre-builtcontainers or useyourown.
3. Hyperparameter tuning jobs - searchesforbestcombination of hyperparameter values by optimizing
values across a series of trials. Available for custom training. Your training app must adhere to
accepting Vertex AI parameters. You need to report metrics to Vertex AI.
Training
https://cloud.google.com/vertex-ai/docs/training/using-hyperparameter-tuning @martonkodok

Pre-built containers for custom training
https://cloud.google.com/vertex-ai/docs/training/pre-built-containers @martonkodok
Tensorflow
ML Framework version 1.15, 2.1-2.4
use with Cuda 11.x GPU
scikit-learn
ML Framework version 0.23
No GPUs
PyTorch
ML Framework version 1.4 - 1.7
use with Cuda 11.x GPU
XGBoost
ML Framework version 1.1
No GPUs

3. Deploying models
Scalably
deploy
model
Evaluate,
monitor,
retrain

“ You can deploy models on VertexAI
and get a HTTPs Endpointto serve predictions
rapidly and reliably.

1. Deploy a model and get aREST endpointto serve predictions realtime or batched
2. You can use models whetherornotthemodelwastrained on Vertex AI.
3. Specify a prediction traffic split in your endpoint.
4. VPC Private Network option for custom-trained models/tabular models
Vertex AI: Endpoints
Vertex AI Endpoints Backend Prediction
deploy REST

Manage your
models with
confidence
Part #3

VertexAI: Manage your models with conﬁdence
Data Labeling
AutoML
Prediction
Experiments
Data Readiness
Feature
Engineering
Training/
HP-Tuning
Model
Monitoring
Model serving
Understanding/
Tuning
Edge
Model
Management
Notebooks
Explainable AI
Hybrid AI
Model
Monitoring
Metadata
Language
Video
AI Accelerators
Models
Datasets
Custom Models
Containers
Python
Endpoints
Vizier
Optimization

VertexAI: Manage your models with conﬁdence
Data Labeling
AutoML
Prediction
Experiments
Data Readiness
Feature
Engineering
Training/
HP-Tuning
Model
Monitoring
Model serving
Understanding/
Tuning
Edge
Model
Management
Notebooks
Explainable AI
Hybrid AI
Model
Monitoring
Metadata
Language
Video
AI Accelerators
Models
Datasets
Custom Models
Containers
Python
Endpoints
Vizier Optimization

Explainable AI
Explainable AI
• Interpret predictions made by ML models
• Receive a score explaining how much each
factor contributed to the model predictions
• What-If Tool lets you investigate model
behavior at a glance
AI Explanations samples
Github: GoogleCloudPlatform/ai-platform-samples
- Training, deploying, and explaining a tabular data model
- Training, deploying, and explaining an image model
Limitations: doesn’t work well on low-contrast, X-rays, one shade,
panoramas, very tall, very wide images.

Explainable AI: What-if Tool (pair-code.github.io/what-if-tool)
What-if Tool
• Model probing, from within any workﬂow
• test performance in hypothetical situations
• analyze the importance of different data
features
• visualize model behavior across multiple
models and subsets of data
Tutorials, demos onpair-code.github.io/what-if-tool
• Available on many platforms (TensorBoard, Jupyter,
Colaboratory, Vertex AI)
• Supports what-if Analyses (explore counterfactuals, fairness
measures, partial dependence plots)
• Visualizes Model Performances (threshold simulation, up to 2
model comparison, dataset summary statistics)

Part #4
Using Pipelines
throughout your
ML workflow

VertexAI: Pipelines - Orchestrate your model
Data Labeling
AutoML
Prediction
Experiments
Data Readiness
Feature
Engineering
Training/
HP-Tuning
Model
Monitoring
Model serving
Understanding/
Tuning
Edge
Model
Management
Notebooks
Explainable AI
Hybrid AI
Model
Monitoring
Metadata
Language
Video
AI Accelerators
Models
Datasets
Custom Models
Containers
Python
Endpoints
Vizier Optimization

“ Why are MLpipelines useful?
Vertex AI: Pipelines for your MLOps workflows @martonkodok

1. Orchestrate ML workflow steps as a process.
We no longer handle all data gathering, model training, tuning, evaluation, deployment as a monolith.
2. Adopt MLOps for production models. We need a repeatable, verifiable, and automatic process for
making any change to a production model.
3. Develop steps independently -as you scale out, enables you to share your ML workflow with others on
your team, so they can run it, and contribute code. Enablesyoutotracktheinputandoutputfromeach
stepinareproducibleway.
Why are ML pipelines useful?
@martonkodok

“Continuousdelivery and automationpipelines
for machinelearning systems.
What is MLOps?

MLOps level 0: Manual process
MLOps level 1: ML pipeline automation
MLOps level 2: CI/CD pipeline automation
Levelsofautomation deﬁnes maturity of theMLprocess
@martonkodok

MLOps level 0: Manual process - Process for building and deploying ML models is entirely manual.
Infrequent release iterations. No CI, No CD. Disconnection between ML and operations.
MLOps level 1: ML pipeline automation - Continuous training of the model by automating the ML pipeline;
achieve continuous delivery of model prediction service. New pipelines mostly based on new data.
MLOps level 2: CI/CD pipeline automation -iteratively try out new ML algorithms and new modeling where
the experiment steps are orchestrated. The output of this stage is the source code of the ML pipeline steps
that are then pushed to a source repository. Build source. Run test. Output is pipeline.
@martonkodok

MLOps level 2: CI/CDpipelineautomation
@martonkodok

Vertex AI: Pipelines
Vertex AI - Unified ML Platform
Source: Piero Esposito
https://github.com/piEsposito/vertex-ai-tutorials

Pipeline SDK: Condition
Vertex AI: Pipelines for your MLOps workflows
automl_tabular_classification_beans.ipynb
github.com/GoogleCloudPlatform/vertex-ai-samples

1. Use of the Google Cloud Pipeline Components, which support easy access to Vertex AI services
2. Custom Components - function that compiles to a task ‘factory’ function that can be used by pipelines
3. No more Kubeﬂow Pipelines that must be deployed on a Kubernetes Cluster.
4. Sharing component speciﬁcations - the YAML format allows the component to be put under version
control and shared with others, or be used by other pipelines by calling the load_from_url function.
5. Leveraging Pipeline step caching to develop and debug
6. Vertex AI Metadata service + Artifacts Lineage tracking - inverse of pipeline DAG
Developer friendly components
@martonkodok

Part #5
Adapting to
changes of data

Automatic CI / CD Perspective with GCP Services
Eventarc
• Detect changes on data
• React to events from Cloud services
• Handle events on Cloud Workﬂows,
Cloud Functions, Cloud Run
• Reuse pipeline spec.json from GCS
• Trigger Vertex AI pipeline
• Detect changes in codebase
• Build pipeline
• Pipeline spec.json to Cloud Storage
• Image to Cloud Registry
Cloud Build
Cloud Scheduler
• Poll for changes of any data
• Launch based on schedule
• In tandem with Cloud Workﬂows

Conclusion

1. Build with the groundbreaking ML tools that power Google
2. Approachable from the non-ML developer perspective (AutoML, managed models, training)
3. Ease the life of a data scientist/ML (has feature store, managed datasets, endpoints, notebooks)
4. End-to-end integration for data and AI with build pipelines that outperform and solve complex ML tasks
5. GitOps-style continuous delivery with Cloud Build
6. Explainable AI and TensorBoard to visualize and track ML experiments
Vertex AI: Enhanced developer experience

Thank you.
Slides available on:
slideshare.net/martonkodok
Reea.net - Integrated web solutions driven by creativity
to deliver projects.

Vertex AI - Unified ML Platform for the entire AI workflow on Google Cloud

Recomendados

Recomendados

Mais conteúdo relacionado

Mais procurados

Mais procurados (20)

Semelhante a Vertex AI - Unified ML Platform for the entire AI workflow on Google Cloud

Semelhante a Vertex AI - Unified ML Platform for the entire AI workflow on Google Cloud (20)

Mais de Márton Kodok

Mais de Márton Kodok (20)

Último

Último (20)

Vertex AI - Unified ML Platform for the entire AI workflow on Google Cloud