35. 名称変更
• Azure Sentinel → Microsoft Sentinel
• Azure Security Center → Microsoft Defender for Cloud
• Azure Defender → Microsoft Defender for Cloud
「どの環境にあるワークロードもAzureのセキュリティの仕組み
を使って保護する」という点には変化なし
42. Azure Arc-enabled servers
Azure Key Vault Integration
System / Security
Administrator
Azure Arc-enabled server
Azure Key Vault (AKV)
User Certificates
Public Key Infrastructure (PKI)
Hybrid Compute
Resource Provider
Extension Service
Azure AD
AKV is configured with Azure Managed Identity for allowing the
Azure Arc-enabled server to access certificates
AKV extension is deployed on the
Azure Arc-enabled server
AKV Extension
Background Service
Certificates URIs are used as
parameters and syncs based on
user-defined time intervals
AKV Extension background service request for
a Managed Identity token in order to retrieve certificates
AKV Extension background service
retrieve AKV certificates based on interval
specified in the extension configuration
Cert Store
The certificates and private keys are stored in the local
certificate store (Windows) or as files in a directory (Linux)
App/Service
(i.e. Web Server)
The AKV agent will then sync down the
new certificate and private key
automatically at its next sync interval
Upon renewal time, the certificates are
renewed only in AKV
(renewed PKI certs can be uploaded as well)
2a
Admin deploy Azure Key Vault
1a
1b
2b
3a
3b
6
5
4
App/Service consumes local
cert store certificate
(as well rebind upon
renewal)
44. Azure Arc-enabled servers
Connectivity Options
Azure VNET
Azure Arc-enabled Server
Azure Arc Service
Public Endpoint Private Endpoint
AzureArcInfrastructure
Service tag
Private Link
Azure Express Route &
Site-to-Site VPN
Internet
Proxy
1. Public endpoint via direct connection
2. Public endpoint via proxy server
3. Private endpoint over Express Route
AAD
ARM
45. Azure Arc-enabled servers - Private Link integration
On-Premises &
Multicloud servers
Azure Virtual Network
Azure Monitor
Private Link Scope
Azure Log Analytics
Workspace
Azure Log Analytics
On-Premises /
Multicloud Gateway
(Proxy)
Azure Monitor
Private Link Endpoint
Azure Automation
Private Link Endpoint
On-Premises / Multicloud Proxy
Azure Arc
Private Endpoint
Azure Arc
Private Link Scope
Azure Arc-enabled servers
Azure Arc metadata
Guest config
Extension downloads
Azure Resource Manager
Azure Active Directory
On-Premises / Multicloud Firewall
ARM
Private Endpoint
Service Tags
Azure ExpressRoute / S2S VPN
On-Premises / Multicloud network
Connectivity via Azure Log Analytics Gateway
Direct connectivity via internet
Azure Automation
accountt
47. Azure Arc-enabled servers
Connected Machine Agent
Azure Arc Connected Server (On-Premises, AWS EC2, etc.)
Azure Arc Connected Machine Agent
Hybrid Instance Metadata Service (HIMDS)
Handles managed identity and metadata sync (heartbeats)
Guest Configuration
Provides In-Guest Policy and Guest Configuration functionality, such
as assessing whether the machine complies with required policies
Extension Manager
Manages VM extensions, including install, uninstall, and upgrade
MMA/AMA
ASC
Configuration passed to the Agent:
• Subscription and resource group
• Azure Region to store metadata
• Network options (direct, proxy, or private link)
• Credential to onboard (device login, AAD token, or SPN)
Azure AD
HTTPS/443
HTTPS/443
Azure Resource Manager (ARM)
Hybrid Compute
Resource Provider
Log Analytics
Azure Portal
Az CLI
Azure SDK
REST API
Azure Admin
Authentication &
Authorization
Guest Configuration
Resource Provider
HTTPS/443
HTTPS/443
Custom Script
48. Azure Arc-enabled servers architecture
Linux OS
On-premises/other clouds
Azure AD
Hybrid Identity Service
Azure Resource
Manager
(ARM)
Log Analytics Workspace
Hybrid Instance
Metadata Service
Azure Automation Azure Monitoring
Azure Security
Center
Azure Sentinel
HTTPS/443
Hybrid Compute
Resource Provider
Extension Manager
Log Analytics Agent
Custom Script
DSC
Microsoft Dependency Agent
/opt/GC_Ext/downloads
/var/lib/waagent/<extension>
Guest Configuration
/var/lib/GuestConfig
Azure Arc Connected Machine Agent
/var/opt/azcmagent/
/var/opt/azcmagent/tokens
Guest Configuration
Resource Provider
System
Administrator
49. Azure Arc-enabled servers architecture
Windows OS
On-premises/other clouds
Azure AD
Hybrid Identity Service
Azure Resource
Manager
(ARM)
Log Analytics Workspace
Hybrid Instance
Metadata Service
System
Administrator
Azure Automation Azure Monitoring
Azure Security
Center
Azure Sentinel
Hybrid Compute
Resource Provider
Extension Manager
Log Analytics Agent
Custom Script
DSC
Microsoft Dependency Agent
%SystemDrive%¥AzureConnectedMachineAgent¥ExtensionService¥downloads
%SystemDrive%¥Packages¥Plugins¥<extension>>
Guest Configuration
%SystemDrive%¥Program Files¥ArcConnectedMachineAgent¥ExtensionService¥GC
%ProgramData%¥GuestConfig
Azure Arc Connected Machine Agent
%ProgramFiles%¥AzureConnectedMachineAgent
%ProgramData%¥AzureConnectedMachineAgent
%ProgramData%¥AzureConnectedMachineAgent¥Tokens
%ProgramData%¥AzureConnectedMachineAgent¥Config
Guest Configuration
Resource Provider
HTTPS/443
50. Azure Arc-enabled servers – Identity and Access
Management
Connected Machine Agent
Azure Resource Manager (ARM)
Host Instance Metadata
Service (HIMDS) managed
identity credentials
Guest Configuration
Extension Service
Guest Configuration
Azure Arc Service
Hybrid Compute
Resource Provider
Guest Configuration
Resource Provider
Log Analytics Workspace
Guest configuration updates
are managed using Azure
token obtained with
managed identity credentials
Guest configuration
Arc service is created
and runs under Local
System account
Guest configuration extension
service is created and runs
under Local System account
HIMDS service is created and runs
under NT Service¥himds virtual
account with least privileges
Log data is ingested into
Log Analytics workspace
using Log Analytics
workspace credentials
Log Analytics Agent
Custom Script
Dependency Agent
Extensions are created using
Local System account
privileges of extension service
Hybrid agent extension applications
local security group is created to
authorize members to request Azure
tokens through HIMDS service
Extensions are managed using
Azure token obtained with
managed identity credentials
Azure Arc-enabled server resource is
created using service principal credentials
Service Principal Managed Identity Azure AD User
Azure Active Directory
Managed Identity is created in Azure AD enterprise applications
Azure Arc-connected server
(On-Premises and/or Multicloud)
Azure Admin
Create service principal and a short-lived client secret with Azure admin
Grant “Azure Connected Machine Onboarding”
role for least privileged access
Grant “Azure Connected Machine Resource Administrator”
role to server admin for least privileged access
Server Admin
Login to Azure portal and generate server onboarding script
Using RDP/SSH or automation tooling
login to server with local admin rights
Run server onboarding script by providing
service principal and client secret
Managed Identity credentials are
stored in local storage using ACL
1a
1b
1c
2a
2b
2c
4
7
5
3a
3b
3c
3e
3d
3f
3g
6
52. vSphere Infrastructure
VMware Guest VMs
vCenter Server
Deploy Azure Migrate appliance for
agentless VMware migrations and
connect to vCenter
Admin
Admin Input Azure Arc
onboarding details using the
Appliance Configuration Manager
The Azure Migrate appliance
initiates remote sessions to
each discovered guest VM
Onboarding script is
executed inside the guest
VM to be onboarded as
Azure Arc-enabled server
Azure Arc-enabled servers
onboarded and available to
manage via the Azure portal
1
2
4
3 5
Azure Migrate + Azure Arc
How it works
64. AKS on Azure Stack HCI Architecture
Azure Stack HCI Cluster Hyper-V
vSwitch
Physical HCI Nodes
AKS-HCI Workload Cluster-01 (Azure Arc enabled Kubernetes)
Kubernetes Workload Nodes & Applications
Hyper-V VM-01 Hyper-V VM-02 Hyper-V VM-n
User Applications
Kubernetes Control Plane
Hyper-V VM-03
System Services
Hyper-V VM-02
Hyper-V VM-01
AKS-HCI Workload Cluster-n (Azure Arc enabled Kubernetes)
Kubernetes Workload Nodes & Applications
Hyper-V VM-01 Hyper-V VM-02 Hyper-V VM-n
User Applications
Kubernetes Control Plane
Hyper-V VM-03
System Services
Hyper-V VM-02
Hyper-V VM-01
AKS-HCI Management Cluster
System Services
Hyper-V VM-01
65. AKS on Azure Stack HCI using Azure VM (Nested Virtualization)
Azure
Resource
Group
Azure Kubernetes Service (AKS) on Azure Stack HCI
Platform Services
Kubernetes Management
Cluster
Kubernetes Workload Cluster
Kubernetes Control Plane
Hyper-V VM
System Services & Containers
Hyper-V VM
System Services & Containers
Kubernetes Worker Nodes
Hyper-V VM
System Services & Containers
Hyper-V VM
System Services & Containers
Hyper-V VM
System Services & Containers
Hyper-V VM
System Services & Containers
Azure VM – AKSHCI Hyper-V Host
Windows Server 2019 Datacenter with Hyper-V enabled (Nested Virtualization)
Hyper-V
vSwitch
Azure Virtual Network
67. Azure Arc-enabled Kubernetes
Onboarding
Azure AD
On-Premises/Cloud Provider Kubernetes Cluster
API Server
etcd
Save state/
Onboarding private key
in k8s datastore
3b
Cluster
Metadata
Operator
Fetch cluster metadata and update custom resource
6a
Resource
Sync
Operator
Cluster
Identity
Operator
Save the Azure Identity Certificate
5c
Push cluster metadata
(uses Managed Identity to authenticate eastus.dp.kubernetesconfiguration.com)
6b
Azure Arc-enabled Kubernetes
Data Plane Service
Hybrid Identity Service
Watch for updates in cluster
metadata custom resource
6c
Fetch connectedCluster Managed Identity certificate
(uses onboarding private key to authenticate eus.his.azure.com)
5a
Microsoft.Kubernetes
Resource Provider (RP)
Update cluster metadata
6d
Send identity metadata
4
Managed
Identity
Service
Fetch the identity certificate
5b
Create Service Principal in AAD
3f
Azure Resource
Manager
(ARM)
PUT connectedCluster resource
along with Managed Identity metadata
3g
Create Managed Identity
3e
Azure
CLI
Uses Helm to deploy Arc-enabled
k8s agents with onboarding
private key
K8s Cluster Admin
3a
az connectedk8s connect
1
PUT resource
Microsoft.Kubernetes/connectedClusters with
public key (management.azure.com)
Microsoft Container
Registry
3d
Pull agent images
3c
Fetch Helm chart
2
68.
69. Azure Arc-enabled Kubernetes GitOps Flow
Arc Connected
Kubernetes Cluster
GitOps
Configurations
git
Repository
Flux Operator +
Helm Operator
Application
Changes
git
merge
Flux
pickup changes
Application V1
(Desired State)
Google Kubernetes
Engine (GKE)
Elastic Kubernetes
Service (EKS)
Rancher K3s
Azure Kubernetes
Service on HCI
1 2 3
4
Application
Deployment
5
6
7
Application V2
(New Desired State)
Application
Rolling Update 8
Any Kubernetes,
any Infrastructure
70. Azure Arc-enabled Kubernetes
GitOps Configuration
On-Premises/Cloud Provider Kubernetes Cluster
Helm Release
obj-x
obj-y
obj-z
helmreleases CR
release-a
gitconfigs CR
config-a
Flux Operator
Flux-Helm
Operator
controller-
manager
Watch gitconfig CRs
Flux-logs
agent
Flux events sent to upstream service
Create or update
Flux Operator or
Flux-Helm Operator
1
7
8
11
Config Agent
Collect Status from Flux
12
Create gitconfigs CR
6
ns
Namespace
Git Repository
YAML
Files
Helm
Releases
CRs
Flux-Helm Operator watches helmreleases CRs,
pulls Helm chart and creates Helm release
10
Flux watches Git repo,
creates k8s resources
based on raw YAML
and helmreleases CRs
9
Azure Arc-enabled
Kubernetes Dataplane
Service
GET Pending sourceControlConfiguration resources
(uses Managed Identity to authenticate)
5
POST status for the Flux agents
to be retrieved with resource GET
13
Microsoft.KubernetesConfiguration
Resource Provider (RP)
Azure Resource
Manager
(ARM)
Store sourceControlConfiguration resource
4
PUT sourceControlConfiguration resource
3
PUT resource
Microsoft.Kubernetes/connectedClusters/clusterName/providers/Microsoft.KubernetesConfiguration/sourceControlConfigurations/configName
(uses ARM Extension Resource pattern)
az k8s-configuration create
2
Azure
CLI
Admin
71. Azure Arc-enabled Kubernetes Cluster
Azure AD Entity
(User Account/Service Principal)
API Server Guard
api
TokenAccessReview,
SubjectAccessReview
allowed/denied
kubectl get pods
If allowed, return list of pods
allowed/denied
checkAccess
Owner
Role assignment in Azure
Azure Arc-enabled Kubernetes
AAD RBAC (public preview)
72. Azure Arc-enabled Kubernetes
Cluster Connect (public preview)
Kubernetes Cluster
Microsoft.Kubernetes
Resource Provider (RP)
listClusterUserCredentials
Client-side proxy
Dataplane Service
Hybrid Connections
Azure Resource Manager (ARM)
Azure AD Entity
(User Account/Service Principal)
az connectedk8s proxy Cluster
connect-
agent
API
Server
Customer
Firewall
heartbeat
Customer Location
(On-Premises/Cloud Provider)
kube-aad-
proxy
73. Azure Arc-enabled Kubernetes
Cluster extensions (public preview)
On-Premises/Cloud Provider Kubernetes Cluster
obj-x
obj-y
obj-z
extensionconfigs CR
extension
Helm Release
extension-manager
Watch extensionconfig CRs
1
8
Config Agent
Collect Status
11
Create extensionconfig CR
7
ns
Namespace
Azure Arc-enabled Kubernetes
Data Plane Service
GET Pending extension resources
(uses Managed Identity to authenticate)
5
POST extension status
12
Microsoft.KubernetesConfiguration
Resource Provider (RP)
Azure Resource
Manager
(ARM)
Store extension resource
4
PUT extension resource
3
PUT resource
Microsoft.Kubernetes/connectedClusters/clusterName/providers/Microsoft.KubernetesConfiguration/extensions/extensionName
(uses ARM Extension Resource pattern)
az k8s-extension create
2
Azure
CLI
Admin
Azure Container Registry or
Microsoft Container Registry
GET version
6
9
Fetch Helm chart
stored as OCI artifact
10
Install helm chart