This session will deep-dive in the under-the-hood of how Windows Azure Virtual Machines actually work. Hence, it requires audience to have basic knowledge or understanding of cloud computing concept and Windows Azure. We’ll start the discussion with the “kernel” Fabric Controller, Hypervisor, and explore more detail on each aspect of the VM including available OS Version, VM specification, Virtual Hard Drive inside the VM, Runtime installed, and so on.

1. A Deep Look Inside Windows Azure AND ITS Virtual Machine Wely Lau (wely@ncs.com.sg) Microsoft MVP, Windows Azure Solutions Architect, NCS Pte Ltd Blog : http://wely-lau.net

2. Agenda Introduction (10 mins) Windows Azure Service Model (10 mins) Fabric Controller Internal (10 mins) Deploying a Service (15 mins) Service Allocation and Service Healing (10 mins) Inside Windows Azure Virtual Machine (15 mins) Q & A (5 mins)

3. introduction

4. What is a “Cloud”? Cloud: on-demand, scalable, multi-tenant, self-service compute and storage resources

5. Types of Cloud Infrastructure as a Service (IaaS): basic compute and storage resources On-demand servers Amazon EC2, VMWarevCloud, Rackspace Platform as a Service (PaaS): cloud application infrastructure On-demand application-hosting environment E.g. Google AppEngine, Salesforce.com, Windows Azure Software as a Service (SaaS): cloud applications On-demand applications E.g. Office 365, GMail, Microsoft Office Web Companions

6. Cloud: Efficiency Versus Control Windows Azure Efficiency Control + Cost

7. Windows Azure Windows Azure is an OS for the data center Model: Treat the data center as a machine Handles resource management, provisioning, and monitoring Manages application lifecycle Allows developers to concentrate on business logic Provides common building blocks for distributed applications Reliable queuing, simple structured storage, SQL storage Application services like access control and connectivity

9. Worker Role

11. Blobs

13. Traffic ManagerReporting Database Data Sync Access Control Service Bus Caching

14. Sidebar: What’s with all these “Fabrics”? The Windows Azure Fabric Controller is totally, completely, unrelated to AppFabric AppFabric is a brand that encompasses: Windows Server AppFabric: a set of components for building composite applications based on Windows Communication Foundation and Windows Workflow Windows Azure AppFabric: Cloud services for connecting cloud and on-premise applications AppFabricAccess Control Service AppFabric Service Bus AppFabric Cache Built as Windows Azure services

15. Windows Azure Service Model

16. Multi-tier Cloud Applications A cloud application is typically made up of different components Front end: e.g. load-balanced stateless web servers Middle worker tier: e.g. order processing, encoding Backend storage: e.g. SQL tables or files Multiple instances of each for scalability and availability Front-End Middle-Tier Middle-Tier Windows Azure Storage,SQL Azure Middle-Tier Front-End HTTP/HTTPS Load Balancer My Cloud Application

17. The Windows Azure Service Model A Windows Azure application is called a “service” Definition information (Role name, Role type, VM size, etc.) Configuration information (# of instances, # of update domains, etc.) At least one “role” Your codes Roles are like DLLs in the service “process” Collection of code with an entry point that runs in its own virtual machine There are currently three role types: Web Role: IIS7 and ASP.NET in Windows Azure-supplied OS Worker Role: arbitrary code in Windows Azure-supplied OS VM Role: uploaded VHD with customer-supplied OS My Service Role: Middle-Tier Definition Type: Worker VM Size: Large Endpoints: Internal-1 Role: Front-End Definition Type: Web VM Size: Small Endpoints: External-1 Configuration Instances: 3 Update Domains: 2 Fault Domains: 2 Configuration Instances: 2 Update Domains: 2 Fault Domains: 2

18. Service Model Files Service definition is in ServiceDefinition.csdef Service configuration is in ServiceConfiguration.cscfg CSPackprogram Zips service binaries and definition into service package file (service.cscfg)

19. Availability: Update Domains Purpose: Ensure service stays up while updating and Windows Azure OS updates System considers update domains when upgrading a service 1/Update domains = percent of service that will be offline Default is 5 and max is 20, but you can override with upgradeDomainCount service definition property The Windows Azure SLA is based on at least two update domains and two role instances in each role Middle Tier-3 Front-End-1 Middle Tier-2 Middle Tier-1 Front-End-2 Front-End-1 Front-End-2 Middle Tier-1 Middle Tier-2 Middle Tier-3 Update Domain 1 Update Domain 2 Update Domain 3

20. Availability: Fault Domains Purpose: Avoid single points of failures Similar concept to update domains But you don’t control the updates Unit of failure based on data center topology E.g. top-of-rack switch on a rack of machines Windows Azure considers fault domains when allocating service roles 2 fault domains per service Will try and spread roles out across more E.g. don’t put all roles in same rack Front-End-2 Front-End-1 Front-End-2 Front-End-1 Middle Tier-2 Middle Tier-2 Middle Tier-1 Middle Tier-1 Middle Tier-3 Middle Tier-3 Fault Domain 1 Fault Domain 2 Fault Domain 3

21. Fabric controller internals

22. “sketch” of Datacenter Architecture Datacenter Routers Aggregation Routers and Load Balancers Agg Agg Agg Agg LB LB LB LB LB LB LB LB Top of Rack Switches TOR TOR TOR TOR TOR TOR TOR TOR TOR … … … … Racks Nodes Nodes Nodes Nodes Nodes Nodes Nodes Nodes Nodes PDU PDU PDU PDU PDU PDU PDU PDU PDU Power Distribution Units

23. Windows Azure Datacenters

24. Datacenter Clusters Datacenters are divided into “clusters” Approximately 1000 rack-mounted server (we call them “nodes”) Each cluster is managed by a Fabric Controller (FC) FC is responsible for: Blade provisioning Blade management Service deployment and lifecycle Datacenter network FC FC FC Cluster 1 Cluster 2 … Cluster n

25. Inside a Cluster FC is a distributed, stateful application running on nodes (servers) spread across fault domains Top blades are reserved for FC Installed by “Utility Fabric Controller” One FC instance is the primary and all others keep view of world in sync Supports rolling upgrade, and services continue to run even if FC fails entirely TOR TOR TOR TOR TOR AGG LB LB LB LB LB FC3 FC1 FC2 FC4 FC5 FC3 … … … … … … … … … … Nodes Rack

26. The Fabric Controller (FC) X The “kernel” of the cloud operating system Manages datacenter hardware Manages Windows Azure services Four main responsibilities: Datacenter resource allocation Datacenter resource provisioning Service lifecycle management Service health management Inputs: Description of the hardware and network resources it will control Service model and binaries for cloud applications Server Kernel Process Datacenter Fabric Controller Service Word SQL Server Exchange Online SQL Azure Windows Kernel Fabric Controller Server Datacenter (DataCenter.xml)

27. Cluster Resource Description The Fabric Controller is bootstrapped with a Utility Fabric Controller (UFC) Single-instance FC Used for bootstrap and FC updates UFC feeds FC a description of the cluster physical and logical resources in Datacenter.xml Server IP addresses Pool of network IP addresses to assign services Network hardware and Power Distribution Unit addresses

28. Inside a Node Physical Node Guest Partition Guest Partition Guest Partition Role Instance Role Instance Role Instance Trust boundary Guest Agent Guest Agent Guest Agent Host Partition Image Repository (OS VHDs, role ZIP files) FC Host Agent Fabric Controller (Primary) Fabric Controller (Replica) Fabric Controller (Replica) …

29. Fabric Viewer Used by Windows Azure Operation Team to view the fabric inside the datacenter racks clusters

30. Deploying a service

31. Deploying a Service to the Cloud:The 10,000 foot view Service Service package uploaded to portal Windows Azure Portal Service passes service package to “Red Dog Front End” (RDFE) Azure service RDFE converts service package to native “RD” version RDFE sends service to Fabric Controller (FC) based on target region FC stores image in repository and deploys and activates service Portal Service RDFE Service US-North Central Datacenter FC

32. Deploying a Service to the Cloud: a DEEP Look

33. Service Allocation and Service Healing

34. Service Resource Allocation Goal: allocate service components to available resources while satisfying all hard constraints HW requirements: CPU, Memory, Storage, Network Fault domains Secondary goal: Satisfy soft constraints Prefer allocations which will simplify servicing the host OS/hypervisor Optimize network proximity: pack nodes Service allocation produces the goal state for the resources assigned to the service components Node and VM configuration (OS, hosting environment) Images and configuration files to deploy Processes to start Assign and configure network resources such as LB and VIPs

35. Service Allocation Example Role B Count: 2 Update Domains: 2 Size: Medium Role A Count: 3 Update Domains: 3 Size: Large www.mycloudapp.net www.mycloudapp.net Load Balancer 10.100.0.185 10.100.0.36 10.100.0.122 Fault Domain 1 Fault Domain 2 Fault Domain 3

36. Node and Role Health Maintenance FC maintains service availability by monitoring the software and hardware health Based primarily on heartbeats Automatically “heals” affected roles

37. Service Healing Role B Worker Role Count: 2 Update Domains: 2 Size: Medium Role A – V2 VM Role (Front End) Count: 3 Update Domains: 3 Size: Large www.mycloudapp.net www.mycloudapp.net Load Balancer 10.100.0.185 10.100.0.191 10.100.0.36 10.100.0.122 Fault Domain 1 Fault Domain 2 Fault Domain 3

38. Inside windows azure vm

39. Windows Azure VM Sizes Each Windows Azure compute instance represents a virtual server. Although many resources are dedicated to a particular instance, some resources associated to I/O performance (network bandwidth and disk subsystem), are shared among the compute instances on the same physical host. The different instance types will provide different minimum performance from the shared resources depending on their size.

40. Local Drives OS Volume C: = Resource local drive (transient storage for VM) D: = OS drive E: = Application’s code (size of the package) Resource Volume Role Volume Guest Agent Role Host Role Entry Point

41. Runtime installed .NET 3.5 SP1 .NET 4 (RTM) VC80 CRT (8.0.50727) VC90 CRT (9.0.30729) URL Rewrite Module 2.0 VC10 CRT (e.g. MSVCR100.DLL) is not fusion-ized and can be packaged together with the application Others? Java runtime (planned in future) PHP PHP SDK for Windows Azure (“Web Platform Installer”) Else? Start-up Task is your friend 

42. OS version Two OS currently managed by Windows Azure Guest OS 1.x: WS08 64-bit compatible Guest OS 2.x: WS08 R2 64-bit compatible Windows Azure Guest OS Releases and SDK Compatibility Matrix http://msdn.microsoft.com/en-us/library/ee924680.aspx

43. Processes in Windows Azure Vm

44. Processes in Windows Azure Vm

45. Network Bandwidth

46. Conclusion The Cloud enables pay-as-you-go self-service provisioning of application resources Platform as a Service is all about reducing management and operations overhead The Windows Azure Fabric Controller is the foundation for Windows Azure’s PaaS Provisions machines Deploys services Configures hardware for services Monitors service and hardware health The Fabric Controller continues to evolve and improve VM in Windows Azure are provisioned VM that’s optimally configured running on Windows Azure Hypervisor

47. References Inside Windows Azure http://channel9.msdn.com/Events/PDC/PDC10/CS08 Inside Windows Azure Virtual Machines http://channel9.msdn.com/Events/PDC/PDC10/CS63 Inside Windows Azure: The Cloud Operating Systems http://channel9.msdn.com/Events/BUILD/BUILD2011/SAC-853T Inside The Web and Worker Role VMs http://blogs.msdn.com/b/jimoneil/archive/2011/01/03/azure-home-part-14-inside-the-webrole-and-workerrole-vms.aspx Windows Azure Role Architecture http://blogs.msdn.com/b/kwill/archive/2011/05/05/windows-azure-role-architecture.aspx

48. Questions? Wely Lau (wely@ncs.com.sg) Microsoft MVP, Windows Azure Solutions Architect, NCS Pte Ltd Blog : http://wely-lau.net