The document discusses supporting Hyper-V 3.0 in CloudStack. It outlines the plugin architecture and progress made in developing an agent that uses WMI to control Hyper-V 3.0 via PowerShell scripts. It emphasizes reusing existing CloudStack code through the agent container and service contracts to maximize efficiency. Development is occurring on Windows using a XenServer surrogate for system VMs, with storage support delayed until a future refactor.

1. Supporting Hyper-V 3.0
Target, Strategy, Progress
donal.lafferty@citix.com
Dec 2nd, 2012

2. “CloudStack has a significant advantage in its
ability to support multiple hypervisor types
within the same zone…
Xen, VMWare, KVM, OVM, but not Hyper-V.”
Hyper-V 2012 (3.0) Support Functional Spec
https://cwiki.apache.org/CLOUDSTACK/hyper-v-2012-30-support.html
2

3. Executive Summary
• Hyper-V Server 2012
ᵒNo NFS client support
ᵒWMI for control
• Plugin architecture
ᵒOptimize Reuse
• Agent Container
• Message Bus
• Service Contracts
• Progress
ᵒDeploy using XenServer surrogate
ᵒDevelop on Windows Understand existing
ᵒConnected Agent, WMI via PowerShell scripts code to maximize
ᵒStorage delayed for Javelin refactor reuse!
3

4. Overview
• Hyper-V 3.0
ᵒWMI
ᵒStorage
ᵒRemote access
• CloudStack Architecture
ᵒPlugins
ᵒAgent Container
• Progress
ᵒVM Startup on Hyper-V
ᵒDev environment
Scope the problem!
ᵒLandmines
4

5. Narrow the Focus
• No Windows Server 2012
ᵒHyper-V Server 2012
• No NFS Client
ᵒGet-WindowsFeature
• No SCVMM 2012 SP1
ᵒService templates
• See http://channel9.msdn.com/Events/TechEd/NorthAmerica/2012/MGT317

6. Storage
• Clients
ᵒLocal
ᵒiSCSI Initiator
• http://blogs.technet.com/b/josebda/archive/2009/02/02/step-by-step-using-the-microsoft-iscsi-software-target-
with-hyper-v-standalone-full-vhd.aspx
ᵒSMB 3.0
• http://blogs.technet.com/b/josebda/archive/2012/08/26/updated-links-on-windows-server-2012-file-server-and-
smb-3-0.aspx
• VHD v VHDX
• http://channel9.msdn.com/Events/TechEd/NewZealand/TechEd-New-Zealand-2012/VIR302 @48 min
• Pushing the bounds of conformance

7. Ultimately Control via WMI
# VM details object
$wmiClassString = "" + $HyperVSvr +
• standardises access "rootvirtualization:Msvm_VirtualSystemGlobalSettingData"
to config and mgmt info $wmiClass = [WMIClass]$wmiClassString
$sets= $wmiClass.CreateInstance()
• rootvirtualization # Set VM properties
• E.g. OpenStack $sets.psbase.Properties.Item("ElementName").value=$VMName
• rootvirtualizationv2 # Create VM using the VirtualSystemManagementService object
• E.g. migration $VSMgmtSvc = gwmi MSVM_VirtualSystemManagementService `
-namespace "rootvirtualization" -computername $HyperVSvr
$VSMgmtSvc.DefineVirtualSystem($settings.psbase.GetText(1))
• PowerShell cmdlets
# Attach the vhdx to the VM and start
Add-VMHardDiskDrive -VMName $VMName -path $NewVHD
Start-vm $VMName

8. Remote Access
• RDP
ᵒConsole GUI
• WBEM (Web-Based Enterprise Management)
ᵒDCOM
• WinRM (Windows Remote Management)
ᵒWS-Management protocol (http://dmtf.org/standards/wsman)
ᵒSOAP
• Remote shell

9. Overview
• Hyper-V 3.0
ᵒWMI
ᵒStorage
ᵒRemote access
• CloudStack Architecture
ᵒPlugins
ᵒAgents
• Progress
ᵒVM Startup on Hyper-V
ᵒDev environment
Do not reinvent!
ᵒLandmines
9

12. Agent Types
CloudStack Manager CloudStack Manager
Hyper-V Plugin Hyper-V Plugin
DirectConnect Agent Agent Manager
WMI Remote Access Message Bus
WMI Connected Agent
Hyper-V 3.0 (e.g. Windows Service)
WMI (or PowerShell)
Hyper-V Server 2012
Hyper-V 3.0
Hyper-V Server 2012

13. Reuse the Service Contracts
public interface HypervisorResource extends ServerResource {
StartAnswer execute(StartCommand cmd);
StopAnswer execute(StopCommand cmd);
RebootAnswer execute(RebootCommand cmd);
}
public interface StoragePoolResource {
PrimaryStorageDownloadAnswer
execute(PrimaryStorageDownloadCommand cmd);
Answer execute(DestroyCommand cmd);
CopyVolumeAnswer execute(CopyVolumeCommand cmd);
CreateAnswer execute(CreateCommand cmd);
}

14. What does the ServerResource do?
Agent as a ServerResource Container
14

15. Agent Container Model
• IAgentShell Server
… …
ᵒBootstrap multiple agents, Resource
O/S interaction, maintainability
• Agent Agent Agent …
ᵒMessaging infrastructure
IAgentShell
• ServerResource
ᵒexecuteRequest
ᵒSupport methods…
O/S Agent Versioning Message Bus
15

16. Message Bus Server
…
Resource
• Java New IO Wrapper Agent …
ᵒMultiplexing
ᵒAsync message handling
ᵒJava-specific Server Server
Handler Handler
• ServerHandler Message
Bus NIO Client …
TCP
16

17. Messaging
ServerResource.
executeRequest
Response
-Answer[]
Request: Agent.Request -Version
-Command[] (out of band) -Sequence
-Version
-Flags
-Sequence
-Flags
AgentRequestHandler
Server Handler
17

18. Overview
• Hyper-V Ecosystem
ᵒHypervisor control
ᵒStorage models
• Plug-in architecture
ᵒPlugins
ᵒAgent Container
• Progress
ᵒVM startup on Hyper-V
ᵒDev environment
„Landmines‟ are
ᵒLandmines
hidden problems
obvious in hindsight!
18

19. Development Deployment
Windows7 AD Domain
(CloudStack Mgmt Server) Controller
Hyper-V Cluster XenServer Cluster
Primary: SMB3.0, iSCSI (Surrogate for
Windows Server 2012
(Hyper-V Role Enabled) System VMs)
Secondary: NFS
Hyper-V Server 2012 Windows Server 2012
(aka Headless HyperV)
XenServer 6.02
Storage Role Enabled

20. Take a tour…
Hyper-V 3.0 VM launch
20

21. Dev Environment
• Management Server on Windows via Cygwin & Maven
• https://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+up+a+CloudStack+dev+envi
ronment+on+Windows
ᵒDifferent Apache repo
• https://github.com/lafferty/cshv3.git
• Landmines
ᵒCygwin v Windows paths
ᵒNo sudo in Cygwin
• mvn errors ; ./utils/src/com/cloud/utils/script/Script.java
ᵒWindows NFS server identity mapping
• http://blogs.technet.com/b/filecab/archive/2012/10/09/nfs-identity-mapping-in-windows-server-
2012.aspx
ᵒOverride the version check.
• ./agent/src/com/cloud/agent/AgentShell.java

22. Launch Codes…
• Management Server
ᵒ./client/tomcatconf/components.xml.in
<adapters key="com.cloud.resource.Discoverer">
<adapter name="HyperV Agent" class="com.cloud.hypervisor.hyperv.HypervServerDiscoverer"/>
…
<adapters key="com.cloud.hypervisor.HypervisorGuru">
<adapter name="HypervGuru" class="com.cloud.hypervisor.guru.HypervGuru"/>
…
ᵒmvn -pl :cloud-client-ui jetty:run
• Agent
ᵒ./agent/conf/agent.properties
resource=com.cloud.agent.resource.HypervResource
host= …
ᵒAdd var & log folders
• E.g. mvn converts /var/run to c:varrun
ᵒmvn exec:java -Dexec.mainClass=com.cloud.agent.AgentShell
• From ./agent
• Reference http://www.vineetmanohar.com/2009/11/3-ways-to-run-java-main-from-maven/

23. Power Shell Calls
• Poor feedback…
protected synchronized void runPowerShell(String psScript) throws Exception{
Runtime runtime = Runtime.getRuntime();
Process proc = runtime.exec(psScript);
proc.getOutputStream().close();
InputStream is = proc.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader reader = new BufferedReader(isr);
String line;
while ((line = reader.readLine()) != null) { s_logger.debug("PowerShell feedback: " + line); }
reader.close();
}

24. Executive Summary
• Hyper-V Server 2012
ᵒNo NFS client support
ᵒWMI for control
• Plugin architecture
ᵒOptimize Reuse
• Agent containers
• Message bus
• Service contracts
• Progress
ᵒDeploy using XenServer surrogate
ᵒDevelop on Windows Understand existing
ᵒConnected Agent, WMI via PowerShell scripts code to maximize
ᵒStorage delayed for Javelin refactor reuse!
24

25. Bonus Track…
Bootstrapping Your Own Hyper-V Cluster
25

26. Host Registration
• No host discovery
ᵒHypervDiscover.find should trigger agent connection
• Agent-side alternative
ᵒConfig in GUI
ᵒLook at database for Hyper-V Cluster‟s zone/pod/cluster
ᵒSet in ./agent/conf/agent.properties
• zone=
• pod=
• cluster=
ᵒDefaults are 1,1,1
ᵒLaunch agent

27. Debugging
• Import projects into Eclipse
ᵒMakes source available!
• Config Maven to launch as debuggable task paused waiting for Eclipse to attach.
export MAVEN_OPTS="-Xdebug -Xnoagent -Djava.compiler=NONE -
Xrunjdwp:transport=dt_socket,address=4000,server=y,suspend=y
- Based on http://www.mojavelinux.com/blog/archives/2007/03/remote_debugging_with_jetty/
• Log4j-cloud.xml.in
ᵒRename to log4j-cloud.xml
ᵒChange Threshold from INFO to TRACE
• :%s/INFO/TRACE/g