Recomendados
Recomendados
Mais conteúdo relacionado
Mais procurados
Mais procurados (20)
Semelhante a Netscaler vpx implementation and troubleshooting
Semelhante a Netscaler vpx implementation and troubleshooting (20)
Mais de solarisyougood
Mais de solarisyougood (20)
Último
Último (20)
Netscaler vpx implementation and troubleshooting
- 1. SUM307: NetScaler VPX Implementation and Troubleshooting Download this slide http://ouo.io/3X4ZA
- 2. • Introduction to NetScaler VPX • Installation and Licensing • Troubleshooting • Use Cases Citrix Confidential - Do Not Distribute Agenda
- 3. Introduction to NetScaler VPX
- 4. App delivery without the expensive tin - Maxwell Cooter, TechWorld Citrix Confidential - Do Not Distribute
- 5. • Virtual NetScaler appliance • Hardware requirements • Hypervisors Supported • Differences between VPX and hardware Citrix Confidential - Do Not Distribute Introduction to NetScaler VPX
- 6. • XenServer • XenCenter • XenConvert Citrix Confidential - Do Not Distribute XenServer components
- 7. XenServer Architecture Xen Hypervisor DomUDom0 NS VPXLinux Drivers PV Drivers Client ServerL2 /L3 eth1 eth0 L2/L3 DomU Guest OS Citrix XenServer
- 8. Citrix Confidential - Do Not Distribute VM Mechanism Binary Translation – Sensitive/Privileged CPU Instructions are replaced with hypervisor code or calls “on the fly” • Advantages: Unmodified Guest OS, No special hardware • Disadvantages: Performance Paravirtualization – The Guest OS is modified so that privileged/sensitive instructions are replaced with calls to the hypervisor • Advantages: Performance, no special hardware required, relatively easy for hypervisor • Disadvantages: Guest OS must be modified Hardware assisted – Sensitive/Privileged CPU Instructions executed by the Guest OS trap out to the hypervisor • Each processor vendor brands and implements this differently. • Intel: “VT-x” • AMD: “AMD-V” • Advantages: Unmodified Guest OS, Relatively easy for hypervisor • Disadvantages: Special hardware required
- 9. Hypervisor Architecture - XenServer Xen Hypervisor DomainU DomainUDomain0 XenCenter Client GuestOS2NS VPX Linux Xen daemon(s) Drivers PV Drivers Xen Tools • Bare metal. • Managed by “Domain0” • Domain 0 manages network and storage I/O of guest VMs • Hardware drivers run in Domain0 • Paravirtualized Guest OS or hardware assist VMs only – no Binary Translation Citrix Confidential - Do Not Distribute
- 10. Xen Hypervisor NS VPX: Internals Dom0 (Linux) DomU (NS VPX ) CPU Scheduler Memory PV Front end Drivers Real Drivers Back-end Drivers Virtual CPUVirtual CPU Virtual MemoryVirtual Memory Xen Tools Xen daemon(s) Citrix Confidential - Do Not Distribute
- 11. • Virtual hardware assist • CPUs • Memory • NIC Citrix Confidential - Do Not Distribute Hardware requirements
- 12. Differences between VPX and hardware Features of VPX Features of Hardware Citrix Confidential - Do Not Distribute • Tagged VLANs not supported •No LACP • No hardware assists • No nCore support (yet) •Only version 9.1 & up • Full L2 support • Hardware assists • nCore with MPX models •All versions compatible with hardware
- 16. Setting the VM memory and VCPUs for the NetScaler VPX
- 17. Citrix Confidential - Do Not Distribute Installation
- 18. Citrix Confidential - Do Not Distribute
- 19. Identifying the VPX from the CLI and GUI
- 20. • 1 Mbps • 20 Mbps • 1000 Mbps • Standard • Enterprise • Platinum • http://support.citrix.com/article/ctx122426 Citrix Confidential - Do Not Distribute NetScaler VPX
- 21. • Free license • Unlimited VMs • XenConvert (P2V and V2V) • Centralized multi-server manangement console • Live motion • Virtual infrastructure patch management • Intelligent server maintenance mode Citrix Confidential - Do Not Distribute XenServer licenses
- 22. • Citrix Essentials • High Availability • Advanced server virtualization management • Intelligent workload placement • Rapid provisioning of new VMs Citrix Confidential - Do Not Distribute XenServer licenses (continued)
- 23. Troubleshooting
- 24. • Issues dealing with Networking • Tracing • Log file locations Citrix Confidential - Do Not Distribute TroubleShooting VPX
- 25. • CPU from XenCenter is 100% • VPX shows lower values • Which is right? Citrix Confidential - Do Not Distribute Why is my CPU so high?
- 26. Citrix Confidential - Do Not Distribute XenCenter View
- 27. Citrix Confidential - Do Not Distribute NetScaler VPX view
- 28. Lack of proper license while adding SSL certificates
- 29. NetScaler VPX missing a valid license
- 30. Some examples of problems with the license file(s) The shell command `cat /var/log/license.log` reveals a missing license: The shell command `cat /var/log/license.log` reveals an invalid license: The shell command `cat /var/log/license.log` reveals an expired license:
- 31. Using Tagged VLANs With the NetScaler VPX Citrix Confidential - Do Not Distribute XenServer Host (Dom0) Virtual Machines (DomU) Trunk Port (tagged VLANs) External Switch Virtual Switches Vlan 1 Vlan 53 Vlan 128 NetScaler VPX Untagged VLANs 1/2 1/3 1/4 Virtual Interfaces
- 32. Identifying the NetScaler VPX interfaces
- 33. Reboot messages in the logs Citrix Confidential - Do Not Distribute
- 34. Reboot messages in the logs Citrix Confidential - Do Not Distribute
- 35. Reboot messages in the logs Citrix Confidential - Do Not Distribute
- 36. Use Cases
- 37. • Lab Environment • Proof of Concept • Separation of traffic • Real world simulation Citrix Confidential - Do Not Distribute Use Cases
- 38. Real World Simulation Citrix Confidential - Do Not Distribute Virtual NetScalers Virtual Server Virtual Router North America Europe Asia Africa Hyper Visor Real World
- 39. • NetScaler Licensing: CTX122426 • NetScaler VPX 9.1 FAQ CTX12191 • NetScaler setup and configuring CTX124306 • How to video: importing and configuring CTX122721 • Importing VPX on ESX CTX123683 • Support.citrix.com (search for VPX) • Forums Citrix Confidential - Do Not Distribute Additional Resources
- 40. Before you leave… • Session surveys are available online at www.citrixsummit.com starting Thursday, May 13 • Provide your feedback and pick up your complimentary Starbucks or iTunes giftcard at the registration desk • Download presentations starting Friday, May 21, from your My Schedule Tool located in your My Synergy Microsite event account
- 41. Questions/Comments? Citrix Confidential - Do Not Distribute
Notas do Editor
- Hello, and welcome to NetScaler VPX Implementation and Troubleshooting. My name is Harvey Miller, and I’m a senior escalation Engineering from the Santa Clara office. I’ve been working for Citrix for the past 3 years, and with supporting Network appliances for 5. My colleague, Gregor Visconty is a TRM based in the Redmond Washington office. He has been working for Citrix for the past 2 1/2 years, and has been supporting networks and application delivery for the last 12.
- Have you ever made an seemingly innocuous change that brought down production resources? Or needed to validate that a particular change would work as needed without impacting production? Sometimes simple changes can have unintended consequences, but having enough hardware dedicated to a lab is sometimes not feasible. We’ll talk today about a way to enable you to validate configuration changes without any impact to your production systems. We’ll take a look at several aspects of the NetScaler including what the product is and can do, how to license it, and how to troubleshoot any issues that might arise. We’ll also look at some good use cases for the product, based on some customer requirements, and dealing with some of the issues I’ve touched upon. We feel that customers can really benefit from using the VPX product. One use case that I find exciting was done for a customer that needed to recreate the world for testing. We’ll talk about that in detail later today.
- First, let’s make sure everyone understands what VPX is, so that we’re all on the same page. We’ll go a bit into the architecture of XenServer and NetScaler VPX to give you a better understanding of how they fit together.
- This is an apt description of NetScaler VPX. The product is software only, so you provide your own hardware. As a software-only product, we can deliver this at a fraction the cost of hardware solutions, and still keep most of the functionality of a hardware NetScaler.
- NetScaler VPX is software that runs under XenServer, VMWare, or soon, HyperV that emulates a hardware NetScaler. Naturally some minor compromises have had to be made, but almost all the functionality of a hardware NetScaler works in the virtual appliance. Hardware requirements include Virtual Machine support in the CPU and BIOS, enough memory for all the planned virtual machines, and a large hard drive. As I said before, we currently support our own XenServer as well as VMWare and Microsoft’s HyperV.
- I’d like to take a minute to go over some terminology used for XenServer and its component parts, since I’ll be using these terms in the rest of the presentation. XenServer is the hypervisor, which is the native OS of the hardware. It’s responsible for any IO as well as managing the guest Oses running. XenCenter is the GUI that allows configuration and access to the virtual machines running. And XenConvert will allow the conversion of a running system to a file suitable to import into a XenServer, or from one virtual format into another. For example XenConvert allows the conversion of a VMWare virtual system into a XenServer format. While we’ll be using XenServer for purposes of this presentation, the other hypervisors have similar functionality.
- XenServer software will be the native OS that runs at the bare hardware level. Once XenServer is installed, virtual machines can be defined to run on top of XenServer. In the graphic, Dom0 is the virtual machine that will be in charge of the hardware and controlling the other guest machines.
- There are 3 methods to provide virtual machines. You see the general descriptions here. Binary translation is the brute-force method of watching the instructions that need to be executed, and dealing with them as necessary. Naturally this is very slow. Paravirtualization can be used if the base operating system is aware that it is running as a virtual machine, and has adapted to the environment. So drivers that normally would interact with real hardware understand that they need to invoke the hypervisor’s drivers instead. Finally we have hardware assisted virtual machines. This allows the guest operating system to run unmodified, but have the hardware trap out instructions that need to be virtualized.
- Here we can see the XenCenter client communicates with the Xen daemon running on a Linux VM on Domain0. The Linux VM on Domain0 manages the hypervisor, as well as manages the network and storage I/O of the guest VMs. Based on the resource requirements and the available resource capacity of the XenServer, other virtual machines, including other NetScaler VPX virtual machines, can be running on the same XenServer
- From an architectural point of view, we see NetScaler VPX runs as a paravirtualized vm. Its drivers have been written to send hardware requests to dom0’s backend drivers which will get the IO requests finally to the hardware. This allows us to keep performance as close to native as possible.
- Let’s dive into the hardware requirements for a XenServer in a bit more detail. Both AMD and Intel have their recent processors like the Intel i5 and i7 or AMD’s Opteron line that support virtual assist mode. This is called Intel VT or AMD-V, and more information can be found on the Intel and AMD websites on which processors include this. Note that the CPU must be a 64-bit processor. In addition, the BIOS for the system needs to also support the Virtual assist mode. As a practical matter, most machines manufactured recently should satisfy this requirement. Since there could be multiple virtual machines running simultaneously, a CPU with multiple cores is recommended. For even modest configurations with NetScaler VPX and a few Windows machines, for example, a 4 core system would be needed. For the ability to run more virtual systems simultaneously, you would need additional cores. Each virtual system will have dedicated memory, so plan to have as much memory as needed for all the systems to be run simultaneously. We recommend that each VPX system running have a dedicated core, and the hypervisor itself should have its own core as well. In practice a small lab system could be configured with a 4-core system with 8GB of memory and a 1TB hard drive. The resources dedicated to the NetScaler would typically be 1 CPU, and at least 1GB of memory. The default install of NetScaler VPX generates a 20GB drive. With the 1GB of dedicated memory, packet processing will have a bit under 600MB to use. For environments that need more memory for the NetScaler, this can easily be adjusted, so plan the hardware memory size accordingly. At least a 1 Gbps NIC is required, but we recommend two.
- While the vast majority of the functionality of a hardware NetScaler is supported, there naturally are a few things that are different between the virtual and real implementations of NetScaler. Tagged VLANs and LACP channels are not supported, but can be defined at the hypervisor level. We’ll see what that looks like in the troubleshooting section. Essentially since the interfaces are not real, the layer 2 support has to be done by the hypervisor. So some of the settings for interfaces aren’t allowed. SSL cards to accelerate the encryption and decryption are naturally not available, since the hardware isn’t present. However since the encryption and decryption can also be done in the software, the only impact is that SSL transactions flowing through the unit will be somewhat slower compared to using a hardware NetScaler. NetScaler VPX only supports one CPU currently, so we have the equivalent of the classic builds, but nCore VPX could be available in the future. Each hardware model has a minimum version and build that are required for it to work properly. With the VPX the minimum version is 9.1.
- Now let’s see what it takes to install NetScaler VPX on your hypervisor (using XenServer as the example). We’ll also show the different licensing options for NetScaler VPX and XenServer.
- Here we see the interface from XenCenter to import the NetScaler VPX virtual machine. Once you have the VPX XVA file downloaded from Citrix, installation is a simple matter of importing it…
- And then defining the characteristics you want for the machine. Choose the XEN home server, storage repository, and the interfaces. The imported VM will be available to be started after a few minutes. Then proceed with the configuration steps as you would for a hardware NetScaler. Give it the NSIP, mask, configure the appropriate SNIPs and any resources. Copy the appropriate license file to /nsconfig/license and reboot. The system will then be available to configure using the normal GUI or CLI commands you’re used to.
- This is where we can set the memory and VCPUs for the NetScaler VPX in XenCenter. Note that 1GB is the minimum RAM setting for NetScaler VPX, and although we could add more CPUs, NetScaler VPX won’t use them at this time. It’s a simple matter to increase the RAM if more is needed.
- This is the console view you will see when the NetScaler is up and logged in. You can use this as the equivalent of the hardware console, and comes in handy for command line processes. Naturally you can use puTTY or another SSH client to access the CLI through the network once the NSIP is established, but when the system is first installed the console is the preferred way to initially configure it.
- And the management GUI looks very much like the GUI for other 9.x versions. Note the NetScaler VPX title to distinguish this as the VPX version of the code. At this point you can do whatever configuration you need to using either the GUI or the CLI. Notice that the interface is exactly what you would see from the a hardware system running the same build.
- The CLI of “show hardware” and the GUI equivalent found by selecting the “System” folder shows that this is a NetScaler VPX rather than a hardware appliance. In the case of the GUI, you can also see that “NetScaler VPX” is in the title bar. In the case of the CLI, the prompt is configurable by the user, and in this case it has been set to “NetScaler VPX”
- Standard, Enterprise, and Platinum licenses are available to determine the set of features that are available. A license for one of the family edition licenses is necessary for proper activation of the system. In addition to the features, there are 3 levels of bandwidth depending on your requirements. A license will be the combination of the bandwidth supported along with the set of features. The NetScaler license can be downloaded from the mycitrix site, and will require the MAC address of the first interface. This process is fully explained in KB article CTX122426. You will need to remember that the license will be tied to that MAC, so if the MAC changes you will need to either adjust it, or get a new license for the new MAC address. The lmhost command will give you the information you need to complete the licensing. This is explained quite well in the KB article, however the lmhost command essentially identifies the appropriate interface and pulls its MAC address.
- The basic XenServer license is free, and includes unlimited VMs, using Windows and Linux guests. Physical 2 Virtual and Virtual 2 Virtual conversion is done using XenConvert. I won’t read the list of features, but certainly with the free XenServer license you have a fully functional virtual environment that can be used in a number of useful ways.
- The basic XenServer license is free, and includes unlimited servers, VMs, using Windows and Linux guests. Physical 2 Virtual and Virtual 2 Virtual conversion is done using XenConvert. Beyond the VMWare ESX features Citrix XenServer also includes a centralized multi-server management console, live motion, virtual infrastructure patch management and intelligent server maintenance mode. For a more robust implementation, Citrix Essentials adds HighAvailability, advanced server virtualization management, intelligent workload placement and rapid provisioning of new virtual machines.
- Now we’ll turn to see how to troubleshoot any issues that can arise with a VPX system. Although the basic techniques for troubleshooting are the same in VPX as in a hardware appliance, there are some issues unique to the virtual environment.
- Getting the networking right is probably the trickiest part of defining a virtual machine. The interfaces can be either virtual or real and can be exposed outside the appliance, or strictly within the hypervisor. Note that the license will be tied to one of the MAC addresses, so if the MACs change, or the order changes, the license could become invalid. If that happens you’ll notice that the features that were enabled will no longer be enabled. We’ll see an example of this later. In general troubleshooting an issue on a VPX appliance is the same process as on real hardware NetScalers. If the issue involves the need to examine network packets, a trace can be taken just as on a real NetScaler. All the logs are in the same places, and have the same types of messages. Let’s look at some types of issues that are unique to the VPX environment.
- Let’s take a look at a CPU issue that has come up. When the VPX system is imported into XenServer, the XenServer view shows the CPU as always being 100%, but looking at CPU from the perspective of the NetScaler we see more normal values. So who is right, and why is this happening? First let’s take a quick look at what this looks like in XenCenter.
- Notice that XenCenter thinks the CPU is pegged, and the value never drops from 100%
- However from the NetScaler’s perspective (in this case from the monitoring view), we see that the CPU is a changing, more realistic value. Given that the NetScaler in this case is not really passing much traffic, we’d expect to see that it’s not really running at maximum capacity. What’s happening is that XenCenter is reporting what it thinks the underlying OS CPU is using. Since the NetScaler is using all spare cycles for polling, XenCenter is reporting a false value. This has been corrected in the later VPX builds, but you might see it if you are running an older build, as in this example. In addition to using the monitor view, we could also see the CPU from newnslog, SNMP, or the stat cpu –detail command.
- Here is another case that some have run into. When you attempt to add an SSL certkey greater that 512 bytes, you are unable to, and receive the error “Certificate with key size greater than RSA512 or DSA512 bits not supported”. This can happen for two reasons; you are running the beta release of the NetScaler VPX, or the VPX license is missing or invalid. In the former case, you must upgrade the VPX to a post-beta release. In the latter case, you must install a valid license on the NetScaler VPX. Let’s look a bit further at what is happening to determine why the license might not be correct.
- Here is the CLI output of `show license` and the GUI version of the same. Notice that “Web Logging”, “SSL Offloading”, Dynamic Routing”, “Access Gateway”, “Rewrite”, and “Responder” are all shown as licensed. This doesn’t indicate that a valid license has been loaded, but is actually a symptom of a system without a license. In fact the system can be run without a license, but it’ll show strange symptoms. For example even though Access Gateway is licensed, you’ll find that it won’t really respond to a configured AGEE VIP.
- The shell command `cat /var/log/license.log` can be used to determine the problem with the VPX license. In the first case, we can see that the license file, which should be in the /nsconfig/license directory is missing, so check for the existence of that file. The file name is arbitrary, but the file extension must be “.lic”, so make sure that the file exists and that the extension is correct. In the second case, we see that the license file was generated with the wrong hostid, so it is invalid. In this case, a new license file must be created and installed. This could have happened if the MAC address of the first interface has changed, as we noted earlier. In the last case, we can see that the license file has expired, so check the system clock and the expiration date of the license file. Remember that whenever a license file is modified or added, the VPX must be rebooted for the changes to take effect.
- One issue that can be a bit confusing in the VPX environment is how to deal with tagged VLANs. VPX doesn’t support tagged VLANs, so you’ll have to work around that by using the hypervisor’s VLAN support. This diagram shows how an environment that needs to have tagged VLANs could be set up. Note that the NetScaler can have untagged VLANs, but the tagged VLANs would need to be set up between the external network and the hypervisor. The box in dark blue represents the VPX virtual machine. In this example, we have 3 interfaces, all defined on different VLANs, which are untagged to the virtual switches in the light blue box. The untagged VLANs are carried through to the hypervisor’s interface, where it will tag the packets going to the external switch.
- The top window is the Network tab of the VPX window in XenCenter. The bottom window is the NetScaler VPX CLI output of `show interfaces`. Match the “Device” number in the VPX Network configuration with the interface device number shown at the end of the first line of each interface in the output of the CLI command `show interfaces`. This will help you ensure that you have bound the correct VLANs and IPs to the correct virtual interfaces in the VPX, which will map to the correct VLANs and physical interface(s) on the XenServer host.
- In the case of a hardware appliance or the VPX, when the NetScaler is rebooted through the shell or NSCLI, once the NetScaler has completed the reboot, you can see details about the reboot command that was issued. This can be seen in the file “/var/log/ns.log” in the case of a reboot issued from the NSCLI, and in the file “/var/log/messages” in the case of a reboot issued from the shell.
- However, if you reboot the VPX through the XenCenter console, there will not be reboot messages in the VPX logs, but will be in the Xen logs.
- In this case, the reboot can be seen in the file “/var/log/xensource.log” on the XenServer, or through the “logs” tab of the virtual machine through XenCenter.
- What sort of cases can the virtual NetScaler be used for? Let’s examine some possibilities.
- The NetScaler VPX is a natural for a lab environment. You can take your production configuration with minimal changes and build a robust place to test configuration changes, upgrades, or examine how things interact. Along those same lines, a quick proof-of-concept can be built to configure a new feature or web site and determine the optimal settings in an environment that won’t impact the production use of your servers. With the low cost of a XenServer/NetScaler VPX system, customers can use these appliances to separate special types of traffic. For example if you have many VIPs on one NetScaler, but one or more has special characteristics, a new NetScaler VPX system could be deployed just for that traffic. That would allow more flexibility to tune the configuration to the specific needs of the traffic. So global settings could be different as necessary.
- I’d like to talk about a situation we had recently with a customer who had very specific requirements for the GSLB site returned for their customers throughout the world. They had GSLB sites scattered around the world, so it was important that a user would reliably be directed to the site closest to them. Additionally they required that when sites or resources from a particular site were unavailable, the user would be directed to the next closest site. However since the production environment couldn’t be impacted, and testing different scenarios would be impossible, we turned to XenServer and NetScaler VPX to encapsulate the entire world. We set up XenServer with 4 GSLB VPX appliances, and 1 Linux-based router. The production configurations were used with minimal changes and put on the various VPX systems. All the VPX systems used the Linux router as their default gateway to complete the picture. The router was configured to route all the traffic in this virtual world to the appropriate interfaces. Scripts were defined to simulate down circuits or systems to evaluate what impact that had on the load balancing decisions from around the world. DIG command simulates DNS requests from various locations, and was used in the scripts to validate the GSLB DNS decisions that were made in the normal case as well as when specific parts of the environment were unavailable. Notice that all the traffic we’ve talked about is contained within the XenServer, so there was no possibility of any testing to impact their production. Talk about a virtual world!
- Here I’ve highlighted a few of the important Knowledge Base articles that are available from support.citrix.com. In addition to these, there is a wealth of information in the product documentation that comes up on a quick search of the site. There are forums specifically devoted to NetScaler VPX. While some of the threads are dealing with beta code, there is still a lot of good information in the forums. Questions can be quickly posted to get other opinions in the community and perhaps a quick answer. The first article goes into detail to walk you through getting your NetScaler license from mycitrite, including retail, partner and evaluation licenses (90 days) as well as the free NetScaler VPX Express license for standard edition, 1Mbps bandwidth. The FAQ answers many of the questions that have come up over the past few months. It deals with licensing, XenServer and VMWare ESX, and talks about what can and can’t be done. This is a good article to read through to see if your plans are reasonable or not. CTX124306 is a video showing the installation of XenCenter followed by importing a VPX image, and then doing the initial configuration of the NSIP, SNIP, and one VIP. This is the first video of a 3 part series. Finally CTX123683 shows the correct way to import a VPX image into ESX.
- That concludes my prepared comments. Please feel free to ask questions now, or find me in the hall or booth afterwards.