Hyper-V Failover Clustering, Beyond the Z Drive using the disk GUID

Are you running Hyper-V?  Have you used up all the available drive letters deploying LUN’s for VM’s in your Failover Cluster?  Running out of drive letters? It is fairly easy to get around the 26 drive letter limit in windows when deploying VM’s in a Hyper-V failover cluster configuration.  It is possible to deploy 2000 LUN’s to your Hyper-V cluster environment allowing you to grow your cluster and take advantage of your hardware.

In order to grow beyond the 26 drive letters you will need to install HotFix KB951308.  Once this hot fix is installed you can create shared storage using the drives GUID rather than drive letter or mount point.  Using the drives GUID you can create up to 2000 shared volumes without worrying about running out of identifyers.

What you need to do:

  • Install hotfix KB951308 on all hosts in the Hyper-V Failover Cluster
  • Reboot your Hyper-V hosts
  • Present storage to the server
  • Bring the new storage on-line
  • Initialize the new storage
  • partition and Format the storage, but don’t assign a drive letter or mount point

Using he command mountvol.exe from the command line you can view the GUID for all storage devices attached to your system.  NOTE: You need to run this command form the server that currently owns the storage resource that you want to locate the GUID for.  Alternatively you can locate the GUID by right clicking the storage device in Failover Cluster Manager and selecting properties.

hyperv_drive_guid

Once you have the GUID for the volume you would like to deploy your new Virtual Machine to do the following:

  • Launch the new virtual machine wizard using Hyper-V manager
  • On the “Specify Name and Location” step check the box next to “Store Virtual Machine in a different location” and paste the volume GUID in the location box.
  • On the “Connect Virtual Hard Disk” step choose “Create a Virtual Hard Disk” and in “Location” box paste the GUID for the volume.
  • Complete the creation of your Virtual Machine

Once your Virtual Machine is created you need to launch “Failover Cluster Manager” and create a new “Virtual Machine” Service.  When you create the new Virtual Machine service the disk will automatically be added as a resource for the Virtual Machine”

You can also acheive this using SCVMM 2008.  Make sure to deploy the new VM on the server that currently owns the disk resource.

Automated Deployment of ESX Hosts Part II

This is part II of a multi-part blog. If you haven’t read Part I you can read it here.

The homework from part Iof this blog was to determine what your standard configuration is.  If your having trouble deciding what this should be hopefully today’s session will help with that.  Below you will see all the information that will make up my “standard” build and this will be the information that I use to develop my script.

Sid’s Standard Build

The following information directly relates to the kickstart portion of the script.

Regional Settings:

  • Keyboard layout: US
  • language: US
  • Time Zone UTC America/New_York

Service Console Configuration items:

Partitions:

  • /boot =  100Mb
  • swap = 1600Mb
  • / = 10240Mb
  • /var = 8192Mb
  • /tmp = 4096Mb
  • /opt = 5120Mb
  • /home = 5120Mb
  • vmkcore = 100Mb
  • /vmfs = fill to max

License Server information:

vcenter.sidtest.com (Licenses server is installed on my vCenter Server.

The remaining information is directly related to the post installation portion of the script

Networking:

8 Network Interfaces

vSwitch0 (vmnic0, vmnic2) – Portgroups (Service Console(Vlan 300) & vMotion(vmkernel)(Vlan301))

vswitch1 (vmnic7, vmnic5) – VM Production Portgroups (vlans 2, 15, 150,151,152)

vswitch2 (vmnic4, vmnic6) – DMZ Portgroups (vlans 200, 201, 203)

vSwitch3 (vmnic1, vmnic3) – Backup Networks (vlans 400 & 401)

The two illustrations below show visual diagrams of my network configurations.

Diagram 1

net_design1

Diagram 2

net_design2

NOTE:

When developing your standard build it is always a great idea to manually build one server and map out the network interfaces to vmnics.  It is also very important to use the same hardware and PCI placement for all servers in a cluster.

I have also decided to reject Forged Transmits and MAC address changes on all the vSwitch’s in my environment.

The following is host specific network information for this particular host:

Hostname: vmware1.sidtest.local
Service Console IP Address: 192.168.10.100
Service Console Subnet: 255.255.255.0
Service Console Gateway: 192.168.10.1
VMkernerl IP address: 192.168.12.100
VMkernel Subnet Mask: 255.255.255.0
VMkernel Gateway: 192.168.12.1
DNS Server 1: 192.168.5.20
DNS Server 2: 192.168.5.21
NTP Server 1: 192.168.5. 30
NTP Server 2: 192.168.5.31
NTP Server 3: 192.168.5.32
AD Server 1: 192.168.5.20
AD Server 2: 192.168.5.21

Remaining Build Items

In my build I will be building Active Directory (krb5) user authentication into the service console.  Users will not be allowed to login directly as root.  They will be forced to login using their active directory username and password.  I will be granting the following users login privileges.

ssmith
jbower
cobrian
talmeda

I will also be assigning a grub password to prevent someone from booting up using init 1 and gaining service console access without entering a password.
I will be preventing users from logging into the physical console as root.
I will be setting the following password policy for local user accounts:

  • Maximum password age of 90 days
  • Minimum of 1 day between password changes
  • 14 Day password warning before change required

I will be setting the following MOTD (Message of the Day) and Welcome.js (For web logins) message:

Warning!!! This computer system is private and may be accessed only
by authorized users.  Data and programs in this system are confidential
and proprietary to the system owner and may not be accessed without
authorization. Unauthorized users or users who exceed their authorized
level of access are subject to disciplinary action, up to and including
termination and are subject to prosecution under state or federal law.
Activity on this computer system is logged.

I will be running the following agents in the Service Console:

  • EMC Naviagent
  • HP System Insight Management Agent

I will be allocating 800Mb of Memory to the Service Console.

This completes my ESX Build information.  This is not an all inclusive list of options that can be set or configured as part of an automated installation.  It is possible to configure many other options but for the purpose of this example the preceding standard build will suffice.

Be sure to check back for Part III of this series.  In Part III we will be building our script to support the build that was laid out in this post.  Then later in Part IV we will then go on to demonstrate the options for deploying your script to your ESX Hosts.

VMware Tools status shows as not running after running VMware Consolidated Backup

Yesterday, VMware posted a Knowledgebase Article about VMware Tools appearing to go off line after a VCB snapshot is taken. This issue occurs after applying the ESX350-200901401-SG hotfix. The KB Article also says it can occur “for some time after the initial snapshot” on unpatched hosts, but I have never seen it.

The work-arounds are simple:

  1. restart the mgmt-vmware service (Make sure you don’t have VMs set to auto start)
  2. Log In and log out of the VM, this will cause VMware Tools to kick itself in the pants.
  3. Use the name or UUID lookup method instead of the ipaddr method.

This was originally addressed here.

It is difficult sometimes to get your backup software to use the name lookup method if it does not use a VCB Integration Kit. If you ARE using an integration kit, you can set this in the config.js file:

VM_LOOKUP_METHOD=”name”;

Since most of the “majors” in backup software, like NetBackup, TSM and Networker were originally Unix based, you may be able to find a config file somewhere to set this option.

Using VCB with any backup software

I don’t know how many times I have helped with VCB issues in the past couple of years on the VMTN Forums. It usually falls down to someone not understanding the VM Backup Guide or how VCB works. Honestly, the guide leaves something to be desired. Because of this, I have published a “Proven Practice” guide on VI:OPS to try to clarify things.

<commercial>

VI:OPS is a VMware Forum that dedicates itself to providing information related to operations surrounding a VMware Infrastructure. The “Proven Practice” documents are submitted and reviewed by moderators before they are published. The published documents allow for peers to comment on the documents.

</commercial>

Although VMware provides integration kits for a few different brands of backup software, it does not cover all of the different brands versions. Some vendors have created their own integration kits as well. But not every brand or version is covered. Because of this, I have outlined a generic method for using VCB to back up and recover VMs. Some other things that really needed to be clarified were using VCB in hot-add mode and performing FullVM backups of selected disks. The doc includes screen shots and command sytax examples.  So head over and check out the doc -> Proven Practice: Setting Up VMware Consolidated Backup for any Backup Software.

Automated Deployment of ESX Hosts Part I

A very important part of deploying your Virtual Infrastructure environment is consistency when it comes to the configuration of the ESX hosts.   Time and time again as common practice I see users deploying ESX hosts manually with no documentation to follow other than what they have in their heads.  Deploying ESX hosts in this fashion can lead to inconsistent configurations between hosts.  Choosing manual installations over scripted installations can lead to all sorts of problems some of which can be very difficult to diagnose.

There is a very common misconception that scripted installations are difficult or not worth the time and effort to create.  This couldn’t be farther from the truth.  Utilizing scripted installations can save countless hours of time spent manually installing ESX and troubleshooting problems from inconsistent manual installs.  This is part I of a multi-part blog in which I am going to walk you through the different options that are available with scripted installations.  I am also going to share with you some sample scripts and methods to streamline the testing of scripts you develop.  Before we begin to develop a script we need to know what our “standard” configuration is going to be.  Without having a standard configuration we don’t know what we need to configure.  It’s important to develop a standard configuration for your environment.  If you have multiple clusters in your environment your standard configuration may have minor changes from cluster to cluster but should ultimately be consistent across the environment to make troubleshooting and maintenance much simpler.

In part II of this blog post I will post my “standard” configuration that will be used to build my sample automated kickstart installation script.  My standard configuration will include the following:

  • Service Console Partitioning scheme
  • Service Console memory size
  • Active Directory Servers for Service Console PAM integration
  • AD Users to have access to the Service Console
  • Service Console Security Policy
  • Agents to be install in the Service Console and their configurations
  • ESX Hosts Networking Configuration including routing, DNS, NTP, etc…
  • Licensing Server Information

Once I have a “standard” configuration I will perform a walk through using it to develop a script that will be used for the Automated Deployment.  Once the script is developed then we will cover the options for deploying a host using the script.  These options include:

  • PXE Boot using a tool like the UDA appliance
  • Install from CD using a hosted kickstart file.
  • Network installation using http, ftp, or nfs
  • Custom ESX installation CD’s.

Your homework assignment before “Automated deployment of ESX hosts par II” is to determine what your “standard” ESX build is and have the information handy to start building your script. So don’t forget to check back for my next post where we start to build our kickstart files.

Getting New Performance Overview Charts Working in VC2.5U4

One of the New features of vCenter 2.5 Update 4 is the “Performance Overview Charts”. But, if you do not follow the proper upgrade steps, it will not work properly. I am a big fan of the “Complete uninstall and install fresh” method. You can make sure all of the bits are gone before the upgrade. Obviously, you will need to make a backup copy of your license files and, if changes were made, the vpxd.cfg file.If you look at the release notes, you will see a link to a VMware Knowledgebase Article about how to get the charts working. It tells you that you will need to install the Java Development Kit6u11. You will also need to set path info into the environment, copy the files from the CD to a local disk and run install from there.

But there are those of us that will just download the zip file, uncompress it, and run setup.exe. No backups, no uninstall first. This is why VMware has several KB articles about how to get the charts working, depending on how you upgraded.

The first KB Article deals with not stopping the Web services first. Another KB Article deals with the need for an updated Oracle ODBC driver if you have an Oracle database.  Finally, for those of us living on the edge, there is a KB Article if you are suing the bundled SQL Express database.

So, here are my steps in a nutshell:

  1. Backup the databases, license file and (if needed) vpxd.cfg
  2. Uninstall the old version of vCenter, VUM, Converter, and Capacity Planner
  3. Install the Java Development Kit6u11
  4. Edit the environment path and append C:Program FilesJavajdk1.6.0_11bin
  5. Add a system variable of JAVA_HOME to point to C:Program FilesJavajdk1.6.0_11
  6. Install vCenter, pointing to the vCenter and VUM databases when prompted
  7. Copy the vpxperfCharts directory from the CD to local disk
  8. Run install.bat from vpxperfCharts on the local disk
  9. Uninstall Capacity Planner (You don’t need it by now, do you???)

Obviously, if you have your Program Files on a different drive or path than the default, it will need to be entered appropriately in steps 4 and 5.

Well, there you have it. Setup Performance Overview Charts in 9 easy steps.