vSphere Install and Upgrade Best Practices KB Articles and Links

So, I use NewsGator to aggregate a BAZILLION feeds from several sources, blogs, like this one, actual news feeds and a bunch of VMware feeds. The VMware feeds are from the VI:OPS and VMTN forums. The VMTN forums allow you to create a custom feed by selecting the RSS link at the bottom right of each page or you can get a feed from a specific section of the forum by clicking the link on the bottom left of a list. On of the custom feed options is to get a feed of the new KB articles.

VMware has released quite a lot of new KB articles surrounding vSphere. They just released nice best practice guidelines for installing or upgrading to ESX 4 and vCenter 4. They are short and to the point. There is also a nice article covering best practices for upgrading an ESX 3.x virtual machine to ESX 4.0. One thing I noticed, but never thought about is this :

“Note: If you are using dynamic DNS, some Windows versions require ipconfig/reregister to be run.”

Eric Seibert over at vSphere-Land posted a nice set of “missing links” for everything vSphere. This is a nice, comprehensive set of links to evetrything you need for vSphere upgrades or installs.So, go check that out as well.

VMware vSphere Upgrade Path Overview

Many of you are wonder how you will go about upgrading to VMware vSphere when it is release. Well I’m here to say don’t worry. The upgrade path from ESX 2.x & 3.x is very painless and fairly simple. A lot of you will remember all the phone you have had in the past performing upgrades and scripting installs, well VMware is quickly trying to make all of that a thing from the past with new features available in vSphere.

Read More here

Just some more vSphere information

Here is some information about vSphere that I thought would be good to share with the world. As with everything else this is just a drop in the bucket. I’m currently working on putting together some upgrade videos and screenshots so take a look back and hopefully I will have them done by the end of the week.

Here is some interesting information about vSpehere and what it supports keep in mind these are just some notes I jotted down:

ESX 4 Hosts (vSphere Host)
256VM’s per host
64 Cores per host
512GB Ram per host

vSphere VMs (Hardware version 7)
8 vCPUs
256GB Ram
VMDirectPath I/O
Hot Plug Support (Supports CPU’s and Memory)
ESX 2.x and 3.x VM support
Paravirtual SCSI adapter
MSCS 2008
Persistent Reservations in vmkernel
LSI Logic SAS (Virtual SAS controller)

Networking Improvements
New iSCSI stack with 10-30% improved performance
TCPIP 2 Support (Based on FreeBSD 6.1 / IPv6 / locking and threading capabilties)
VMXNet3
MSI/MSI-X
Receive Side Scaling
VLAN offloading
VMware Directpath I/O

Storage Improvements
SCSI-3 Compliant
VMFS still SCSI-2
Target PortGroup Support (TPGS)
Asymmetric Logical Unit (ALUA)
Pluggable Storage Architecture (PSA)
Updated iSCSI stack
Native SATA

Service Console
64-bit, 2.6 based Linux kernel compatible with RHEL 5.2
Supports for both 32bit and 64bit applications
root file system stored in VMDK
vmkernel runs and owns device drivers only 64-bit
Address Space layout Randomization (ASLR)
No Linux dev packagers and libraries

CPU
Enhanced Intel step down
Enhanced AMD Power Now

Security
Trusted Platform Module (TPM)
Digitally signed and validated modules
Memory integrity techniques with microprocessor capabilities to protect against buffer-overflow

Guided Consolidation
500 Simultaneous Physical Machines
Modular Plug-in can be installed on different machine

Coverter

Physical / Virtual / 3rd party
Server 2008 Support
Covert Hyper-V Machines to VM’s

Update Manager

ESX / ESXi and Virtual Appliance Upgrades
Upgrade Virtual Hardware
VMware Tools
Base Line Groups

Upgrade vcenter steps

No SQL 2000 Support
2.x & 3.x Upgade Path
Upgrade vCenter
Upgrade Update Manager
Use Upgrade Manager to Upgrade Hosts
Upgrade VMware Tools, then the VMware Hardware.

vSphere Host Update Utility
3.x to 4.x
Dosn’t Upgrade VMFS Datastores or VMs
Installs with vSphere client
Support Rollback for ESX only
Can be used to install patch releases to standalone hosts
Copies Script and ISO to ESX hosts reboots and installs

VMware vSphere 4 under the covers – First Look

Tuesday April 21st VMware announced they will be releasing vSphere 4 by the end of 2nd quarter. This is exciting news for many looking to take advantage of some of the new features available with this release. In this post I’m going to walk through a handful of some of these new features. There are over 100 new features in vSphere 4 and this post doesn’t come close to covering them all but I will be touching on some really exciting ones with more to come in my next few posts.

Let’s start with the new home screen. It’s a handy way to navigate all the configuration areas of vSphere.

vsphere_home_screen1

Next let’s take a look at the new “Hardware Status” screen. In this screen shot there is limit hardware information which is due to my hardware. On actual server grade hardware you can view just about anything you want regarding your hardware. If you remember from my other vSphere post you can also trigger alarms based on most of these sensors as well.

vsphere_hardware_status1

Next lets take a look at the changes made to the Host Summary screen. Notice the arrow pointing to the Datastore with the alert. One of the alarms we can set is based on storage usage so we no longer have to manually verify that the free storage is within the acceptable levels.
vsphere_host_info

With ESX 3.5 we had to manually make sure we didn’t overuse resources to maintain and N+1 configuration for HA. I wrote an article on VMware HA and how to size your environment to maintain N+1. Well when you determine that you need to have 37.5% overhead available on each server you can now specify that in your HA configuration rather than manually making sure you don’t exceed.

vsphere_cluster_overview
Take a look at the arrow in the above screen shot it’s showing the reserved capacity of the hosts to ensure the proper failover capacity. The screen shot below show the HA setting to configure this functionality.

vsphere_ha_screen

There are some exciting and new interesting features surrounding networking. Once new feature is the distributed switch. The distributed switch is a really exciting improvement as it comes with support for private vlans, network vMotion, and of course support for 3rd party switches such as the Cisco Nexus v1000 switch.

vsphere_distributed_switch

There are many other enhancements to the networking such as the new VMXNET Generation 3 driver and other features like IP pools similar to whats available in Lab manager.

vsphere_ip_pools

vsphere_ip_pools_2

vsphere_vm_options

In my last post on vSphere I showed configuration items available as part of host profiles. Below is a screen shot showing the hosts compliance similar to that of Update manager.

vsphere_profile_compliance1

One exciting change to the virtual machines is the ability to hot add memory and vCPU’s to the virtual machines. This gives even more flexibility to make changes to servers without having to schedule downtime.

vsphere_mem_cpu_hotplug
The new resource view is very handy. Giving you a snapshot of what is really going on with your vm. It gives you insight into how much memory is private to the vm and how much is shared memory with other vm’s in the environment. This allows you to see how much memory is being “de-duplicated” It also allows you to see the host overhead, memory being swapped, and if ballooning is taking place.

vsphere_vm_resource_allocation

VMware finally moved aware form the annoying license server and now has integrated licensing into vCenter itself. This should make alot of users very happy to not have to deal with managing those license files any longer.

vsphere_license_server

Update manager now has built in support for a shared repository. So if you have a large deployment you can easily manage your update repositories across multiple Update Manager servers.

vsphere_update_manager

Have you had those annoying issues where some of your services would crash o stop running that vCenter is dependent on? Well it’s easy to kep track of your vCenter service status now with the new vCenter Service Status information.

vsphere_service_status

One thing that I think is still lacking is the scheduled tasks. That have added a few new options that can be scheduled, but i would have expected some additional improvements in this area.

vsphere_schedule_tasks

I hope you enjoyed this preview and be sure to check back as I will be covering some additional new features including “Fault Tolerance”. Over the next few weeks and month I will be putting together more overview posts, best practice articles as well as video tutorials.

VMware vSphere 4 (ESX 4.0, vCenter 4.0) Alarms and Host Profiles

Some are speculating that next Tuesday VMware is going to announce the release of VMware vSphere which is what essentially is Virtual Infrastructure 4.0 which would include ESX 4.0. I can’t say what VMware is going to do but over the next few weeks I will be publishing information on vSphere as well as some instructional videos. For now I have some teasers for you.

Here is a screen shot of the alarms available in vSphere. A you can see they have expanded the alarm feature from what was available in VI3.

vsphere_alarms

I’m sure most of you have heard of the new host profiles. If you haven’t had the fortune of checking out this cool new feature here are some screenshots to show you what options are available to you as part of a host profile. If you are not much for scripting and just can’t stand those pesky automated build scripts then you will love this feature. It gives you the ability to configure just about every aspect of the ESX host without having to deal with any scripting.

vsphere_host_profiles_1

vsphere_host_profiles_2

vsphere_host_profiles_3

vsphere_host_profiles_4

vsphere_host_profiles_5

vsphere_host_profiles_6

vsphere_host_profiles_7

As you can see in this screenshot all these settings are very easy to set via the GUI.

vsphere_host_profiles_8

So stay tuned as there is much more to come. I’m currently working on making videos covering installing, and configuring vSphere from the ground up and plan on getting into all of the new feature available in this release.

VMware Virtual Center – Physical or Virtual?

Over the years there have been some controversy over this topic. Should Virtual Center (vCenter) be physical or virtual? There is the argument that it should be physical to ensure consistent management of the virtual environment. Of course there is also the fact that Virtual Center requires a good amount of resources to handle the logging and performance information.

I’m a big proponent for virtualizing Virtual Center. With the hardware available today there is no reason not to. Even in large environments that really tax the Virtual Center server you can just throw more resources at it.

Many companies are virtualizing mission critical application to leverage VMware HA to protect these applications. How is Virtual Center any different. So what do you do if Virtual Center crashes? How do you find and restart Virtual Center when DRS is enabled on the cluster?

You ave a few options here.

  1. Override the DRS setting for the Virtual Center vm and set it to manual. Now you will always know where your virtual center server is if you need to resolve issues with it.
  2. Utilize Powershell to track the location of your virtual machines. I wrote an article that included a simple script to do this which I will include on our downloads page for easy access.
  3. Run an isolated 2 node ESX cluster for infrastructure machines.

So my last option warrants a little explaining. Why would you want to run a dedicated 2 node cluster just for infrastructure vms? The real question is why wouldn’t you? Think about it. Virtual Center is a small part of the equation. VC and your ESX hosts depend on DNS, NTP, AD, and other services. What happens if you loose DNS? You loose your ability to manage your ESX hosts through VC if you follow best practice and add them by FQDN. Now if AD goes down you have much larger issues, but if your AD domain controllers are virtual and you somehow loose them both that’s a problem. It’s a problem that could affect your ability to access Virtual Center. So why not build an isolated two node cluster that houses your infrastructure servers. You’ll always know where they will be, you can use affinity rules to keep servers that back each other up on separate hosts, and you can always have a cold spare available for Virtual Center.

Obviously this is not a good option for small environments, but if you have 10,30, 40, 80, 100 ESX hosts and upwards of a few hundred VM’s I believe this is not only a great design solution but a much needed one. If you are managing this many ESX hosts it’s important to know for sure where your essential infrastructure virtual machines reside that affect a large part of your environment if not all.

VMware HA Cluster Sizing Considerations

To properly size a HA fail over cluster there are a few things that need to be determined.  You need to know how many hosts are going to be in your cluster, how many hosts you want to be able to fail (N+?), and it helps to know resource utilization information about your vm’s to gauge fluctuation.  Once we know this information we can use a simple formula to determine the maximum utilization for each host to maintain the appropriate DRS fail over level.

Here is an example:

Let’s say we have 5 hosts in a DRS cluster and we want to be able to fail (1) hosts (N+1).  We also want to have 10% overhead on each server to account for resource fluctuation.  First we need take 10% off the top of all (5) servers which leaves up with 90% utilizable resources on all hosts.  Next we need to account for the loss of (1) hosts.  In the event that a host is loss we need to distribute its load across the remaining (4) host.  To do this we need to divide up one hosts 90% possible resources by (4) remaining hosts.  This tells us that we need to distribute 22.5% of the servers load to each of the remaining hosts.

Taking in to account the original 10% over head plus the 22.5% capacity needed for fail over we need to have 32.5% of each hosts resources available which means we can only utilize 67.5% of each host in the cluster to maintain an N+1 fail over cluster with 10% overhead for resource fluctuation.  The formula for this would be:

((100 – %Overhead)*#host_failures)/(num_hosts – #host_failures)+%overhead = overhead needed per ESX host

Example 1:

((100-10)*1)/(5-1)+10 = 32.5    
(5 Server cluster with 10% overhead allowing 1 host failure) 67.5& of each host usable

((100-20)*2)/(8 -2)+20 =46.6   
(8 Server cluster with 20% overhead allowing for 2 host failures) 53.4% of each host usable

Example 2:

Fail over of 1 host

((100-20)*1)/(8 -1)+20 =31.4   
(8 Server cluster with 20% overhead allowing for 1 host failures) 68.6% of each host usable

Fail over of 2 hosts

((100-20)*2)/(8 -2)+20 =46.6   
(8 Server cluster with 20% overhead allowing for 2 host failures) 53.4% of each host usable

Determining the %Overhead can be tricky without a good capacity assessment so be careful if you don’t allocate enough overhead and you have host failures performance can degrade and you could experience contention within the environment.  I know some of the numbers seem dramatic but redundancy comes with a cost no matter what form of redundancy it may be.

Service Console Memory for ESX 3.x

I’ve been asked this question a lot lately, “How much memory should we assign to the service console?”  My default answer is always 800Mb.  I have a number of reasons why I would recommend this, but the short answer is “Why not?”  What do you really have to loose by assigning the service console 800Mb?  The answer is nothing, but you do have a lot to gain.  Even if you are not running any agents there are benefits to this.  One thing most people dont’ realize is even if you don’t install any third party agents at the service console you are still running agents.  There is the vpxa agent that allows you to communicate with vCenter, and there is the HA agent if you are running VMware HA, and if you have more than 4Gb of memory installed VMware recommends increasing the RAM to 512Mb.

Considering all this and that most systems today have 16Gb of memory or a lot more I just don’t understand why anyone would leave the service console at 272mb.  When deploying a new server always create a swap partition of 1600Mb which is double the maximum amount of service console memory.  This will at least allow you to increase the service console memory later without having to re-deploy your host.   Having an easy option when you call tech support with a problem and they tell you to increase the memory to 800Mb is always a great idea.  I’ve seen a large number of users having HA issues that have called tech support and the first thing they are told is to increase the SC memory to 800Mb.  So before you deploy your next ESX server take the Service Console memory into consideration, and at least create the 1600Mb SWAP partition so you can easily bump the memory up to the max of 800Mb.

Getting New Performance Overview Charts Working in VC2.5U4

One of the New features of vCenter 2.5 Update 4 is the “Performance Overview Charts”. But, if you do not follow the proper upgrade steps, it will not work properly. I am a big fan of the “Complete uninstall and install fresh” method. You can make sure all of the bits are gone before the upgrade. Obviously, you will need to make a backup copy of your license files and, if changes were made, the vpxd.cfg file.If you look at the release notes, you will see a link to a VMware Knowledgebase Article about how to get the charts working. It tells you that you will need to install the Java Development Kit6u11. You will also need to set path info into the environment, copy the files from the CD to a local disk and run install from there.

But there are those of us that will just download the zip file, uncompress it, and run setup.exe. No backups, no uninstall first. This is why VMware has several KB articles about how to get the charts working, depending on how you upgraded.

The first KB Article deals with not stopping the Web services first. Another KB Article deals with the need for an updated Oracle ODBC driver if you have an Oracle database.  Finally, for those of us living on the edge, there is a KB Article if you are suing the bundled SQL Express database.

So, here are my steps in a nutshell:

  1. Backup the databases, license file and (if needed) vpxd.cfg
  2. Uninstall the old version of vCenter, VUM, Converter, and Capacity Planner
  3. Install the Java Development Kit6u11
  4. Edit the environment path and append C:Program FilesJavajdk1.6.0_11bin
  5. Add a system variable of JAVA_HOME to point to C:Program FilesJavajdk1.6.0_11
  6. Install vCenter, pointing to the vCenter and VUM databases when prompted
  7. Copy the vpxperfCharts directory from the CD to local disk
  8. Run install.bat from vpxperfCharts on the local disk
  9. Uninstall Capacity Planner (You don’t need it by now, do you???)

Obviously, if you have your Program Files on a different drive or path than the default, it will need to be entered appropriately in steps 4 and 5.

Well, there you have it. Setup Performance Overview Charts in 9 easy steps.