Dear VMware: Pick a Common (SUPPORTED) Virtual Appliance OS…Please….

One of my pet peeves is that each virtual appliance coming out of VMware is that each different virtual appliance released by them is based on a different OS. Some of these do not even have documented methods for updating the OS. We all know that no matter what OS is running on a system, there will be updates for stability and security. Almost every time I begin an engagement with a customer and it involves using a virtual appliance, their security wonks get all pissy with me and I need to show that I have the latest security patches installed before I even connect the appliance to their network.

This all started with the HealthCheck Appliance, which is a tool available to partners. Its running Ubuntu 7.10 Server JEOS. Great! It is an unsupported, deprecated OS. If you know anything about Ubuntu, you know that the “Long Term Support” (LTS) versions are released every other year. So, the latest LTS version is 8.04 and the previous is 6.06. No big deal, right?

Now to further complicate things, the VMware Data Recovery appliance and the vSphere Management Assistant run completely different OS versions. The VDR runs CentOS with a kernel version 2.6.18-92-el5. The vMA runs RedHat Enterprise Linux with a kernel version 2.6.18-128.1.1.el5.

Updating the vHA

The documentation that comes with the vHA explains how to update the OS using apt-get, and it explains it in such a way that anyone can do it. BUT…Ubuntu 7.10 has been deprecated and the repositories were recently removed. Running apt-get update results in a bunch of http 404 errors because the repositories are no longer where the OS thinks they belong. Now what?

I did a quick search on google and found a blog post on NewAdventuresInSoftware about a work around, so thanks goes out to Dan Dyer for providing this solution. Its pretty straight forward and I added how to install VMware Tools to it. The 7.10 repositories haven’t been completely removed, they were just moved to a different url: http://old-releases.ubuntu.com/releases. The apt-get utility uses the file “/etc/apt/sources.list” to determine where to go for patches and software packages. In order to upgrade the OS to 8.04, you need to install the update-manager-core first and then upgrade the OS. So, you need to point apt-get to the new url to install the update-manager-core package and all the dependancies. But, before upgrading the OS, you need to point back to the original repositories, because that is where the 8.04 packages reside. Here is a step-by-step list of how to get this done:

1. Make a backup copy of the /etc/apt/sources.list file:

sudo cp /etc/apt/sources.list.710

Edit the original file:

sudo vi /etc/apt/sources.list

Comment out any reference to a CDROM using #

Use the global search and replace command in vi to change the references:

:%s/us.archive/old-releases/g

Save and quit:

:wq

Install the update-manager-core package and all dependancies:

sudo apt-get install update-manager-core

Copy the original sources.list file back because it gets changed during the upgrade:

sudo cp /etc/apt/sources.list /etc/apt/sources.list.updt
sudo cp /etc/apt/sources.list.710 /etc/apt/sources.list

Edit the original sources.list:

sudo vi /etc/apt/sources.list

Comment out any reference to a CDROM using #

Save and quit:

:wq

Run the OS upgrade routine:

sudo do-release-upgrade

Install the packages required for VMware Tools to be compiled:

sudo apt-get install linux-headers-server build-essential

Install VMware Tools as listed in the instructions (Replace the ?.?.?-?????? with the proper tools version)

mount /cdrom
cp /cdrom/VMwareTools-?.?.?-??????.i386.tar.gz /tmp
cd /tmp
tar -zxvf VMwareTools-?.?.?-??????.i386.tar.gz

cd vmware-tools-distrib

sudo ./vmware-install.pl

You should be prompted to automatically run vmware-config-tools.pl

Once the installation is completed, install the vmxnet drivers:

sudo /etc/init.d/networking stop
sudo rmmod pcnet32
sudo rmmod vmxnet

sudo modprobe vmxnet
sudo /etc/init.d/networking start

Verify your IP address:

ifconfig

Updating the vMA

The vMA has a well documented process for updates, using sudo vima-update scan and sudo vima-update update for updating the OS. I am assuming that eventually, patches will become available from VMware, but there is nothing right now. The vima-update utility can also be configured to look at a different repostory for patches. That is documented in the Admin Guide and I won’t get into it here. The is nothing about updating VMware Tools, but a simple VMware Tools process for RPM based distros will work. Just copy the VMwareTools…rpm from the tools cd image and run rpm -i VMwareTools…rpm. Substitute the proper file version.

Updating VDR

HA! Nothing is documented for VDR about updates. Nothing. Not even a mention. It is running an older, unpatches kernel and an old version of VMware Tools. I found a post on the communities about how to update the OS using yum update and VMware Tools. Basically, vmware-config-tools.pl is hard-coded to to older versions of libssl and libcrypto, so symlinks need to be added to install VMware Tools properly:

cd /lib
ln -s libssl.so.0.9.8e libssl.so.0.9.8
ln -s libcrypto.so.0.9.8e libcrypto.so.0.9.8

DISCLAIMER:

Since some of these methods are NOT documented by VMware, they may not be supported. Sometimes, you have to weigh security concerns against ultimate support ooptions.

June 17, 2009

Storage Protocol Differences and FCoE Diagrams

Just thought I would share these diagrams that I used in a recent training session. I used them to explain the differences in the storage protocols that may be used for a vStorage Cloud and how FCoE works. Click on the images for a larger view.

The first image shows the differences between the common storage protocols and what it takes for the data to get from point A to point B.

This diagram demonstrates the FCoE packet. The top block is an Ethernet Packet and the bottom block is the FCoE data.

This diagram shows the data flow within a Converged Network Adapter (CNA).

This diagram is the Converged Enhanced Ethernet Bridge. CEE in one end, FC out the other.

June 1, 2009

VMware Capacity Planner 2.7 – Lipstick on a Pig

Last week’s upgrade / outage of the VMware Capacity Planner Dashboard was a COMPLETE surprise to me. I was trying to access data on Friday with no success. Why? Because I just don’t pay attention to the notices on the front page of the dashboard. Low and behold, the Capacity Planner Dashboard is now available! It has been upgraded to version 2.7 with perty colors and everything!

Actually, I use “Lipstick on a Pig” lovingly. Capacity Planner is huge – thus the “Pig” part. It collects and analyzes monster amounts of data rather well. I use it frequently. I am currently involved in an assessment of about 1300 systems. I have learned from experience to “scale” the Capacity Planner “Data Collectors” by using multiple collector machines, limiting to about 200-250 systems per collector. All of the inventory and performance data gets collected efficiently and is uploaded to the mother ship in multiple, but smaller, chunks. All of the heavily lifting is done at the mother ship, so you don’t need a beefy machine for the data collection. Creating reports and Optimization Scenarios (Formerly known as Consolidation Scenarios) in the Dashboard is fairly straight-forward and the reports generate in about 10-15 minutes with larger assessments. Far better than some of the competing products that I have used.

The new version brings some nice new features as well. It makes it easier to perform desktop virtualization assessments and it looks like they are gearing up to provide application virtualization assessments as well. They have also tweaked users, groups, access and permissions as well. Although it works fine on my Linux desktop running Firefox 3, sadly, VMware only officially supports Internet Exploser 5.5 and above.

So what is the difference between CP and the “competing” products? Why are people still paying for something that they can get for free from VMware or a VAC partner? The first is access to the data. You need a login to access the CP Dashboard. Other products are run locally. I say “So what?!”You can get your VMware guru to collect your data and then generate optimization scenarios and reports for you. They will give you some nice stuff with plenty of information. All you have to do is ask.

The other thing at issue with CP is the ability to generate graphs and charts for the corner office people. The CP Dashboard has a few graphs mixed in, but there are many other things you may want to put into a graph. In order to do this with CP, you need to dump the data into a spreadsheet and generate graphs and charts with the spreadsheet software. This can sometimes be a daunting task to some.

One of the few useful graphs in CP — One of the Few Useful Graphs in the CP Dashboard

Just an aside: As you can see from the screenshot above, even with a ton of servers, the vast majority of systems only show 10% or less processor utilization. This is typical for an assessment.

The final reason why you might NOT want to use Capacity Planner is that the Optimization Scenarios are locked in to VMware ESX or VMware Server. You cannot run a scenario against XEN, KVM or *GASP* Hyper-V…. But that doesn’t matter because you really WANT to use VMware anyway. So, what are you waiting for? Go forth and virtualize!

Below are the release notes:

VMware Capacity Planner Release Notes

Current Version	2.7	Build	32117
Last Updated	5/20/09

VMware Capacity Planner Version 2.7 is an upgrade from Version 2.6.x. The purpose of this upgrade is to release new features.

What’s New in Capacity Planner 2.7

Capacity Planner 2.7 has a new look and feel. Many of the menu options have changed, and reports have been enhanced. The following items are the main changes in this version.

Desktop Virtualization. VDI assessments enable you to virtualize destops utilizing software profiles and base images.
Software Profiles. Software profiles replace application profiles and can now be edited by Partners. Software Profiles allow tags to describe the software. Software Profiles can represent applications and operating systems. They keep track of individual process utilization as well as system-wide use. More computing resource utilization dimensions are shown for each profile.
Base Image Creation. System Software Cluster analysis is used to build a few images that maximize software usage.
VM Template Sizing. You can create VM Templates, based on various base images, during an Optimization Scenario Analysis.
Reporting. Optimization reports now includes new reports. These reports are formally known as the Consolidation Estimator Reports. The new report is a complete assessment report. The controls for the output are located in the Assessment Global Settings. The link to get to the Global Settings is at the bottom of the Optimization Report Page. This report is the only place that contains the following information at this time: VM-to-VM Template mapping, VM Template Sizes, and Base Image Report. For the Custom Report, the display limit is set to 10,000. If the amount of data exceeds this limit, the data that exceeds the limit is not displayed.
Scenario. The scenario now includes the ability to select by system attributes. It also has a Base Image selection page. Selecting Base Images is required to include the Base Image, VM Template Size, and VM-to-VM Template mapping sections in the Assessment Report.
User Groups. You can now create a user group to give users access to a company, template, report, or scenario.
Access and Permissions. The security model that has been used by company roles is now extended to templates. This allows individual access to templates by a single user or a group of users. Partners and VMware can create templates that are meant only for a certain group of users. This will remove the need to create multiple companies to manage users and templates.
Date Range Selection. Users can now select a range of dates to be used for the assessment.
Alerts and Anomalies. The behavior of alerts and anomalies has changed in this release.
User Self-provisioning. A Partner Company (only partner) can adjust the security settings in their company to allow users with the same email suffix that is supplied in the company information to request and automatically approve a login account. The Partner will need to create a suffix to enable this feature adjust the Security Policy to allow self-provisioning.
Collector SSH Port setting. The collector now allows the user to change the SSH port to something other than 22. This is a global setting and will not allow per system port settings for now.
Collector/Dashboard Inventory Additions. The Collector and the Dashboard now collect desktop inventory and show Video Card, PnP Devices, Pagefile, and Printers for the purpose of doing desktop assessments.
Create new CE users. You can now create a user within a CE assessment.
Multiple Assessments. More than one assessment per company is now supported.
Sudo support. Sudo support has been added in this release.

Redesigned Interface

This release introduces a new look for the Dashboard. Many of the menus have changed. Online help is now available from the Help menu. The Online Library containing the Installation Guide, Getting Started with Capacity Planner, the Troubleshooting Guide, and the Reference Guide is available from the Portal. In addition, the Installation Guide and Getting Started with Capacity Planner are available as PDF files in the Portal.

The major changes for the Dashboard include:

Style changes
- Style. The background and logo have changed.
- Labels, Titles, Menus. The menu structure and labeling have changed. For example, Consolidation has changed to Optimization. The Roles label has changed to Access and Permissions.
- Forms and Wizards. Several have been improved: New Assessment, New User, Access and Permissions
Feature changes
- Notifications. Notification creation is now simpler.
- Architecture. An analysis engine and a reporting engine have been added.
- Software Profiles. Application Profiles is replaced with an improved Software Profile and Report feature.
- Online Help is a new addition available from the Help link. Other online documentation is available from the Portal link.
- Reports. Reports have been enhanced. A storage report has been promoted to first class from the custom reports. It is located under Performance and does not include all the columns of the custom report.
- Application Analysis. Application Analysis allows you to analyze application usage and create Base Images.

All of the documentation is provided in HTML format. We would like to know what you think. Please take a moment to do our survey: