VMware vSphere 4 (ESX 4.0, vCenter 4.0) Alarms and Host Profiles

Some are speculating that next Tuesday VMware is going to announce the release of VMware vSphere which is what essentially is Virtual Infrastructure 4.0 which would include ESX 4.0. I can’t say what VMware is going to do but over the next few weeks I will be publishing information on vSphere as well as some instructional videos. For now I have some teasers for you.

Here is a screen shot of the alarms available in vSphere. A you can see they have expanded the alarm feature from what was available in VI3.

vsphere_alarms

I’m sure most of you have heard of the new host profiles. If you haven’t had the fortune of checking out this cool new feature here are some screenshots to show you what options are available to you as part of a host profile. If you are not much for scripting and just can’t stand those pesky automated build scripts then you will love this feature. It gives you the ability to configure just about every aspect of the ESX host without having to deal with any scripting.

vsphere_host_profiles_1

vsphere_host_profiles_2

vsphere_host_profiles_3

vsphere_host_profiles_4

vsphere_host_profiles_5

vsphere_host_profiles_6

vsphere_host_profiles_7

As you can see in this screenshot all these settings are very easy to set via the GUI.

vsphere_host_profiles_8

So stay tuned as there is much more to come. I’m currently working on making videos covering installing, and configuring vSphere from the ground up and plan on getting into all of the new feature available in this release.

ESX automated deployment email completion notification

How would you like to kick off your ESX installation, then go have some coffee, go for a jog, or just hang out by the water cooler until it is finished without worrying if you’re wasting time while it’s waiting done and waiting for you. Well you can with this ESX email script. Incorporating this script as part of your ESX automated deployment script allows you to configure your server to email you once the post installation configuration is finished.

So what do you need to do? It simple you can get the mail_notify script that I found on yellow-bricks.com from our downloads page. Once you have the script you will need to get it on to your server along with the MIME Lite.pm file that you can download here. Once you download and extract the package you can find the Lite.pm file under /lib/MIME/ folder.

The take the Lite.pm file and the mail_notify.pl file and tar them together for easy retrieval. Then upload the mail_notify.tar file to your web server. Next include the following in your automated deployment script.

##### Setting up Mail Notification ########
echo Setting up mail notification
echo Setting up mail notification >> /var/log/post_install.log

cd /tmp
lwp-download http://[server ip]/path/mail_notify.tar
tar xvf mail_notify.tar
mkdir /usr/lib/perl5/5.8.0/MIME
mv Lite.pm /usr/lib/perl5/5.8.0/MIME/

##### Move the files to where they belong #######
mv mail_notify.pl /usr/local/bin/
chmod +x /usr/local/bin/mail_notify.pl

####### Let’s send an email that the install is finished #####
/usr/local/bin/mail_notify.pl -t youremail@yourdomain.com -s “Server installation complete” -a /var/log/post_install.log -m “Server Installation complete please review the attached log file to verify your server installed correctly” -r [your smtp server]

Optionally you could set the smtp server in the mail_notify.pl script and not have to specify when sending a mail message.

if you include this at the end of your post installation portion part of your script but before the EOF line you will get a nice email notification informing you that your installation has finished with the post_install.log file attached.

Network configuration for automated ESX deployment

I have been asked this question a few times so I thought it would be wise to post an article on it. When deploying an automated build script with the kickstart and/or installation files located on http, ftp, or nfs there are network configuration dependencies that you need to be aware of.

The ESX installer is a modified version of anaconda which is the same installer used for RedHat and a few or Linux variants. Anaconda is what allows for the kickstart portion of the automated build script. Anaconda itself has some limitations as far as what it supports.

Anaconda does not support 802.1q VLAN tagging. If you plan on tagging the service console network traffic this will affect your kickstart installation. The anaconda installer will not tag the vlan id to the traffic and therefor will not be able to perform the installation. You have a few options on how to handle this.

  1. Don’t have the networking folks tag the vlan until after the install finished.  However this can cause problems if your post installation script needs to grab some files from across the network so be aware of what you are doing during your post installation.
  2. Use a dedicated deployment network.  If you use this option take a look at my ESX 3.x Deployment script #2 located on our download page.
  3. Don’t tag the service console traffic.  If you share vSwitch0 with both the vmkernel(vMotion) interface and the service console only tag the vmkernel traffic.  This still allows for isolation of the traffic.  Have your network guys set the service console vlan as the native(untagged)vlan.
  4. Create a custom installation CD with all the necessary files located on the CD.

Using the Ultimate Deployment Appliance to test ESX kickstart scripts – Part II

In Part 2 of this series we are going to deploy our virtual ESX host in a VMware Workstation 6.5 virtual machine. We will utilize the UDA setup that we created in the first part to this series. If you haven’t setup your UDA you will want to do so before proceeding. Make sure you check out the sample deployment scripts available on our download page. In this example I am deploying VMware ESX 3.5 Update 4 in VMware Workstation 6.5 build 126130.



Using the Ultimate Deployment Appliance to test ESX kickstart scripts – Part I

In this series I am going to walk you through setting up the Ultimate Deployment Appliance (UDA) and VMware Workstation 6.5 to test Automated ESX Deployment Scripts (kickstart).  The same principals that you will learn in this video also apply to using the UDA in a physical environment. The UDA is a very powerful appliance and I have found many uses for it. Using it as a medium to quickly and effectively test deployment scripts that I develop is just one.

Even in environments where the UDA is not allowed it can still be utilized. I regularly carry a 5 port gigabit switch which I can use to connect to my laptop to up to (4) servers to quickly deploy up to (4) ESX hosts at a time.



Deploying Automted Kickstart Scripts Over HTTP

Originally I was going to cover all the various options for initiating your automated kickstart installation as “Automated Deployment of ESX Hosts Part IV”, but I have since decided to cover each method individually as there is a lot to cover and it makes more sense to break them out.

In this post I am going to cover deploying your servers over the network utilizing HTTP. To begin you will need a few things in place for this to work.  Below is a list of what you will need:

  • Web Server to hosts the kickstart files and optionally your ESX installation.
  • ESX Installation media or ISO’s for all versions of ESX you plan to deploy
  • Your kickstart script

The first thing we need to do is setup our web server so we can host our kickstart files and optionally our installation files.  You can utilize apache, IIS, or whatever your favorite web server is for hosting HTTP.  You will need to configure a folder under your web server root for the files to be stored.  Below is my recommended structure.

-webroot
—deployment
——ESX35U1
——ESX35U2
——ESX35U3
——ESX35U4
——kickstart

Once the folder structure is created we need to copy the contents of the installation media to the respective folder. To do this you will literally copy everything on the CD and place it in the folder. Then next you will need to copy your kickstart.cfg files to the kickstart folder.

Once you have all the files uploaded to the web server it is a good idea to use your web browser to test that you are able to access them.

As part of our kickstart we define where we are going to be installing from with the following line replacing server_IP with your server IP address and ESX25U4 with the version you would like to install.

url –url http://server_IP/deployment/ESX35U4

If you wanted to pull just your kickstart.cfg files form the http server but install from the local CD media you would replace the above string with “cdrom” to let the kickstart know to look to the cdrom drive for the installation media.

Now that we have our web server up, our installation copied to our webserver, and our kickstart.cfg files on the server we can kick off our kickstart installation.

To do this we need to boot the server from the installation CD. You can boot from the CD in the cdrom drive or remote mounted over a lights out port like iLo, DRAC, or RSA. If you are going to remote mount the CD over a lights out connection you can use a much smaller portion of the ESX CD.

On your ESX installation media there is an iso file named boot.iso located under the “images” folder on the CD. You can extract that ISO image which is roughly 4mb and remote mount that to your server for the boot process if you intend to install over HTTP.

OK so now we boot off of our media either the full ESX CD or the boot.iso image and when the ESX installation screen appears we need to tell the installation where to find the kickstart file. There are a couple of options for this which are below:

If you are using dhcp then your installation string will look similar to the below string:

esx append ip=dhcp ksdevice=eth0 network ks=http://server_name/deployment/kickstart/kickstart.cfg

If you are not using dhcp it would like similar to the follow string:

esx append ip=192.168.1.2 netmask=255.255.255.0 gateway=192.168.1.1 ksdevice=eth0 network ks=http://Server_IP/deployment/kickstart/kickstart.cfg

The statement ksdevice=eth0 tells anaconda (the installer) to use the eth0 interface for the install. I recommend always using eth0 for your installs. ESX will by default make the install interface the Service Console interface. So it will become the interface that is assigned to vSwitch0.

If you are using a seperate kickstart file for each server then you can call each one by name. If you are using a script like the one I discuss here then you will only need to have one kickstart file.

ESX 3.x Deployment Script # 3

This script is very similar to ESX 3.x Deployment Script #1, but I made a handy change. I built this script to allow for easier modification for each ESX host you want to deploy. Once you change all the settings you need changed there is one important area where you will add information about all your ESX hosts.

Below if the area that you will need to be concerned with:

if ['hostname -s' == "esxhost1" ] ; then
esxcfg-vswif -i [Service_Console_IP] -n [Service_Console_Netmask] vswif0
esxcfg-vmknic -a -i [VMKernel_IP] -n [VMKernel_Netmask] "vMotion"
fi

You will create this if statement for each of your esxhosts you want to deploy. Once you setup each servers information in this area all you need to do is change the hostname to match the server you are deploying and that is it. If you use dhcp to set the initial installation IP and it is able to resolve to the appropriate hostname then you won’t even have to change the script.

For example if you change this line:

network --device eth0 --bootproto static --ip [SC IP ADDRESS] --netmask [SC NETMASK] --gateway [SC GATEWAY] --nameserver [NAMESERVERS comma serperated] --hostname [HOSTNAME] --addvmportgroup=0

to the following:

network --device eth0 --bootproto dhcp

and then add the following setting the appropriate IP addresses and hostnames:

if ['hostname -s' == "esxhost1" ] ; then
esxcfg-vswif -i [Service_Console_IP] -n [Service_Console_Netmask] vswif0
esxcfg-vmknic -a -i [VMKernel_IP] -n [VMKernel_Netmask] "vMotion"
fi

if ['hostname -s' == "esxhost2" ] ; then
esxcfg-vswif -i [Service_Console_IP] -n [Service_Console_Netmask] vswif0
esxcfg-vmknic -a -i [VMKernel_IP] -n [VMKernel_Netmask] "vMotion"
fi

and you setup each ESX server in dhcp and DNS you will never need to modify this script. You need to ensure that the DNS and gateway that the server initially get’s from DHCP are correct. If you are doing this on a different subnet then what you will be running your ESX server on then you will need to do this a little differently. This can be done with my ESX 3.x Deployment Script #2.

I have included a script with this code included in our download section.

Automated Deployment of ESX Hosts Part III

This is Part III of a multi-part blog. If you haven’t read Part I or II I recommend that you do before continuing.

In Part II we developed a standard build for our hosts that we are going to use to build our automated build script.  Keep in mind the information I provided in my standard build is not all inclusive and is limited for the demostration of building the script.

First we will start with the kickstart portion of the script.  The kickstart is what configures the basic part of the installation that you would normally do manually from the CD.  The ESX 3.5 kickstart is a modified version of anaconda.

In the begining of our script we have the Regional Settings that we need to set. Below are the regional settings for my installation.

# Regional Settings
keyboard us
lang en_US
langsupport --default en_US
timezone --utc America/New_York

Next we have some important installation settings. Part of the installtion section is the location of our ESX installations. In my script I have included some samples. cdrom, ftp, & http. I have chosen http for my installation so I did not comment http out. I personally prefer http because it can be much less problematic than ftp or nfs.


# Installatition settings
skipx
mouse none
firewall --disabled

# Authentication
auth –enableshadow –enablemd5

# Unencrypted root password: password
rootpw –iscrypted $1$5a17$In5zYe6YsCty76AycpGaf/

#Reboot Server after finished
reboot

#install ESX3.5U3 do not perform upgrade
install

#Location of installation medium
#cdrom
#url –url ftp://192.168.12.200/esx/esx353/
url –url http://192.168.12.200/esx/esx353/

In this next secition we are going to be configuring out hard disk drive partitions and boot loader options. You’ll notice that my disk drive is cciss/c0d0. This is because I will be installing on HP hardware with HP SCSI controllers. If I were to script this for IBM, DELL, or other servers I would typically utilize sda for my drive.


# Bootloader options
bootloader --driveorder=cciss/c0d0 --location=mbr

# Partitioning – This area is where you define your partitioning scheme
clearpart –all –initlabel –drives=cciss/c0d0
part /boot –fstype ext3 –size 100 –ondisk=cciss/c0d0 –asprimary
part swap –size 1600 –ondisk=cciss/c0d0 –asprimary
part / –fstype ext3 –size 10240 –ondisk=cciss/c0d0 –asprimary
part /var –fstype ext3 –size 8192 –ondisk=cciss/c0d0
part /tmp –fstype ext3 –size 4096 –ondisk=cciss/c0d0
part /opt –fstype ext3 –size 5120 –ondisk=cciss/c0d0
part /home –fstype ext3 –size 5120 –ondisk=cciss/c0d0
part None –fstype vmkcore –size 100 –ondisk=cciss/c0d0
part None –fstype vmfs3 –size 1 –grow –ondisk=cciss/c0d0

In this next secition we are configuring the networking settings for our Service Console. I recommend always using nic0(eth0) (vmnic0) for the Service Console and always perform your automated installation over this network interface. It is possible to do an automated installation over a different network interface but requires additional scritping to properly allcoate the interfaces.


# Network Configurations for service console. This will be applied to the Network interface that the kickstart is performed on. We are also choosing to not create a default portgroup.
network --device eth0 --bootproto static --ip 192.168.10.100 --netmask 255.255.255.0 --gateway 192.168.10.1 --nameserver 192.168.5.30 --hostname vmware1.sidtest.local --addvmportgroup=0

# VMWare License options
#Accept VMware License Agreement
vmaccepteula
#Configure host to talkt o license server
vmlicense –mode=server –server=27000@vcenter.sidtest.local –edition=esxFull –features=backup,vsmp

%vmlicense_text

%packages
@base

%post

That completes the kickstart portion of our script.

Everything beyong the %post portion of the kickstart is our post installation script.  The post installtion script is run on the first boot of the server.  A post installation script can contain bash, perl and other scripting but for simplicity I will be using bash scripts executing ESX and some other commands for my post installation part of the script.

I will start my Post Installation script off with the netowork configuration for the host.

#!/bin/sh
###### Configure Networking##########
###Setup vSwitch0######
echo Adding vmnic2 to vSwitch0 >> /var/log/post_install.log
esxcfg-vswitch -L vmnic2 vSwitch0

######### Add PortGroup for VMotion vmkernel adapter #########
echo Creating VMotion Portgroup
esxcfg-vswitch -A “VMotion” vSwitch0

## Tag VLAN to VMkernel ##
esxcfg-vswitch -p VMotion -v 301 vSwitch1

## Creating VMKnic and Assigning VMkernel IP and Gateway ##
echo Assigning VMKernel IP and Gateway – Please Wait
echo Assigning VMKernel IP and Gateway – Please Wait >> /var/log/post_install.log
esxcfg-vmknic -a -i 192.168.12.100 -n 255.255.250.0 “VMotion”
esxcfg-route 192.168.12.1

## Restart the Managment service so vimsh will notice changes to vSwitch0 ##
service network restart
sleep 300
service mgmt-vmware restart

## Enable vmkernel on vmk0 for vmotion ##
echo Enabling vMotion on VMkernel interface – Please Wait
echo Enabling vMotion on VMkernel interface >> /var/log/post_install.log
vmware-vim-cmd hostsvc/vmotion/vnic_set vmk0 >> /var/log/post_install.log

## Set both vmnic0 and vmnic2 to active for vSwitch0 ##
echo Confgiuring both vmnic0 and vmnic2 to be active for vSwitch0
echo Confgiuring both vmnic0 and vmnic2 to be active for vSwitch0 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy –nicorderpolicy-active=vmnic0,vmnic2 vSwitch0 >> /var/log/post_install.log

## Configure NIC Priority Order for VMkernel and Service Console ##
echo Configuring NIC Priority for SC and VMkernel – Please Wait
echo Configuring NIC Priority for SC and VMkernel >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/portgroup_set –nicorderpolicy-active=vmnic0 –nicorderpolicy-standby=vmnic2 vSwitch0 “Service Console” >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/portgroup_set –nicorderpolicy-active=vmnic2 –nicorderpolicy-standby=vmnic0 vSwitch0 “VMotion” >> /var/log/post_install.log

## Reject Forged Transmits and Mac Address Changes for vSwitch0 ##
echo Rejecting Forged Transmits and MAC Address CHanges for vSwitch0
echo Rejecting Forged Transmits and MAC Address CHanges for vSwitch0 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy –securepolicy-forgedxmit=false vSwitch0 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy –securepolicy-macchange=false vSwitch0 >> /var/log/post_install.log

## Reject Forged Transmits and Mac Address Changes for Service Console PortGroup ##
echo Rejecting Forged Transmits and MAC Address CHanges for Service Console PortGroup
echo Rejecting Forged Transmits and MAC Address CHanges for Service Console PortGroup >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/portgroup_set –securepolicy-forgedxmit=flase vSwitch0 “Service Console” >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/portgroup_set –securepolicy-macchange=false vSwitch0 “Service Console” >> /var/log/post_install.log

## Reject Forged Transmits and Mac Address Changes for VMotion Portgroup ##
echo Rejecting Forged Transmits and MAC Address CHanges for VMotion PortGroup
echo Rejecting Forged Transmits and MAC Address CHanges for VMotion PortGroup >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/portgroup_set –securepolicy-forgedxmit=flase vSwitch0 “VMotion” >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/portgroup_set –securepolicy-macchange=false vSwitch0 “VMotion” >> /var/log/post_install.log

echo Settings for vSwitch0 complete
echo Settings for vSwitch0 complete >> /var/log/post_install.log

####### Setup vSwitch1 #######
echo Configuring settings for vSwitch1 - Please Wait
echo Configuring settings for vSwitch1 - Please Wait >> /var/log/post_install.log
## Creating vSwitch1, assigning vmnics, creating portgroups and assigning vlans ##
echo Creating vSwitch1
echo Creating vSwitch1 >> /var/log/post_install.log
esxcfg-vswitch -a vSwitch1
echo Adding vmnic5, and vmnic7 to vSwitch1
echo Adding vmnic5, and vmnic7 to vSwitch1 >> /var/log/post_install.log
esxcfg-vswitch -L vmnic5 vSwitch1
esxcfg-vswitch -L vmnic7 vSwitch1

 

echo Creating PortGroups on vSwitch1
echo Creating PortGroups on vSwitch1 >> /var/log/post_install.log
esxcfg-vswitch -A "VLAN2" vSwitch1
esxcfg-vswitch -A "VLAN15" vSwitch1
esxcfg-vswitch -A "VLAN150" vSwitch1
esxcfg-vswitch -A "VLAN151" vSwitch1
esxcfg-vswitch -A "VLAN152" vSwitch1

 

echo Adding vlan assignments to PortGroups on vSwitch1
echo Adding vlan assignments to PortGroups on vSwitch1 >> /var/log/post_install.log
esxcfg-vswitch -p VLAN2 -v 2 vSwitch1
esxcfg-vswitch -p VLAN15 -v 15 vSwitch1
esxcfg-vswitch -p VLAN150 -v 150 vSwitch1
esxcfg-vswitch -p VLAN151 -v 151 vSwitch1
esxcfg-vswitch -p VLAN152 -v 152 vSwitch1

 

## Restart the Managment service so vimsh will notice changes to vSwitch1
service mgmt-vmware restart
## Wait 4 minutes for the hostd-vmdb service to fully start running it can take awhile for it to fully load and vimsh to work ##
echo Sleeping for 4 minutes - Please Wait
sleep 240
## Setting all vmnics to active for vSwitch1 ##
echo Confgiuring all vmnics to be active for vSwitch1
echo Confgiuring all vmnics be active for vSwitch1 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy --nicorderpolicy-active=vmnic5,vmnic7 vSwitch1 >> /var/log/post_install.log
## Reject Forged Transmits and Mac Address Changes for vSwitch1 ##
echo Rejecting Forged Transmits and MAC Address CHanges for vSwitch1
echo Rejecting Forged Transmits and MAC Address CHanges for vSwitch1 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-forgedxmit=false vSwitch1 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy --securepolicy-macchange=false vSwitch1 >> /var/log/post_install.log
echo Settings for vSwitch1 complete
echo Settings for vSwitch1 complete >> /var/log/post_install.log
 echo Configuring settings for vSwitch2 - Please Wait
echo Configuring settings for vSwitch2 - Please Wait >> /var/log/post_install.log

## Creating vSwitch2, assigning vmnics, creating portgroups and assigning vlans ##
echo Creating vSwitch2
echo Creating vSwitch2 >> /var/log/post_install.log
esxcfg-vswitch -a vSwitch2

echo Adding vmnic4, vmnic6 to vSwitch2
echo Adding vmnic4, vmnic6 to vSwitch2 >> /var/log/post_install.log
esxcfg-vswitch -L vmnic6 vSwitch2
esxcfg-vswitch -L vmnic4 vSwitch2

echo Creating PortGroups on vSwitch2
echo Creating PortGroups on vSwitch2 >> /var/log/post_install.log
esxcfg-vswitch -A “VLAN200” vSwitch2
esxcfg-vswitch -A “VLAN201” vSwitch2
esxcfg-vswitch -A “VLAN203” vSwitch2

echo Adding vlan assignments to PortGroups on vSwitch2
echo Adding vlan assignments to PortGroups on vSwitch2 >> /var/log/post_install.log
esxcfg-vswitch -p VLAN200 -v 200 vSwitch2
esxcfg-vswitch -p VLAN201 -v 201 vSwitch2
esxcfg-vswitch -p VLAN203 -v 203 vSwitch2

## Restart the Managment service so vimsh will notice changes to vSwitch2 ##
service mgmt-vmware restart

## Wait 4 minutes for the hostd-vmdb service to fully start running it can take awhile for it to fully load and vimsh to work ##
echo Sleeping for 4 minutes – Please Wait
sleep 240

## Setting all vmnics to active for vSwitch2 ##
echo Confgiuring all vmnics to be active for vSwitch2
echo Confgiuring all vmnics be active for vSwitch2 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy –nicorderpolicy-active=vmnic4,vmnic6 vSwitch2 >> /var/log/post_install.log

## Reject Forged Transmits and Mac Address Changes for vSwitch2 ##
echo Rejecting Forged Transmits and MAC Address CHanges for vSwitch2
echo Rejecting Forged Transmits and MAC Address CHanges for vSwitch2 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy –securepolicy-forgedxmit=false vSwitch2 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy –securepolicy-macchange=false vSwitch2 >> /var/log/post_install.log

echo Settings for vSwitch2 complete
echo Settings for vSwitch2 complete >> /var/log/post_install.log

echo Configuring settings for vSwitch3 – Please Wait
echo Configuring settings for vSwitch3 – Please Wait >> /var/log/post_install.log

## Creating vSwitch3, assigning vmnics, creating portgroups and assigning vlans ##
echo Creating vSwitch3
echo Creating vSwitch3 >> /var/log/post_install.log
esxcfg-vswitch -a vSwitch3

echo Adding vmnic1 and vmnic3 to vSwitch3
echo Adding vmnic10and vmnic3 to vSwitch3 >> /var/log/post_install.log
esxcfg-vswitch -L vmnic1 vSwitch3
esxcfg-vswitch -L vmnic3 vSwitch3

echo Creating PortGroups on vSwitch3
echo Creating PortGroups on vSwitch3 >> /var/log/post_install.log
esxcfg-vswitch -A “VLAN400” vSwitch3
esxcfg-vswitch -A “VLAN401” vSwitch3

echo Adding vlan assignments to PortGroups on vSwitch3
echo Adding vlan assignments to PortGroups on vSwitch3 >> /var/log/post_install.log
esxcfg-vswitch -p VLAN400 -v 400 vSwitch3
esxcfg-vswitch -p VLAN401 -v 401 vSwitch3

## Restart the Managment service so vimsh will notice changes to vSwitch3 ##
service mgmt-vmware restart

## Wait 4 minutes for the hostd-vmdb service to fully start running it can take awhile for it to fully load and vimsh to work ##
echo Sleeping for 4 minutes – Please Wait
sleep 240

## Setting all vmnics to active for vSwitch3 ##
echo Confgiuring all vmnics to be active for vSwitch3
echo Confgiuring all vmnics be active for vSwitch3 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy –nicorderpolicy-active=vmnic1,vmnic3 vSwitch3 >> /var/log/post_install.log

## Reject Forged Transmits and Mac Address Changes for vSwitch3 ##
echo Rejecting Forged Transmits and MAC Address CHanges for vSwitch3
echo Rejecting Forged Transmits and MAC Address CHanges for vSwitch3 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy –securepolicy-forgedxmit=false vSwitch3 >> /var/log/post_install.log
vmware-vim-cmd hostsvc/net/vswitch_setpolicy –securepolicy-macchange=false vSwitch3 >> /var/log/post_install.log

echo Settings for vSwitch3 complete
echo Settings for vSwitch3 complete >> /var/log/post_install.log 

Next I will configure the firewall rules to allow all the services I need to communicate in/out of the ESX host.

echo Configuring Firewall Rules - Please Wait
echo Configuring Firewall Rules >> /var/log/post_install.log

esxcfg-firewall -o 2381,tcp,in,hpim
esxcfg-firewall -o 2381,tcp,out,hpim
esxcfg-firewall -o 88,tcp,out,KerberosClient
esxcfg-firewall -o 6389,tcp,in,Naviagent
esxcfg-firewall -o 88,udp,out,
esxcfg-firewall -o 464,tcp,out,KerberosPasswordChange
esxcfg-firewall -e smbClient
esxcfg-firewall -e sshClient
esxcfg-firewall -e ntpClient
esxcfg-firewall -e CIMHttpServer
esxcfg-firewall -e snmpd
echo Configuring Firewall rules complete
echo Configuring Firewall rules complete >> /var/log/post_install.log

Here we are going to configure our DNS Servers 

echo Configuring DNS Servers - Please Wait
echo Configuring DNS Servers >> /var/log/post_install.log
echo Making backup of /etc/resolv.conf
echo Making backup of /etc/resolv.conf >> /var/log/post_install.log
cp /etc/resolv.conf /etc/resolv.conf.bak
echo nameserver 192.168.5.20 > /etc/resolv.conf
echo nameserver 192.168.5.21 >> /etc/resolv.conf

echo DNS Server configuration complete
echo DNS Server configuration complete >> /var/log/post_install.log

Now we will configure our NTP Settings.

echo configuring NTP Settings - Please Wait
echo configuring NTP Settings >> /var/log/post_install.log
echo Making backup of /etc/ntp.conf
echo Making backup of /etc/ntp.conf >> /var/log/post_install.log
cp /etc/ntp.conf /etc/ntp.conf.bak
echo restrict default kod nomodify notrap noquery nopeer > /etc/ntp.conf
echo restrict 127.0.0.1 >> /etc/ntp.conf
echo server 192.168.5.30 >> /etc/ntp.conf
echo server 192.168.5.31 >> /etc/ntp.conf
echo server 192.168.5.32 >> /etc/ntp.conf
echo driftfile /var/lib/ntp/drift >> /etc/ntp.conf
echo broadcastdelay 0.008 >> /etc/ntp.conf
echo Making backup of /etc/ntp/step-tickers
echo Making backup of /etc/ntp/step-tickers >> /var/log/post_install.log
cp /etc/ntp/step-tickers /etc/ntp/step-tickers.bak
echo server 192.168.5.30 > /etc/ntp/step-tickers
echo server 192.168.5.31 > /etc/ntp/step-tickers
echo server 192.168.5.32 > /etc/ntp/step-tickers
chkconfig --level 345 ntpd on
hwclock --systohc
service ntpd start >> /var/log/post_install.log
echo NTP settings complete
echo NTP settings complete >> /var/log/post_install.log
echo Restarting VMware Management Service
echo Restarting VMware Management Service >> /var/log/post_install.log
echo Sleeping for 4 minutes - Please Wait
service mgmt-vmware restart >> /var/log/post_install.log
sleep 240
echo VMware management Service restarted
echo VMware management Service restarted >> /var/log/post_install.log

Now I will set the service consoel memory to 800Mb

echo Setting Service Console Memory to 800Mb
echo Setting Service Console Memory to 800Mb >> /var/log/post_install.log

sed -i 's/memSize = "272"/memSize = "800"/g' /etc/vmware/esx.conf
esxcfg-boot -g
esxcfg-boot -b
echo Service Console Memory assigned to 800Mb complete
echo Service Console Memory assigned to 800Mb complete >> /var/log/post_install.log

Now we will configure AD integration for service console logins

echo Configuring PAM AD Integration
echo Configuring PAM AD Integration >> /var/log/post_install.log
esxcfg-auth --enablead --addomain=hq.nt.newyorklife.com --addc=hq.nt.newyorklife.com >> /var/log/post_install.log
echo Backing up /etc/krb5.conf
echo Backing up /etc/krb5.conf >> /var/log/post_install.log
cp /etc/krb5.conf /etc/krb5.conf.bak
echo Creating /etc/krb5.conf
echo [domain_realm] > /etc/krb5.conf
echo sidtest.local = sidtest.local >> /etc/krb5.conf
echo .sidtest.local = sidtest.local >> /etc/krb5.conf
echo [libdefaults] >> /etc/krb5.conf
echo default_realm = sidtest.local >> /etc/krb5.conf
echo [realms] >> /etc/krb5.conf
echo sidtest.local = { >> /etc/krb5.conf
echo admin_server = sidtest.local:464 >> /etc/krb5.conf
echo default_domain = sidtest.local >> /etc/krb5.conf
echo kdc = dc1.sidtest.local:88 >> /etc/krb5.conf
echo kdc = dc2.sidtest.local:88 >> /etc/krb5.conf
echo } >> /etc/krb5.conf
echo Finished Configuring PAM AD Integration
echo Finished Configuring PAM AD Integration >> /var/log/post_install.log
echo Adding authorized Active Directory users - Please Wait
echo Adding authorized Active Directory users >> /var/log/post_install.log
useradd -m ssmith -g wheel -G adm
useradd -m jbower -g wheel -G adm
useradd -m cobrian -g wheel -G adm
useradd -m talmeda -g wheel -G adm
useradd -m t01bo3L -g wheel -G adm
useradd -m t44bblk -g wheel -G adm
echo Autorized Active Directory Users added
echo Autorized Active Directory Users added >> /var/log/post_install.log

 

Now we will prevent root login from the physical console

 

echo Preventing root form loggin on to console
echo Preventing root form loggin on to console >> /var/log/post_install.log
echo Backing up /etc/securetty
echo Backing up /etc/securetty >> /var/log/post_install.log
mv /etc/securetty /etc/securetty.bak
touch /etc/securetty
echo Preventing root form loggin on to console complete
echo Preventing root form loggin on to console complete >> /var/log/post_install.log

Here we will set our local password policy for all local accounts

echo Setting local password policy
echo Setting local password policy >> /var/log/post_install.log
echo Setting maximum number of days to keep a password
esxcfg-auth --passmaxdays=90
echo Setitng password minimum days between changes
esxcfg-auth --passmindays=1
echo Setting Password warning time befor change required
esxcfg-auth --passwarnage=14
echo Setting local password policy complete
echo Setting local password policy complete>> /var/log/post_install.log

Now let’s set the Message of Day and Welcome.js message

echo COnfiguring MOTD login banner – Please Wait
echo COnfiguring MOTD login banner >> /var/log/post_install.log

echo “

Warning!!! This computer system is private and may be accessed only
by authorized users. Data and programs in this system are confidential
and proprietary to the system owner and may not be accessed without
authorization. Unauthorized users or users who exceed their authorized
level of access are subject to disciplinary action, up to and including
termination and are subject to prosecution under state or federal law.
Activity on this computer system is logged.

” > /etc/motd

echo MOTD Login Banner COnfiguration Complete
echo MOTD Login Banner COnfiguration Complete >> /var/log/post_install.log

echo COnfiguring /usr/lib/vmware/hostd/docroot/en/welcomeRes.js banner – Please Wait
echo COnfiguring /usr/lib/vmware/hostd/docroot/en/welcomeRes.js banner >> /var/log/post_install.log

echo Backing up existing WelcomeRes.js
echo Backing up existing WelcomeRes.js >> /var/log/post_install.log
mv /usr/lib/vmware/hostd/docroot/en/welcomeRes.js /usr/lib/vmware/hostd/docroot/en/welcomeRes.js.bak

esxcfg-firewall –allowOutGoing
sleep 20

cd /tmp

lwp-download http://192.168.12.200/welcomeRes.js

cp /tmp/welcomeRes.js /usr/lib/vmware/hostd/docroot/en/welcomeRes.js

esxcfg-firewall –blockOutGoing
sleep 20

echo COnfiguring /usr/lib/vmware/hostd/docroot/en/welcomeRes.js banner Complete
echo COnfiguring /usr/lib/vmware/hostd/docroot/en/welcomeRes.js banner Complete >> /var/log/post_install.log

Last but not least we will install the EMC Naviagent and the HP SIM agent

echo Permorming the installation of the EMC NaviAgentCli
echo Permorming the installation of the EMC NaviAgentCli >> /var/log/post_install.log

echo Downloading the EMC NaviAgentCli – Please Wait
echo Downloading the EMC NaviAgentCli – Please Wait >> /var/log/post_install.log
esxcfg-firewall –allowOutGoing
sleep 20

cd /tmp
lwp-download http://192.168.12.200/naviagentcli-6.19.4.7.0-1.noarch.rpm

esxcfg-firewall –blockOutGoing
sleep 20

echo Installing the EMC NaviAgentCli – Please wait
echo Installing the EMC NaviAgentCli – Please wait >> /var/log/post_install.log
rpm -ivh naviagentcli-6.19.4.7.0-1.noarch.rpm
sleep 20
/etc/init.d/naviagent start

echo Permorming the installation of the EMC NaviAgentCli Complete
echo Permorming the installation of the EMC NaviAgentCli Complete >> /var/log/post_install.log

echo Permorming the installation of the HP SIM Agent ver 7.9.1
echo Permorming the installation of the HP SIM Agent ver 7.9.1 >> /var/log/post_install.log

echo Downloading the HP SIM Agent – Please Wait
echo Downloading the HP SIM Agent – Please Wait >> /var/log/post_install.log
esxcfg-firewall –allowOutGoing
sleep 20

cd /tmp
lwp-download http://192.168.12.200/NYL/hpmgmt-7.9.1-vmware3x.tgz

esxcfg-firewall –blockOutGoing
sleep 20

echo Building HP SIM Answer File – Please wait
echo Building HP SIM Abswer File – Please Wait >> /var/log/post_install.log

echo export CMASILENT=”YES” > /tmp/sidtest_AF.conf
echo export CMANOSTARTINSTALL=”hpasmd” >> /tmp/sidtest_AF.conf
echo export ENABLEHPIMPORT=Y >> /tmp/sidtest.conf
echo export ENABLESNMPSERVICE=Y >> /tmp/sidtest_AF.conf
echo export ENABLESIMCERTPORT=Y >> /tmp/sidtest_AF.conf

echo Installing the HP SIM Agent – Please wait
echo Installing the HP SIM Agent – Please wait >> /var/log/post_install.log
tar xvfz /tmp/hpmgmt-7.9.1-vmware3x.tgz
cd /tmp/hpmgmt/791/
./installvm791.sh –silent –inputfile /tmp/sidtest_AF.conf
sleep 60
echo Installing the HP SIM Agent – Complete
echo Installing the HP SIM Agent – Complete >> /var/log/post_install.log

echo Configuring SNMPd.conf – Please wait
echo Configuring SNMPD.conf – Please wait >> /var/log/post_install.log

echo Making a backup of /etc/snmp/snmpd.conf
echo Making a backup of /etc/snmp/snmpd.conf >> /var/log/post_install.log

mv /etc/snmp/snmpd.conf /etc/snmp/snmpd.conf.bak

echo dlmod cmaX /usr/lib/libcmaX.so > /etc/snmp/snmpd.conf
echo rocommunity hrasmidep 127.0.0.1 >> /etc/snmp/snmpd.conf
echo rocommunity hrasmidep 172.31.44.12 >> /etc/snmp/snmpd.conf
echo rocommunity hrasmidep 172.28.137.14 >> /etc/snmp/snmpd.conf
echo trapcommunity hrasmidep >> /etc/snmp/snmpd.conf
echo trapsink 172.31.44.12 hrasmidep >> /etc/snmp/snmpd.conf
echo trapsink 172.28.137.14 hrasmidep >> /etc/snmp/snmpd.conf
echo syscontact root@localhost >> /etc/snmp/snmpd.conf
echo syslocation CNJ >> /etc/snmp/snmpd.conf
echo dlmod SNMPESX /usr/lib/vmware/snmp/libSNMPESX.so >> /etc/snmp/snmpd.conf

echo Configuring SNMPD.conf – Complete
echo Configuring SNMPD.conf – Complete >> /var/log/post_install.log

echo Permorming the installation of the HP SIM Agent ver 7.9.1 Complete
echo Permorming the installation of the HP SIM Agent ver 7.9.1 Complete >> /var/log/post_install.log

echo Recording Build Script data to host > /etc/build_info
echo Built using Sid_Smith_kickstart_test_script_v1_0.cfg >> /etc/build_info
echo Script Version 1.0 >> /etc/build_info
echo ESX 3.5 Update 3 >> /etc/build_info


######## Installationa and Configuration is finished ######
echo Your server is now installed and configured.  Please review the installation
echo log file at /var/log/post_install.log to verify that there are no errors. Your
echo server will now entered into maintenace mode and will reboot.  Please follow the
echo remaining steps in your automated deployment documentation. 
echo          
echo     Script developed by: Sid Smith
##########
sleep 30
##### Put Server in Maintenance Mode #####
echo Server will now enter maintenance mode
echo Server will now enter maintenance mode >> /var/log/post_install.log
vmware-vim-cmd /hostsvc/maintenance_mode_enter >> /var.log.post_install.log
EOF1
###Make esxcfg.sh eXcutable
chmod +x /tmp/esxcfg.sh

###Backup original rc.local file
cp /etc/rc.d/rc.local /etc/rc.d/rc.local.bak

###Make esxcfg.sh run from rc.local and make rc.local reset itself
cat >> /etc/rc.d/rc.local <<EOF
cd /tmp
/tmp/esxcfg.sh
mv -f /etc/rc.d/rc.local.bak /etc/rc.d/rc.local
shutdown -r now
EOF

It’s important to remember that you should always use a linux compatible editor to edit your script.  If you have been using Notepad remember to open change and save your script in a linux compatible editor before trying to run.  In the next and final part to this blog we will go throuigh deployment options and perform a test deployment of this script.

 I have also included the script as an attachment to this post below:

ESX Automated Deployment Script

*Please excuse some of the formatting in this post.  I am still getting use to the interface and had some issues with the formatting.

Automated Deployment of ESX Hosts Part II

This is part II of a multi-part blog. If you haven’t read Part I you can read it here.

The homework from part Iof this blog was to determine what your standard configuration is.  If your having trouble deciding what this should be hopefully today’s session will help with that.  Below you will see all the information that will make up my “standard” build and this will be the information that I use to develop my script.

Sid’s Standard Build

The following information directly relates to the kickstart portion of the script.

Regional Settings:

  • Keyboard layout: US
  • language: US
  • Time Zone UTC America/New_York

Service Console Configuration items:

Partitions:

  • /boot =  100Mb
  • swap = 1600Mb
  • / = 10240Mb
  • /var = 8192Mb
  • /tmp = 4096Mb
  • /opt = 5120Mb
  • /home = 5120Mb
  • vmkcore = 100Mb
  • /vmfs = fill to max

License Server information:

vcenter.sidtest.com (Licenses server is installed on my vCenter Server.

The remaining information is directly related to the post installation portion of the script

Networking:

8 Network Interfaces

vSwitch0 (vmnic0, vmnic2) – Portgroups (Service Console(Vlan 300) & vMotion(vmkernel)(Vlan301))

vswitch1 (vmnic7, vmnic5) – VM Production Portgroups (vlans 2, 15, 150,151,152)

vswitch2 (vmnic4, vmnic6) – DMZ Portgroups (vlans 200, 201, 203)

vSwitch3 (vmnic1, vmnic3) – Backup Networks (vlans 400 & 401)

The two illustrations below show visual diagrams of my network configurations.

Diagram 1

net_design1

Diagram 2

net_design2

NOTE:

When developing your standard build it is always a great idea to manually build one server and map out the network interfaces to vmnics.  It is also very important to use the same hardware and PCI placement for all servers in a cluster.

I have also decided to reject Forged Transmits and MAC address changes on all the vSwitch’s in my environment.

The following is host specific network information for this particular host:

Hostname: vmware1.sidtest.local
Service Console IP Address: 192.168.10.100
Service Console Subnet: 255.255.255.0
Service Console Gateway: 192.168.10.1
VMkernerl IP address: 192.168.12.100
VMkernel Subnet Mask: 255.255.255.0
VMkernel Gateway: 192.168.12.1
DNS Server 1: 192.168.5.20
DNS Server 2: 192.168.5.21
NTP Server 1: 192.168.5. 30
NTP Server 2: 192.168.5.31
NTP Server 3: 192.168.5.32
AD Server 1: 192.168.5.20
AD Server 2: 192.168.5.21

Remaining Build Items

In my build I will be building Active Directory (krb5) user authentication into the service console.  Users will not be allowed to login directly as root.  They will be forced to login using their active directory username and password.  I will be granting the following users login privileges.

ssmith
jbower
cobrian
talmeda

I will also be assigning a grub password to prevent someone from booting up using init 1 and gaining service console access without entering a password.
I will be preventing users from logging into the physical console as root.
I will be setting the following password policy for local user accounts:

  • Maximum password age of 90 days
  • Minimum of 1 day between password changes
  • 14 Day password warning before change required

I will be setting the following MOTD (Message of the Day) and Welcome.js (For web logins) message:

Warning!!! This computer system is private and may be accessed only
by authorized users.  Data and programs in this system are confidential
and proprietary to the system owner and may not be accessed without
authorization. Unauthorized users or users who exceed their authorized
level of access are subject to disciplinary action, up to and including
termination and are subject to prosecution under state or federal law.
Activity on this computer system is logged.

I will be running the following agents in the Service Console:

  • EMC Naviagent
  • HP System Insight Management Agent

I will be allocating 800Mb of Memory to the Service Console.

This completes my ESX Build information.  This is not an all inclusive list of options that can be set or configured as part of an automated installation.  It is possible to configure many other options but for the purpose of this example the preceding standard build will suffice.

Be sure to check back for Part III of this series.  In Part III we will be building our script to support the build that was laid out in this post.  Then later in Part IV we will then go on to demonstrate the options for deploying your script to your ESX Hosts.

Automated Deployment of ESX Hosts Part I

A very important part of deploying your Virtual Infrastructure environment is consistency when it comes to the configuration of the ESX hosts.   Time and time again as common practice I see users deploying ESX hosts manually with no documentation to follow other than what they have in their heads.  Deploying ESX hosts in this fashion can lead to inconsistent configurations between hosts.  Choosing manual installations over scripted installations can lead to all sorts of problems some of which can be very difficult to diagnose.

There is a very common misconception that scripted installations are difficult or not worth the time and effort to create.  This couldn’t be farther from the truth.  Utilizing scripted installations can save countless hours of time spent manually installing ESX and troubleshooting problems from inconsistent manual installs.  This is part I of a multi-part blog in which I am going to walk you through the different options that are available with scripted installations.  I am also going to share with you some sample scripts and methods to streamline the testing of scripts you develop.  Before we begin to develop a script we need to know what our “standard” configuration is going to be.  Without having a standard configuration we don’t know what we need to configure.  It’s important to develop a standard configuration for your environment.  If you have multiple clusters in your environment your standard configuration may have minor changes from cluster to cluster but should ultimately be consistent across the environment to make troubleshooting and maintenance much simpler.

In part II of this blog post I will post my “standard” configuration that will be used to build my sample automated kickstart installation script.  My standard configuration will include the following:

  • Service Console Partitioning scheme
  • Service Console memory size
  • Active Directory Servers for Service Console PAM integration
  • AD Users to have access to the Service Console
  • Service Console Security Policy
  • Agents to be install in the Service Console and their configurations
  • ESX Hosts Networking Configuration including routing, DNS, NTP, etc…
  • Licensing Server Information

Once I have a “standard” configuration I will perform a walk through using it to develop a script that will be used for the Automated Deployment.  Once the script is developed then we will cover the options for deploying a host using the script.  These options include:

  • PXE Boot using a tool like the UDA appliance
  • Install from CD using a hosted kickstart file.
  • Network installation using http, ftp, or nfs
  • Custom ESX installation CD’s.

Your homework assignment before “Automated deployment of ESX hosts par II” is to determine what your “standard” ESX build is and have the information handy to start building your script. So don’t forget to check back for my next post where we start to build our kickstart files.