Removing a VMFS Datastore

To remove VMFS datastores in the past, I always made sure there were no VMs left on the datastore, right-clicked and chose "Delete".  Well, apparently thats the wrong way to do it!  I must have been lucky, as VMware claims this method could result in an APD (All Paths Down) state.  If you don't know what that is, let me tell you it's bad (I have experienced this but for a different reason).  Your host(s) will lose access to storage.

I stumbled upon this vSphere blog post that has the procedure to remove a datastore the right way: Best Practice: How to correctly remove a LUN from an ESX host

UPDATE: For ESXi 5.0, I found it better to follow the KB mentioned in that blog post:
Unpresenting a LUN in ESXi 5.x

For ESXi 5.0, here's how I do a slightly modified vesion of the procedure listed in the post above with more information on items such as HA:

1. Make sure all VMs are evacuated from the datastore/LUN.
2. Using Datastore Browser, make sure there aren't any left-over directories or files.  If there are, delete them (make sure they can safely deleted first, of course).  Exception: the HA directory - you'll remove that a different way later.  Also note that you won't be able to delete a file if it's in use.
3. Make sure all vSphere features are disabled for the datastore (i.e. SIOC, Storage DRS)
1. HA Datastore Heartbeating:  There may be some cases where HA has chosen the datastore you're trying to remove.  In this case, edit cluster settings and change Datastore Heartbeating to "select only from my preferred datastores", then select at least three datastores other than the one you're trying to remove.  I highly recommend changing this setting back to "Select any of the cluster datastores" after you're finished removing this one.
4. Next, for each host, go to Configuration\Storage\Datastores View, right-lick on the datastore and choose "unmount":
1. The "Unmount Datastore Wizard" appears. Make sure all hosts are selected.
2. Click next.  Hopefully the next step will look like this:
   
3. Click "Next" then "Finish".  After a minute the datastore will be grayed-out and italicized.
5. Then go to Configuration\Storage\Devices View, right-lick on the datastore and choose "detach"
1. You get a pop-up dialog window similar to the one show above.  Choose "OK.  The device will be grayed-out and italicized after a few seconds.
6. Go in to your SAN and remove LUN masking/unpresent the LUNs to the ESXi hosts.
7. Finally, right-click on your cluster and choose "Rescan for Datastores".  You may get a StorageConnectivityAlarm alert for every host in your cluster unless you disable this alert first.

I like to go into every host and check the storage adapter for the LUN just to make sure but I haven't found a problem with this procedure yet.

vSphere Memory Utilization

Determining actual host memory utilization can present a challenge.  The issue boils down to when the host breaks large memory pages into small pages which can then be shared (aka Transparent Page Sharing).  TPS only works with small pages.

To determine actual host memory usage, especially after the host is using a significant amount (65-70%+), you have to monitor an esxtop parameter called COWH.  For example, you may get to 70% utilization and find adding another host to the cluster doesn’t decrease utilization, but adding another VM or two does(!).  This is because at some point the host will  break some of the large pages into small pages and then TPS kicks in.

For more in depth reading see:

Large Pages, Transparent Page Sharing and how they influence the consolidation ratio

Large Pages – a problem of perception and measurement

Large Pages, theory and usage with SQL Server 2008