vSphere Memory Utilization

Determining actual host memory utilization can present a challenge.  The issue boils down to when the host breaks large memory pages into small pages which can then be shared (aka Transparent Page Sharing).  TPS only works with small pages.

To determine actual host memory usage, especially after the host is using a significant amount (65-70%+), you have to monitor an esxtop parameter called COWH.  For example, you may get to 70% utilization and find adding another host to the cluster doesn’t decrease utilization, but adding another VM or two does(!).  This is because at some point the host will  break some of the large pages into small pages and then TPS kicks in.

For more in depth reading see:

Large Pages, Transparent Page Sharing and how they influence the consolidation ratio

Large Pages – a problem of perception and measurement

Large Pages, theory and usage with SQL Server 2008