VMware Ballooning

VMware ballooning is one of the hard concept to grasp. There are a lot of misunderstanding out there about this feature. I have been discussing this feature with customers and students during the last 5 years. This is my attempt to explain balloning.

VMware ballooning is a memory reclamation  technique used when and ESXi host is running low on memory. You should not see balloning if your hosts is performing like it should. To understand ballooning we would have to take a look at the following picture:

This picture shows the three levels of memory in a virtual environment. In a physical world we would only have the two top levels (virtual memory & guest physical memory) but in the virtual world we also have the host physical memory. What is important to know is that the hypervisor (ESXi) has no knowledge of what is happening inside the virtual machine (grey area). The hypervisor maps memory when the virtual machines asks for it. The hypervisor will then give it memory from “host physical memory” but only if memory is available. If memory is not available the memory can med mapped to the .vswp file on a vmfs or nfs datastore. The virtual machine has no knowledge if the memory is mapped to physical memory or to a disk. This is called hypervisor swapping, and this is the last resort for the vmkernel to use this mechanism.

Ballooning in short is a process where the hypervisor reclaims  memory back from the virtual machine. Ballooning is an activity that happens when the ESXi host is running out of physical memory. The demand of the virtual machine is too high for the host to handle.

Lets take a high level example:

1. Inside a virtual machine you start an application. For instance solitaire
2. solitaire as an application will ask the guest operating system (in this case windows) for memory. Windows will give it memory and map it from the virtual memory -> guest physical memory 
3. what happens next is that the hypervisor sees the request for memory and the hypervisor maps guest physical memory -> host physical memory
4. Now everything is perfect. You play solataire for a few hours. And then you close it down.
5. When you close solitaire the guest operating system will mark the memory as “free” and make it available for other applications. BUT since the hypervisor does not have access to Windows’ “free memory” list the memory will still be mapped in “host physical memory” and putting memory load on the ESXi host.
6. This is where ballooning comes into place. In case of an ESXi host running low on memory the hypervisor will ask the “balloon” driver installed inside the virtual machine (with VMware Tools) to “inflate”
7. The balloon driver will inflate and because it is “inside” the operating system it will start by getting memory from the “free list”. The hypervisor will detect what memory the balloon driver has reclaimed and will free it up on the “host physical memory” layer!

The balloon driver can inflate up to a maximum of 65%. For instance a VM with 1000MB memory the balloon can inflate to 650MB. The way to avoid ballooning is not to uninstall the balloon driver but to create a “Memory Reservation” for the virtual machine. In case of full inflation for this particular VM the result is the hypervisor gets 650MB memory reclaimed. The downfall of this is that you risk your VM to do Guest OS Swapping to its page file! Just remember page file swapping is better than hypervisor swapping. Hypervisor swapping happens without the guest operating system is aware of it. Page file swapping it is the OS that decides what pages to swap to disk!

To check for ballooning you can either open ESXTOP or the vCenter Performance Graphs.

===================================================================

Virtual memory ballooning allows a physical host to recapture unused memory on its guest virtual machines and allocate the resource where needed.

Virtual memory ballooning is a computer memory reclamation technique used by a hypervisor to allow the physical host system to retrieve unused memory from certain guest virtual machines (VMs) and share it with others. Memory ballooning allows the total amount of RAM required by guest VMs to exceed the amount of physical RAM available on the host. When the host system runs low on physical RAM resources, memory ballooning allocates it selectively to VMs.

If a VM only uses a portion of the memory that it was allocated, the ballooning technique makes it available for the host to use. For example, if all the VMs on a host are allocated 8 GB of memory, some of the VMs will only use half the allotted share. Meanwhile, one VM might need 12 GB of memory for an intensive process. Memory ballooning allows the host to borrow that unused memory and allocate it to the VMs with higher memory demand.  

The guest operating system runs inside the VM, which is allocated a portion of memory. Therefore, the guest OS is unaware of the total memory available. Memory ballooning makes the guest operating system aware of the host's memory shortage.

Virtualization providers such as VMware enable memory ballooning. VMware memory ballooning, Microsoft Hyper-V dynamic memory, and the open source KVM balloon process are similar in concept. The host uses balloondrivers running on the VMs to determine how much memory it can take back from an under-utilizing VM. Balloon drivers must be installed on any VM that participates in the memory ballooning technique.

Balloon drivers get the target balloon size from the hypervisor and then inflate by allocating the proper number of guest physical pages within the VM. This process is known as inflating the balloon; the process of releasing the available pages is known as deflating the balloon. 

VM memory ballooning can create performance problems. When a balloon driver inflates to the point where the VM no longer has enough memory to run its processes within itself, it starts using another VM memory technique known as memory swapping. This will slow down the VM, depending upon the amount of memory to recoup and/or the quality of the storage IOPSdelivered to it.