Virtual memory refers to the memory available for virtual machines from the physical memory present on the host. VMkernel makes the operating system in a virtual machine take the virtual memory and use it as a physical memory for all its operations. Virtual memory is not limited to the size of the physical memory available, and can be extended or limited as per requirements of the virtual machine.
vSphere offers flexible memory management solutions and different kind of memories as per different needs. Three different memories used by vSphere are:
- Host physical memory: The memory which is physically present on the host. This memory is managed by VMkernel and provides addressable memory space to the virtual machine.
- Guest operating system physical memory: The memory which is presented to the virtual machine once it is created.
- Guest operating system virtual memory: The memory which is presented to the application by the guest operating system.
Virtual machine memory over-commitment
Memory over commitment occurs when the physical memory installed on the physical ESXi host is less than the sum of the memory allocated to the virtual machines on that host. When an ESXi host memory is over committed, VMkernel uses several techniques for virtual machines to utilize the memory. In this section, we will talk about these few techniques. Basically, it is preferable to avoid memory over commitment to ensure that all virtual machines can address the memory resources they need.
Virtual machine swap files
Virtual machine swap files are created when a virtual machine is powered on. The size of the virtual machine’s swap file is equal to the size of the allocated memory minus its reservation.
Memory reclamation techniques
VMware uses various techniques to reclaim memory assigned to ESXi hosts. These are:
- Transparent page sharing: TPS allows pages with identical content to be stored only once. So if there are multiple virtual machines with identical pages (for example, in Virtual Desktop Infrastructure setups), TPS will only store them once in the memory, and hence allow higher level of memory commitment. In December 2014/January 2015, VMware decided to disable this feature between particular virtual machines by default because of independent research studies regarding TPS. Those results demonstrated that it might be possible to gain access to shared memory resources. To address this, VMware also implemented a more secure memory salting mechanism. Administrators are able to enable Intra-VM TPS as per their needs. Details can be found on the VMware knowledge base at http://kb.vmware.com/kb/2080735..
- Ballooning: During the times of memory contention, VMkernel looks for the opportunity to reclaim idle and unallocated memory from virtual machines, and transfer that to some other virtual machines that need more memory. Ballooning happens using the VMware memory control driver (VMMEMCTL) that is installed with VMware tools. The VMMEMCTL driver is also called the balloon driver.
- Memory Compression: Memory compression is another technique which VMkernel uses to reclaim host physical memory. This technique attempts to reclaim some memory by compressing memory pages. Using this mechanism produces a negligible amount of overhead, but it is still more effective than paging memory to disk.
- Host Level SSD swapping: This option will be used after TPS, ballooning, and memory compression have been tried. In this option, the administrator can configure a SSD as a host cache. This host cache will be used as a swap space for the virtual machine.
- Page virtual machine memory out to disk: This is the last and the least desirable technique in which the virtual machine memory is moved out to the disk. The performance is very poor when this technique is used. Basically, the administrator should size ESXi hosts so that this technique is not used during normal operation.
An administrator can configure a virtual machine with as many as 128 vCPUs on vSphere 6.0 (vSphere 5.5: 64). The VMkernel scheduler dynamically schedules vCPUs on the physical CPU of the server. A vCPU is the central processing unit of the v2ioirtual machine. When a vCPU of a single or multi vCPU virtual machine has to be scheduled, VMkernel maps the vCPU to available logical processor.
CPU load balancing
The CPU scheduler can use each logical processor individually to map it to virtual machines. VMkernel manages the processor time to guarantee that the load is distributed across all the cores in the system. Every two to 40 miliseconds, VMkernel looks to migrate the vCPU from one logical processor to another, to keep the load fairly balanced.
VKkernel does the best to schedule virtual machines on different cores rather than have two logical processors on the same core.
Hyper-threading gives the ability to a core to execute to threads or two sets of instructions simultaneously. Hyper-threading, when enabled, can increase the performance of the CPU. Beginning with vSphere 5.x, VMware highly recommends enabling Hyper-threading on ESXi hosts. To enable Hyper-threading, the following steps need to be performed:
- Check if your system supports Hyper-threading. Check your vendor’s manual and CPU datasheet regarding this.
- Enable Hyper-threading in the system’s BIOS settings.
- Login into vCenter Server using vSphere Web Client.
- Select the host from the inventory.
- Click Manage, Settings, and select Advanced System Settings under System. To turn on Hyper-threading, set the value for the setting VMkernel.Boot.hyperthreading to true.