Describe ballooning and how it works. Explain the two values
and why they are there.
Point the places where a user can see more balloon information and
how each driver operates.
Signed-off-by: Alexander Atanasov <[email protected]>
---
Documentation/mm/balloon.rst | 138 +++++++++++++++++++++++++++++++++++
1 file changed, 138 insertions(+)
create mode 100644 Documentation/mm/balloon.rst
diff --git a/Documentation/mm/balloon.rst b/Documentation/mm/balloon.rst
new file mode 100644
index 000000000000..9fe9e7b228de
--- /dev/null
+++ b/Documentation/mm/balloon.rst
@@ -0,0 +1,138 @@
+===========================================
+Balloon: usage information visible by guest
+===========================================
+Background:
+===========
+The ballooning mechanism allows VM guests to reduce their memory size
+(thus relinquishing memory to the Host) and to increase it back (thus
+taking back memory from the Host).
+During OOM guest issues or guest low-performance issues
+investigations it is important to know if the Host has grabbed some of the
+Guest memory via the ballooning mechanism.
+
+Implementation description:
+===========================
+/proc/meminfo::
+
+ InflatedTotal: 2097152 kB
+ InflatedFree: 0 kB
+
+The difference comes from the way drivers account for inflated memory:
+ - Drivers that call adjust_managed_page_count InflateTotal
+ - Drivers that do NOT call adjust_managed_page_count InflateFree
+
+ * It is possible for one driver to operate in both modes depending on config options.
+
+
+The balloon statistics are also printed by show_mem() function, which
+is called on OOM condition or Alt+SysRQ+m is pressed.
+The show_mem() string is similar to /proc/meminfo and it is like::
+
+ Balloon InflatedTotal:XXXkB InflatedFree:YYYkB
+
+Additional balloon information is available via debugfs:
+ - KVM features file: /sys/devices/pci\*/\*/virtio\*/features
+ - Hyper-V balloon guest file: /sys/kernel/debug/hv-balloon
+ - VMware balloon guest file: /sys/kernel/debug/vmmemctl
+
+KVM balloon
+-----------
+The ballooning is implemented via virtio balloon device.
+Depending on the options the ballooned memory is accounted for in two ways:
+
+1. If deflate on OOM is enabled - ballooned memory is accounted as used.
+2. If deflate on OOM is not enabled - ballooned memory is subtracted
+ from total RAM.
+
+Q: How to check if "deflate on OOM" feature is enabled?
+A: Check balloon "features" file content.
+To decipher balloon bits are defined in include/uapi/linux/virtio_balloon.h
+Currently "deflate on OOM" feature is stored in the 2nd bit::
+ #define VIRTIO_BALLOON_F_DEFLATE_ON_OOM 2 /* Deflate balloon on OOM */
+Examples::
+
+ Without deflate on OOM:
+ # cat /sys/devices/pci0000:00/0000:00:03.0/virtio0/features
+ 0100000000000000000000000000110010000000000000000000000000000000
+ With deflate on OOM:
+ # cat /sys/devices/pci0000:00/0000:00:03.0/virtio0/features
+ 0110000000000000000000000000110010000000000000000000000000000000
+How to find virtio balloon device among other virtio devices?
+(check if the "virtio_balloon" module is loaded)::
+ # ls -l /sys/bus/virtio/drivers/virtio_balloon/virtio*
+ /sys/bus/virtio/drivers/virtio_balloon/virtio3 ->
+ ../../../../devices/pci0000:00/0000:00:07.0/virtio3
+
+To check virtio_balloon features::
+
+ # cat /sys/bus/virtio/drivers/virtio_balloon/virtio*/features
+ 0110000000000000000000000000110010000000000000000000000000000000
+Balloon guest statistics output example::
+
+ # cat /sys/kernel/debug/virtio-balloon
+ InflatedTotal: 0 kB
+ InflatedFree: 0 kB
+
+- If "InflatedTotal" is not zero, it means the "deflate on OOM" feature is
+ **not** set and the provided amount of memory is subtracted from the total RAM
+ inside the Guest.
+- If "InflatedFree" is not zero, it means "deflate on OOM" feature is set and
+ the provided amount of memory is accounted as "used" inside the Guest.
+- Both "InflatedTotal" and "InflatedFree" cannot be non-zero at the same time.
+
+Hyper-V balloon
+---------------
+Balloon guest statistics output example::
+
+ # cat /sys/kernel/debug/hv-balloon
+ host_version : 2.0 // Hyper-V version the Guest is running under
+ capabilities : enabled hot_add
+ state : 1 (Initialized)
+ page_size : 4096
+ pages_added : 0 // pages that are hot_add-ed to the Guest
+ pages_onlined : 0 // pages that are added and then put online
+ // as available/used
+ pages_ballooned_out : 0 // pages the Host have taken back
+ vm_pages_commited : 795365 // total pages used by the Guest userspace
+ total_pages_commited : 977790 // total pages used by the Guest user+kernel
+ max_dynamic_page_count: 268435456 // maximum pages the Guest can have added
+ // via hot_add
+Hyper-V balloon driver changes the total RAM size reported by the Guest,
+thus the "InflatedTotal" counter will be non-zero in memory statistic
+reported during OOM or upon Alt+SysRQ+m.
+
+VMWare balloon
+---------------
+Balloon guest statistics output example::
+
+ # cat /sys/kernel/debug/vmmemctl
+ balloon capabilities: 0x1e
+ used capabilities: 0x6
+ is resetting: n
+ target: 0 pages
+ current: 0 pages
+ rateSleepAlloc: 2048 pages/sec
+ timer: 118
+ doorbell: 0
+ start: 1 ( 0 failed)
+ guestType: 1 ( 0 failed)
+ 2m-lock: 0 ( 0 failed)
+ lock: 0 ( 0 failed)
+ 2m-unlock: 0 ( 0 failed)
+ unlock: 0 ( 0 failed)
+ target: 118 ( 0 failed)
+ prim2mAlloc: 0 ( 0 failed)
+ primNoSleepAlloc: 0 ( 0 failed)
+ primCanSleepAlloc: 0 ( 0 failed)
+ prim2mFree: 0
+ primFree: 0
+ err2mAlloc: 0
+ errAlloc: 0
+ err2mFree: 0
+ errFree: 0
+ doorbellSet: 0
+ doorbellUnset: 1
+
+VMware balloon driver makes ballooned pages accounted as "used" in the
+Guest OS thus the "InflatedFree" counter will be non-zero in memory
+the statistic reported during OOM or upon Alt+SysRQ+m.
--
2.31.1
Alexander Atanasov <[email protected]> writes:
> Describe ballooning and how it works. Explain the two values
> and why they are there.
> Point the places where a user can see more balloon information and
> how each driver operates.
>
> Signed-off-by: Alexander Atanasov <[email protected]>
> ---
> Documentation/mm/balloon.rst | 138 +++++++++++++++++++++++++++++++++++
> 1 file changed, 138 insertions(+)
> create mode 100644 Documentation/mm/balloon.rst
When you add a new RST file, you also need to add it to index.rst so
that it becomes part of the docs build.
Thanks,
jon