Coalesced pio is based on coalesced mmio and can be used for some port
like rtc port, pci-host config port and so on.
Specially in case of rtc as coalesced pio, some versions of windows guest
access rtc frequently because of rtc as system tick. guest access rtc like
this: write register index to 0x70, then write or read data from 0x71.
writing 0x70 port is just as index and do nothing else. So we can use
coalesced pio to handle this scene to reduce VM-EXIT time.
When starting and closing a virtual machine, it will access pci-host config
port frequently. So setting these port as coalesced pio can reduce startup
and shutdown time.
without my patch, get the vm-exit time of accessing rtc 0x70 and piix 0xcf8
using perf tools: (guest OS : windows 7 64bit)
IO Port Access Samples Samples% Time% Min Time Max Time Avg time
0x70:POUT 86 30.99% 74.59% 9us 29us 10.75us (+- 3.41%)
0xcf8:POUT 1119 2.60% 2.12% 2.79us 56.83us 3.41us (+- 2.23%)
with my patch
IO Port Access Samples Samples% Time% Min Time Max Time Avg time
0x70:POUT 106 32.02% 29.47% 0us 10us 1.57us (+- 7.38%)
0xcf8:POUT 1065 1.67% 0.28% 0.41us 65.44us 0.66us (+- 10.55%)
Peng Hao (3):
kvm/x86 : add coalesced pio support
kvm/x86 : add document for coalesced mmio
kvm/x86 : add document for coalesced pio
Documentation/virtual/kvm/api.txt | 28 +++++++++++++++++++++++++++
include/uapi/linux/kvm.h | 11 +++++++++--
virt/kvm/coalesced_mmio.c | 12 +++++++++---
virt/kvm/kvm_main.c | 2 ++
4 files changed, 48 insertions(+), 5 deletions(-)
--
1.8.3.1
Signed-off-by: Peng Hao <[email protected]>
---
Documentation/virtual/kvm/api.txt | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 647f941..9615b9e 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -3676,6 +3676,30 @@ Returns: 0 on success, -1 on error
This copies the vcpu's kvm_nested_state struct from userspace to the kernel. For
the definition of struct kvm_nested_state, see KVM_GET_NESTED_STATE.
+4.116 KVM_(UN)REGISTER_COALESCED_MMIO
+
+Capability: KVM_CAP_COALESCED_MMIO
+Architectures: all
+Type: vm ioctl
+Parameters: struct kvm_coalesced_mmio_zone
+Returns: 0 on success, < 0 on error
+
+Coalesced mmio is a performance optimization that defers hardware
+register write emulation so that userspace exits are avoided. It is
+typically used to reduce the overhead of emulating frequently accessed
+hardware registers.
+
+When a hardware register is configured for coalesced mmio, write accesses
+do not exit to userspace and their value is recorded in a shared coalesced
+ring in the kernel.
+
+Coalesced mmio is applied to the following scenario typically.
+If a write access to a hardware register can be deferred, following a
+read/write access to another hardware register on the same device
+will cause the shared coalesced ring to be processed by userspace
+before emulating the current access. That will reduce the first write
+access to userspace.
+
5. The kvm_run structure
------------------------
--
1.8.3.1
Signed-off-by: Peng Hao <[email protected]>
---
Documentation/virtual/kvm/api.txt | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 9615b9e..d3a0497 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -3678,18 +3678,19 @@ the definition of struct kvm_nested_state, see KVM_GET_NESTED_STATE.
4.116 KVM_(UN)REGISTER_COALESCED_MMIO
-Capability: KVM_CAP_COALESCED_MMIO
+Capability: KVM_CAP_COALESCED_MMIO (for coalesced mmio)
+ KVM_CAP_COALESCED_PIO (for coalesced pio)
Architectures: all
Type: vm ioctl
Parameters: struct kvm_coalesced_mmio_zone
Returns: 0 on success, < 0 on error
-Coalesced mmio is a performance optimization that defers hardware
+Coalesced I/O is a performance optimization that defers hardware
register write emulation so that userspace exits are avoided. It is
typically used to reduce the overhead of emulating frequently accessed
hardware registers.
-When a hardware register is configured for coalesced mmio, write accesses
+When a hardware register is configured for coalesced I/O, write accesses
do not exit to userspace and their value is recorded in a shared coalesced
ring in the kernel.
@@ -3700,6 +3701,9 @@ will cause the shared coalesced ring to be processed by userspace
before emulating the current access. That will reduce the first write
access to userspace.
+Coalesced pio is based on coalesced mmio. There is little difference between
+coalesced mmio and pio except that coalesced pio is used for ioport.
+
5. The kvm_run structure
------------------------
--
1.8.3.1
add coalesced pio support.
Signed-off-by: Peng Hao <[email protected]>
---
include/uapi/linux/kvm.h | 11 +++++++++--
virt/kvm/coalesced_mmio.c | 12 +++++++++---
virt/kvm/kvm_main.c | 2 ++
3 files changed, 20 insertions(+), 5 deletions(-)
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 251be35..3c55ecd 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -420,13 +420,19 @@ struct kvm_run {
struct kvm_coalesced_mmio_zone {
__u64 addr;
__u32 size;
- __u32 pad;
+ union {
+ __u32 pad;
+ __u32 pio;
+ };
};
struct kvm_coalesced_mmio {
__u64 phys_addr;
__u32 len;
- __u32 pad;
+ union {
+ __u32 pad;
+ __u32 pio;
+ };
__u8 data[8];
};
@@ -953,6 +959,7 @@ struct kvm_ppc_resize_hpt {
#define KVM_CAP_NESTED_STATE 157
#define KVM_CAP_ARM_INJECT_SERROR_ESR 158
#define KVM_CAP_MSR_PLATFORM_INFO 159
+#define KVM_CAP_COALESCED_PIO 160
#ifdef KVM_CAP_IRQ_ROUTING
diff --git a/virt/kvm/coalesced_mmio.c b/virt/kvm/coalesced_mmio.c
index 9e65feb..3710342 100644
--- a/virt/kvm/coalesced_mmio.c
+++ b/virt/kvm/coalesced_mmio.c
@@ -83,6 +83,7 @@ static int coalesced_mmio_write(struct kvm_vcpu *vcpu,
ring->coalesced_mmio[ring->last].phys_addr = addr;
ring->coalesced_mmio[ring->last].len = len;
memcpy(ring->coalesced_mmio[ring->last].data, val, len);
+ ring->coalesced_mmio[ring->last].pio = dev->zone.pio;
smp_wmb();
ring->last = (ring->last + 1) % KVM_COALESCED_MMIO_MAX;
spin_unlock(&dev->kvm->ring_lock);
@@ -140,6 +141,9 @@ int kvm_vm_ioctl_register_coalesced_mmio(struct kvm *kvm,
int ret;
struct kvm_coalesced_mmio_dev *dev;
+ if (zone->pio != 1 && zone->pio != 0)
+ return -EINVAL;
+
dev = kzalloc(sizeof(struct kvm_coalesced_mmio_dev), GFP_KERNEL);
if (!dev)
return -ENOMEM;
@@ -149,8 +153,9 @@ int kvm_vm_ioctl_register_coalesced_mmio(struct kvm *kvm,
dev->zone = *zone;
mutex_lock(&kvm->slots_lock);
- ret = kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, zone->addr,
- zone->size, &dev->dev);
+ ret = kvm_io_bus_register_dev(kvm,
+ zone->pio ? KVM_PIO_BUS : KVM_MMIO_BUS,
+ zone->addr, zone->size, &dev->dev);
if (ret < 0)
goto out_free_dev;
list_add_tail(&dev->list, &kvm->coalesced_zones);
@@ -174,7 +179,8 @@ int kvm_vm_ioctl_unregister_coalesced_mmio(struct kvm *kvm,
list_for_each_entry_safe(dev, tmp, &kvm->coalesced_zones, list)
if (coalesced_mmio_in_range(dev, zone->addr, zone->size)) {
- kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS, &dev->dev);
+ kvm_io_bus_unregister_dev(kvm,
+ zone->pio ? KVM_PIO_BUS : KVM_MMIO_BUS, &dev->dev);
kvm_iodevice_destructor(&dev->dev);
}
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index f986e31f..37965f9 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2946,6 +2946,8 @@ static long kvm_vm_ioctl_check_extension_generic(struct kvm *kvm, long arg)
#ifdef CONFIG_KVM_MMIO
case KVM_CAP_COALESCED_MMIO:
return KVM_COALESCED_MMIO_PAGE_OFFSET;
+ case KVM_CAP_COALESCED_PIO:
+ return 1;
#endif
#ifdef CONFIG_HAVE_KVM_IRQ_ROUTING
case KVM_CAP_IRQ_ROUTING:
--
1.8.3.1
On 14/10/2018 01:09, Peng Hao wrote:
> Coalesced pio is based on coalesced mmio and can be used for some port
> like rtc port, pci-host config port and so on.
>
> Specially in case of rtc as coalesced pio, some versions of windows guest
> access rtc frequently because of rtc as system tick. guest access rtc like
> this: write register index to 0x70, then write or read data from 0x71.
> writing 0x70 port is just as index and do nothing else. So we can use
> coalesced pio to handle this scene to reduce VM-EXIT time.
>
> When starting and closing a virtual machine, it will access pci-host config
> port frequently. So setting these port as coalesced pio can reduce startup
> and shutdown time.
>
> without my patch, get the vm-exit time of accessing rtc 0x70 and piix 0xcf8
> using perf tools: (guest OS : windows 7 64bit)
> IO Port Access Samples Samples% Time% Min Time Max Time Avg time
> 0x70:POUT 86 30.99% 74.59% 9us 29us 10.75us (+- 3.41%)
> 0xcf8:POUT 1119 2.60% 2.12% 2.79us 56.83us 3.41us (+- 2.23%)
>
> with my patch
> IO Port Access Samples Samples% Time% Min Time Max Time Avg time
> 0x70:POUT 106 32.02% 29.47% 0us 10us 1.57us (+- 7.38%)
> 0xcf8:POUT 1065 1.67% 0.28% 0.41us 65.44us 0.66us (+- 10.55%)
>
>
> Peng Hao (3):
> kvm/x86 : add coalesced pio support
> kvm/x86 : add document for coalesced mmio
> kvm/x86 : add document for coalesced pio
>
> Documentation/virtual/kvm/api.txt | 28 +++++++++++++++++++++++++++
> include/uapi/linux/kvm.h | 11 +++++++++--
> virt/kvm/coalesced_mmio.c | 12 +++++++++---
> virt/kvm/kvm_main.c | 2 ++
> 4 files changed, 48 insertions(+), 5 deletions(-)
>
Queued, thanks (squashing 1 and 3 together).
Paolo
On Sun, Oct 14, 2018 at 07:09:54AM +0800, Peng Hao wrote:
> Coalesced pio is based on coalesced mmio and can be used for some port
> like rtc port, pci-host config port and so on.
>
> Specially in case of rtc as coalesced pio, some versions of windows guest
> access rtc frequently because of rtc as system tick. guest access rtc like
> this: write register index to 0x70, then write or read data from 0x71.
> writing 0x70 port is just as index and do nothing else. So we can use
> coalesced pio to handle this scene to reduce VM-EXIT time.
>
> When starting and closing a virtual machine, it will access pci-host config
> port frequently. So setting these port as coalesced pio can reduce startup
> and shutdown time.
>
> without my patch, get the vm-exit time of accessing rtc 0x70 and piix 0xcf8
> using perf tools: (guest OS : windows 7 64bit)
> IO Port Access Samples Samples% Time% Min Time Max Time Avg time
> 0x70:POUT 86 30.99% 74.59% 9us 29us 10.75us (+- 3.41%)
> 0xcf8:POUT 1119 2.60% 2.12% 2.79us 56.83us 3.41us (+- 2.23%)
>
> with my patch
> IO Port Access Samples Samples% Time% Min Time Max Time Avg time
> 0x70:POUT 106 32.02% 29.47% 0us 10us 1.57us (+- 7.38%)
> 0xcf8:POUT 1065 1.67% 0.28% 0.41us 65.44us 0.66us (+- 10.55%)
>
>
> Peng Hao (3):
> kvm/x86 : add coalesced pio support
> kvm/x86 : add document for coalesced mmio
> kvm/x86 : add document for coalesced pio
>
> Documentation/virtual/kvm/api.txt | 28 +++++++++++++++++++++++++++
> include/uapi/linux/kvm.h | 11 +++++++++--
> virt/kvm/coalesced_mmio.c | 12 +++++++++---
> virt/kvm/kvm_main.c | 2 ++
> 4 files changed, 48 insertions(+), 5 deletions(-)
Nice, thanks for documenting the API!