Hi all,
This is the v5 of Qemu patches and v5 makes below changes:
* Since this series patches add a new mechanism that let virtgpu and Qemu can negotiate
their reset behavior, and other guys hope me can improve this mechanism to virtio pci
level, so that other virtio devices can also benefit from it. So instead of adding
new feature flag VIRTIO_GPU_F_FREEZE_S3 only serves for virtgpu, v5 add a new parameter
named freeze_mode to struct VirtIODevice, when guest begin suspending, set freeze_mode
to VIRTIO_PCI_FREEZE_MODE_FREEZE_S3, and then all virtio devices can get this status,
and notice that guest is suspending, then they can change their reset behavior . See
the new commit "virtio-pci: Add freeze_mode case for virtio pci"
* The second commit is just for virtgpu, when freeze_mode is VIRTIO_PCI_FREEZE_MODE_FREEZE_S3,
prevent Qemu destroying render resources, so that the display can come back after resuming.
V5 of kernel patch:
https://lore.kernel.org/lkml/[email protected]/T/#t
The link to trace this issue:
https://gitlab.com/qemu-project/qemu/-/issues/1860
Best regards,
Jiqian Chen
v4:
Hi all,
Thanks for Gerd Hoffmann's advice. V4 makes below changes:
* Use enum for freeze mode, so this can be extended with more
modes in the future.
* Rename functions and paratemers with "_S3" postfix.
And no functional changes.
Link:
https://lore.kernel.org/qemu-devel/[email protected]/
No v4 patch on kernel side.
v3:
Hi all,
Thanks for Michael S. Tsirkin's advice. V3 makes below changes:
* Remove changes in file include/standard-headers/linux/virtio_gpu.h
I am not supposed to edit this file and it will be imported after
the patches of linux kernel was merged.
Link:
https://lore.kernel.org/qemu-devel/[email protected]/T/#t
V3 of kernel patch:
https://lore.kernel.org/lkml/[email protected]/T/#t
v2:
makes below changes:
* Change VIRTIO_CPU_CMD_STATUS_FREEZING to 0x0400 (<0x1000)
* Add virtio_gpu_device_unrealize to destroy resources to solve
potential memory leak problem. This also needs hot-plug support.
* Add a new feature flag VIRTIO_GPU_F_FREEZING, so that guest and
host can negotiate whenever freezing is supported or not.
Link:
https://lore.kernel.org/qemu-devel/[email protected]/T/#t
V2 of kernel patch:
https://lore.kernel.org/lkml/[email protected]/T/#t
v1:
Hi all,
I am working to implement virtgpu S3 function on Xen.
Currently on Xen, if we start a guest who enables virtgpu, and then run "echo mem >
/sys/power/state" to suspend guest. And run "sudo xl trigger <guest id> s3resume" to
resume guest. We can find that the guest kernel comes back, but the display doesn't.
It just shown a black screen.
Through reading codes, I founded that when guest was during suspending, it called
into Qemu to call virtio_gpu_gl_reset. In virtio_gpu_gl_reset, it destroyed all
resources and reset renderer. This made the display gone after guest resumed.
I think we should keep resources or prevent they being destroyed when guest is
suspending. So, I add a new status named freezing to virtgpu, and add a new ctrl
message VIRTIO_GPU_CMD_STATUS_FREEZING to get notification from guest. If freezing
is set to true, and then Qemu will realize that guest is suspending, it will not
destroy resources and will not reset renderer. If freezing is set to false, Qemu
will do its origin actions, and has no other impaction.
And now, display can come back and applications can continue their status after guest
resumes.
Link:
https://lore.kernel.org/qemu-devel/[email protected]/
V1 of kernel patch:
https://lore.kernel.org/lkml/[email protected]/
Jiqian Chen (2):
virtio-pci: Add freeze_mode case for virtio pci
virtgpu: do not destroy resources when guest does S3
hw/display/virtio-gpu-gl.c | 9 ++++++++-
hw/display/virtio-gpu.c | 12 ++++++++++--
hw/virtio/virtio-pci.c | 5 +++++
hw/virtio/virtio.c | 1 +
include/hw/virtio/virtio.h | 2 ++
5 files changed, 26 insertions(+), 3 deletions(-)
--
2.34.1
When guest vm do S3, Qemu will reset and clear some things of virtio
devices, but guest can't aware that, so that may cause some problems.
For excample, Qemu calls virtio_reset->virtio_gpu_gl_reset, that will
destroy render resources of virtio-gpu. As a result, after guest resume,
the display can't come back and we only saw a black screen. Due to guest
can't re-create all the resources, so we need to let Qemu not to destroy
them when S3.
For above purpose, this patch add a new parameter named freeze_mode to
struct VirtIODevice, and when guest suspends, guest can write freeze_mode
to be FREEZE_S3, so that virtio devices can change their reset behavior
on Qemu side according to that mode.
Signed-off-by: Jiqian Chen <[email protected]>
---
hw/virtio/virtio-pci.c | 5 +++++
hw/virtio/virtio.c | 1 +
include/hw/virtio/virtio.h | 2 ++
3 files changed, 8 insertions(+)
diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c
index edbc0daa18..7a3cca79b4 100644
--- a/hw/virtio/virtio-pci.c
+++ b/hw/virtio/virtio-pci.c
@@ -1651,6 +1651,11 @@ static void virtio_pci_common_write(void *opaque, hwaddr addr,
proxy->vqs[vdev->queue_sel].enabled = 0;
}
break;
+ case VIRTIO_PCI_COMMON_F_MODE:
+ virtio_pci_freeze_mode_t freeze_mode = (virtio_pci_freeze_mode_t)val;
+ if ((1 << freeze_mode) & VIRTIO_PCI_FREEZE_MODE_MASK)
+ vdev->freeze_mode = freeze_mode;
+ break;
default:
break;
}
diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 969c25f4cf..4278cedef4 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -3201,6 +3201,7 @@ void virtio_init(VirtIODevice *vdev, uint16_t device_id, size_t config_size)
vdev->vhost_started = false;
vdev->device_id = device_id;
vdev->status = 0;
+ vdev->freeze_mode = VIRTIO_PCI_FREEZE_MODE_UNFREEZE;
qatomic_set(&vdev->isr, 0);
vdev->queue_sel = 0;
vdev->config_vector = VIRTIO_NO_VECTOR;
diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h
index c8f72850bc..f8bfd9da28 100644
--- a/include/hw/virtio/virtio.h
+++ b/include/hw/virtio/virtio.h
@@ -21,6 +21,7 @@
#include "qemu/event_notifier.h"
#include "standard-headers/linux/virtio_config.h"
#include "standard-headers/linux/virtio_ring.h"
+#include "standard-headers/linux/virtio_pci.h"
#include "qom/object.h"
/*
@@ -106,6 +107,7 @@ struct VirtIODevice
DeviceState parent_obj;
const char *name;
uint8_t status;
+ virtio_pci_freeze_mode_t freeze_mode;
uint8_t isr;
uint16_t queue_sel;
/**
--
2.34.1
After guest VM resumed, you will get a black screen, and the display
can't come back. It is because when guest did resuming, it called
into qemu to call virtio_gpu_gl_reset. In that function, it destroyed
resources created by command VIRTIO_GPU_CMD_RESOURCE_CREATE_*, which
were used for display. As a result, guest's screen can't come back to
the time when it was suspended.
So when freeze_mode is set FREEZE_S3 by guest, we can know that guest
is doing S3, and we can prevent Qemu to destroy the resources.
Signed-off-by: Jiqian Chen <[email protected]>
---
hw/display/virtio-gpu-gl.c | 9 ++++++++-
hw/display/virtio-gpu.c | 12 ++++++++++--
2 files changed, 18 insertions(+), 3 deletions(-)
diff --git a/hw/display/virtio-gpu-gl.c b/hw/display/virtio-gpu-gl.c
index e06be60dfb..2519dc12ff 100644
--- a/hw/display/virtio-gpu-gl.c
+++ b/hw/display/virtio-gpu-gl.c
@@ -100,7 +100,14 @@ static void virtio_gpu_gl_reset(VirtIODevice *vdev)
*/
if (gl->renderer_inited && !gl->renderer_reset) {
virtio_gpu_virgl_reset_scanout(g);
- gl->renderer_reset = true;
+ /*
+ * If guest is suspending, we shouldn't reset renderer,
+ * otherwise, the display can't come back to the time when
+ * it was suspended after guest resumed.
+ */
+ if (vdev->freeze_mode != VIRTIO_PCI_FREEZE_MODE_FREEZE_S3) {
+ gl->renderer_reset = true;
+ }
}
}
diff --git a/hw/display/virtio-gpu.c b/hw/display/virtio-gpu.c
index 93857ad523..d363b886dc 100644
--- a/hw/display/virtio-gpu.c
+++ b/hw/display/virtio-gpu.c
@@ -1412,11 +1412,19 @@ static void virtio_gpu_device_unrealize(DeviceState *qdev)
static void virtio_gpu_reset_bh(void *opaque)
{
VirtIOGPU *g = VIRTIO_GPU(opaque);
+ VirtIODevice *vdev = &g->parent_obj.parent_obj;
struct virtio_gpu_simple_resource *res, *tmp;
int i = 0;
- QTAILQ_FOREACH_SAFE(res, &g->reslist, next, tmp) {
- virtio_gpu_resource_destroy(g, res);
+ /*
+ * If guest is suspending, we shouldn't destroy resources,
+ * otherwise, the display can't come back to the time when
+ * it was suspended after guest resumed.
+ */
+ if (vdev->freeze_mode != VIRTIO_PCI_FREEZE_MODE_FREEZE_S3) {
+ QTAILQ_FOREACH_SAFE(res, &g->reslist, next, tmp) {
+ virtio_gpu_resource_destroy(g, res);
+ }
}
for (i = 0; i < g->parent_obj.conf.max_outputs; i++) {
--
2.34.1