This series allows to optimize the deactivation of virtual interrupts
associated to a vfio platform device IRQ. Let's call this optimization:
ARM IRQ forwarding.
Without that optimization the deactivation of the physical IRQ is done
by the host and the deactivation of the virtual IRQ, by the guest,
is trapped by KVM with a maintenance IRQ. With ARM IRQ forwarding, the
physical IRQ is automatically deactivated by the GIC interrupt controller
when the guest deactivates the virtual IRQ.
The series enables the auto-negotiation of this optimization, on ARM.
The auto-negotiation is coordinated by the IRQ bypass manager. The VFIO
platform driver and KVM/irqfd cooperate to turn that feature on.
The series implements the setup of IRQ forwarding and adapt the existing
VGIC state machine so that it works.
When the IRQ is forwarded, the VFIO platform driver does not need to
mask the physical IRQ anymore before signaling the eventfd. Indeed
genirq lowers the running priority, enabling other physical IRQ to hit
except that one.
The injection still is based on irqfd triggering. The only impact on
irqfd process is the resamplefd is not triggered anymore for level
sensitive IRQs since deactivation is not trapped by KVM.
More backgroung on ARM IRQ forwarding in the text below and at
http://www.linux-kvm.org/images/a/a8/01x04-ARMdevice.pdf.
This was tested on:
- Calxeda Midway, assigning the xgmac main IRQ.
- AMD Seattle, where all XGBE interrupts are assigned:
- 2 level sensitive and 4 DMA edge sensitive IRQs
applies on top of:
KVM: arm/arm64: leave the LR active state on GICD_ICENABLERn access
All those pieces can be found at:
https://git.linaro.org/people/eric.auger/linux.git/shortlog/refs/heads/v4.4-rc1-forward-v4
History:
v3 -> v4:
- dependencies now are fixed (IRQ bypass manager, guest synchronous
halt/resume, VGIC architected timer)
- only applies on top of:
KVM: arm/arm64: leave the LR active state on GICD_ICENABLERn access
- Rebase on vgic series related to forwarded shared interrupts, mainly
[PATCH v2 0/8] Rework architected timer and forwarded IRQs handling &
[PATCH 0/2] arm/arm64: KVM: Fix architected timer issues
- took into account Alex's comment on VFIO part:
- vfio_platform_set_forwarded replacing vfio_platform_set_automasked
- automasked flag not updated
- vfio_platform_is_active renamed into vfio_platform_irq_is_active
- handle returned value for vfio_platform_irq_is_active
- remove WARN_ON on irq_bypass_register_producer failure
- took into account Christoffer's comments on vgic part
- clarify comments
- clarify edge-sensitive handling and validate_injection of edge
mapped unshared IRQ with level == 0
- use actual vcpu_id instead of VCPU 0
- revisit unset_forward implement using unqueue
- moved KVM: arm/arm64: select IRQ_BYPASS_MANAGER in this series
- add support for clear active and clear pending
v2 (RFC) -> v3(PATCH):
- all dependencies now have a patch status
- we dropped the producer active boolean exchanged between the
VFIO producer and irqfd arm consumer. As a consequence, on
unforward, if the IRQ is active, this latter is deactivated
without VFIO-masking it. So we do not exactly come back to the
exact state where we would be in unforwarded state. A new
physical IRQ can hit while the previous virtual IRQ is under
treatment. Its injection in the guest may be rejected thanks
to the VGIC state machine. This IRQ will be lost but I don't
think this is a severe issue. In case no new IRQ hits, the
guest deactivation of the virtual IRQ will trigger the resamplefd
which will VFIO unmask the non-masked IRQ. This also has no
consequence.
- VFIO platform driver consumer_add now can fail. It rejects the
transition for forwarding state in case the IRQ is active
- the series is rebased on new irq_vcpu_affinity series
- no dependency anymore on "chip/vgic adaptations for forwarded irq"
which was partially integrated into Marc's series. A fix is still
needed through.
- Guest synchronous halt/resume patch re-integrated into this series
- integrate a new patch file coming mixing
[PATCH v4 11/11] KVM: arm/arm64: vgic: Allow HW interrupts for
non-shared devices &
[RFC v2 2/4] KVM: arm: vgic: fix state machine for forwarded IRQ
v1 -> v2:
- irq bypass manager and irqfd consumer moved in a separate patch
- kvm_arm_[halt,resume]_guest moved in a separate patch
- remove VFIO external functions since we do not need them anymore
- apply container_of strategy advised by Paolo. Only active field
remains and discussions will tell whether we get rid of it.
- renamed kvm_arch functions
- kvm-vfio v6 -> RFC v1 based on IRQ bypass manager
see previous history in https://lkml.org/lkml/2015/4/13/353).
Best Regards
Eric
Eric Auger (12):
KVM: arm/arm64: select IRQ_BYPASS_MANAGER
VFIO: platform: registration of a dummy IRQ bypass producer
VFIO: platform: test forwarded state when selecting the IRQ handler
VFIO: platform: single handler using function pointer
VFIO: platform: add vfio_platform_set_forwarded
VFIO: platform: add vfio_platform_irq_is_active
VFIO: platform: add irq bypass producer management
KVM: arm/arm64: vgic: support irqfd injection of a mapped IRQ
KVM: arm/arm64: vgic: forwarding control
KVM: arm/arm64: vgic: implement clear active for non shared mapped IRQ
KVM: arm/arm64: vgic: implement clear pending for non shared mapped
IRQ
KVM: arm/arm64: implement IRQ bypass consumer functions
Marc Zyngier (1):
KVM: arm/arm64: vgic: adapt state machine for non shared mapped
interrupts
arch/arm/kvm/Kconfig | 3 +
arch/arm/kvm/arm.c | 37 +++++
arch/arm64/kvm/Kconfig | 3 +
drivers/vfio/platform/vfio_platform_irq.c | 120 +++++++++++++-
drivers/vfio/platform/vfio_platform_private.h | 4 +
include/kvm/arm_vgic.h | 8 +-
virt/kvm/arm/arch_timer.c | 2 +-
virt/kvm/arm/vgic.c | 221 ++++++++++++++++++++++++--
8 files changed, 376 insertions(+), 22 deletions(-)
--
1.9.1
Select IRQ_BYPASS_MANAGER when CONFIG_KVM is set
Signed-off-by: Eric Auger <[email protected]>
---
arch/arm/kvm/Kconfig | 2 ++
arch/arm64/kvm/Kconfig | 2 ++
2 files changed, 4 insertions(+)
diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 95a0005..73d7201 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -3,6 +3,7 @@
#
source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
menuconfig VIRTUALIZATION
bool "Virtualization"
@@ -32,6 +33,7 @@ config KVM
select KVM_VFIO
select HAVE_KVM_EVENTFD
select HAVE_KVM_IRQFD
+ select IRQ_BYPASS_MANAGER
depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
---help---
Support hosting virtualized guest machines.
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index a5272c0..64f460d 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -3,6 +3,7 @@
#
source "virt/kvm/Kconfig"
+source "virt/lib/Kconfig"
menuconfig VIRTUALIZATION
bool "Virtualization"
@@ -36,6 +37,7 @@ config KVM
select HAVE_KVM_EVENTFD
select HAVE_KVM_IRQFD
select KVM_ARM_VGIC_V3
+ select IRQ_BYPASS_MANAGER
---help---
Support hosting virtualized guest machines.
We don't support KVM with 16K page tables yet, due to the multiple
--
1.9.1
Register a dummy producer with void callbacks
Signed-off-by: Eric Auger <[email protected]>
---
v3 -> v3:
- replace WARN_ON by pr_info() in case irq_bypass_register_producer
fails
v2 -> v3:
- rename vfio_platform_irq_bypass_resume into *_start
---
drivers/vfio/platform/vfio_platform_irq.c | 34 +++++++++++++++++++++++++++
drivers/vfio/platform/vfio_platform_private.h | 2 ++
2 files changed, 36 insertions(+)
diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c
index 46d4750..ab7658a 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -20,6 +20,7 @@
#include <linux/types.h>
#include <linux/vfio.h>
#include <linux/irq.h>
+#include <linux/irqbypass.h>
#include "vfio_platform_private.h"
@@ -177,6 +178,27 @@ static irqreturn_t vfio_irq_handler(int irq, void *dev_id)
return IRQ_HANDLED;
}
+static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
+{
+}
+
+static void vfio_platform_irq_bypass_start(struct irq_bypass_producer *prod)
+{
+}
+
+static int vfio_platform_irq_bypass_add_consumer(
+ struct irq_bypass_producer *prod,
+ struct irq_bypass_consumer *cons)
+{
+ return 0;
+}
+
+static void vfio_platform_irq_bypass_del_consumer(
+ struct irq_bypass_producer *prod,
+ struct irq_bypass_consumer *cons)
+{
+}
+
static int vfio_set_trigger(struct vfio_platform_device *vdev, int index,
int fd, irq_handler_t handler)
{
@@ -187,6 +209,7 @@ static int vfio_set_trigger(struct vfio_platform_device *vdev, int index,
if (irq->trigger) {
irq_clear_status_flags(irq->hwirq, IRQ_NOAUTOEN);
free_irq(irq->hwirq, irq);
+ irq_bypass_unregister_producer(&irq->producer);
kfree(irq->name);
eventfd_ctx_put(irq->trigger);
irq->trigger = NULL;
@@ -217,6 +240,17 @@ static int vfio_set_trigger(struct vfio_platform_device *vdev, int index,
return ret;
}
+ irq->producer.token = (void *)trigger;
+ irq->producer.irq = irq->hwirq;
+ irq->producer.add_consumer = vfio_platform_irq_bypass_add_consumer;
+ irq->producer.del_consumer = vfio_platform_irq_bypass_del_consumer;
+ irq->producer.stop = vfio_platform_irq_bypass_stop;
+ irq->producer.start = vfio_platform_irq_bypass_start;
+ ret = irq_bypass_register_producer(&irq->producer);
+ if (ret)
+ pr_info("irq bypass producer (token %p) registration fails: %d\n",
+ irq->producer.token, ret);
+
if (!irq->masked)
enable_irq(irq->hwirq);
diff --git a/drivers/vfio/platform/vfio_platform_private.h b/drivers/vfio/platform/vfio_platform_private.h
index 42816dd..979c191 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -17,6 +17,7 @@
#include <linux/types.h>
#include <linux/interrupt.h>
+#include <linux/irqbypass.h>
#define VFIO_PLATFORM_OFFSET_SHIFT 40
#define VFIO_PLATFORM_OFFSET_MASK (((u64)(1) << VFIO_PLATFORM_OFFSET_SHIFT) - 1)
@@ -37,6 +38,7 @@ struct vfio_platform_irq {
spinlock_t lock;
struct virqfd *unmask;
struct virqfd *mask;
+ struct irq_bypass_producer producer;
};
struct vfio_platform_region {
--
1.9.1
Add a new forwarded flag in vfio_platform_irq. In case the IRQ
is forwarded, the VFIO platform IRQ handler does not need to
disable the IRQ anymore.
When setting the IRQ handler we now also test the forwarded state. In
case the IRQ is forwarded we select the vfio_irq_handler.
Signed-off-by: Eric Auger <[email protected]>
---
v2:
- add a new forwarded flag and do not use irqd_irq_forwarded anymore
---
drivers/vfio/platform/vfio_platform_irq.c | 3 ++-
drivers/vfio/platform/vfio_platform_private.h | 1 +
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c
index ab7658a..d4462fd 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -265,7 +265,8 @@ static int vfio_platform_set_irq_trigger(struct vfio_platform_device *vdev,
struct vfio_platform_irq *irq = &vdev->irqs[index];
irq_handler_t handler;
- if (vdev->irqs[index].flags & VFIO_IRQ_INFO_AUTOMASKED)
+ if (vdev->irqs[index].flags & VFIO_IRQ_INFO_AUTOMASKED &&
+ !irq->forwarded)
handler = vfio_automasked_irq_handler;
else
handler = vfio_irq_handler;
diff --git a/drivers/vfio/platform/vfio_platform_private.h b/drivers/vfio/platform/vfio_platform_private.h
index 979c191..55e8043 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -39,6 +39,7 @@ struct vfio_platform_irq {
struct virqfd *unmask;
struct virqfd *mask;
struct irq_bypass_producer producer;
+ bool forwarded;
};
struct vfio_platform_region {
--
1.9.1
A single handler now is registered whatever the use case: automasked
or not. A function pointer is set according to the wished behavior
and the handler calls this function.
The irq lock is taken/released in the root handler. eventfd_signal can
be called in regions not allowed to sleep.
Signed-off-by: Eric Auger <[email protected]>
---
v4: creation
---
drivers/vfio/platform/vfio_platform_irq.c | 21 +++++++++++++++------
drivers/vfio/platform/vfio_platform_private.h | 1 +
2 files changed, 16 insertions(+), 6 deletions(-)
diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c
index d4462fd..c58e0ad 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -148,11 +148,8 @@ static int vfio_platform_set_irq_unmask(struct vfio_platform_device *vdev,
static irqreturn_t vfio_automasked_irq_handler(int irq, void *dev_id)
{
struct vfio_platform_irq *irq_ctx = dev_id;
- unsigned long flags;
int ret = IRQ_NONE;
- spin_lock_irqsave(&irq_ctx->lock, flags);
-
if (!irq_ctx->masked) {
ret = IRQ_HANDLED;
@@ -161,8 +158,6 @@ static irqreturn_t vfio_automasked_irq_handler(int irq, void *dev_id)
irq_ctx->masked = true;
}
- spin_unlock_irqrestore(&irq_ctx->lock, flags);
-
if (ret == IRQ_HANDLED)
eventfd_signal(irq_ctx->trigger, 1);
@@ -178,6 +173,19 @@ static irqreturn_t vfio_irq_handler(int irq, void *dev_id)
return IRQ_HANDLED;
}
+static irqreturn_t vfio_handler(int irq, void *dev_id)
+{
+ struct vfio_platform_irq *irq_ctx = dev_id;
+ unsigned long flags;
+ irqreturn_t ret;
+
+ spin_lock_irqsave(&irq_ctx->lock, flags);
+ ret = irq_ctx->handler(irq, dev_id);
+ spin_unlock_irqrestore(&irq_ctx->lock, flags);
+
+ return ret;
+}
+
static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
{
}
@@ -230,9 +238,10 @@ static int vfio_set_trigger(struct vfio_platform_device *vdev, int index,
}
irq->trigger = trigger;
+ irq->handler = handler;
irq_set_status_flags(irq->hwirq, IRQ_NOAUTOEN);
- ret = request_irq(irq->hwirq, handler, 0, irq->name, irq);
+ ret = request_irq(irq->hwirq, vfio_handler, 0, irq->name, irq);
if (ret) {
kfree(irq->name);
eventfd_ctx_put(trigger);
diff --git a/drivers/vfio/platform/vfio_platform_private.h b/drivers/vfio/platform/vfio_platform_private.h
index 55e8043..8a8c050 100644
--- a/drivers/vfio/platform/vfio_platform_private.h
+++ b/drivers/vfio/platform/vfio_platform_private.h
@@ -40,6 +40,7 @@ struct vfio_platform_irq {
struct virqfd *mask;
struct irq_bypass_producer producer;
bool forwarded;
+ irqreturn_t (*handler)(int irq, void *dev_id);
};
struct vfio_platform_region {
--
1.9.1
This function allows to change the forwarded mode and selects
the IRQ handler accordingly.
Signed-off-by: Eric Auger <[email protected]>
---
v3 -> v3:
- renamed vfio_platform_set_automasked into
vfio_platform_set_forwarded
- do not change VFIO_IRQ_INFO_AUTOMASKED setting when turning
forwarding on/off
v1 -> v2:
- set forwarded flag
---
drivers/vfio/platform/vfio_platform_irq.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c
index c58e0ad..257200b 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -186,6 +186,23 @@ static irqreturn_t vfio_handler(int irq, void *dev_id)
return ret;
}
+static int vfio_platform_set_forwarded(struct vfio_platform_irq *irq,
+ bool forwarded)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&irq->lock, flags);
+ irq->forwarded = forwarded;
+
+ if (!forwarded && (irq->flags & VFIO_IRQ_INFO_AUTOMASKED))
+ irq->handler = vfio_automasked_irq_handler;
+ else
+ irq->handler = vfio_irq_handler;
+
+ spin_unlock_irqrestore(&irq->lock, flags);
+ return 0;
+}
+
static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
{
}
--
1.9.1
This function returns whether the IRQ is active at irqchip level or
VFIO masked. If either is true, the IRQ is considered active.
Currently there is no way to differentiate userspace masked IRQ from
automasked IRQ. There might be false detection of activity. However
it is currently acceptable to have false detection.
Signed-off-by: Eric Auger <[email protected]>
---
v3 -> v4:
- rename vfio_platform_is_active into vfio_platform_irq_is_active
- remove BUG_ON and return the error if irq_get_irqchip_state fails
---
drivers/vfio/platform/vfio_platform_irq.c | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c
index 257200b..cb7d5e9 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -203,6 +203,25 @@ static int vfio_platform_set_forwarded(struct vfio_platform_irq *irq,
return 0;
}
+static int vfio_platform_irq_is_active(struct vfio_platform_irq *irq)
+{
+ unsigned long flags;
+ bool active;
+ int ret;
+
+ spin_lock_irqsave(&irq->lock, flags);
+
+ ret = irq_get_irqchip_state(irq->hwirq, IRQCHIP_STATE_ACTIVE, &active);
+ if (ret)
+ goto out;
+
+ ret = active || irq->masked;
+
+out:
+ spin_unlock_irqrestore(&irq->lock, flags);
+ return ret;
+}
+
static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
{
}
--
1.9.1
This patch populates the IRQ bypass callacks:
- stop/start producer simply consist in disabling/enabling the host irq
- add/del consumer: basically set the automasked flag to false/true
Signed-off-by: Eric Auger <[email protected]>
---
v3 -> v4:
- use vfio_platform_set_forwarded in place of vfio_platform_set_automasked
- use vfio_platform_irq_is_active and its returned value
- reword comments related to auto/explicit VFIO masking
v2 -> v3:
- vfio_platform_irq_bypass_add_consumer now returns an error in case
the IRQ is recognized as active
- active boolean not set anymore
- do not VFIO mask the IRQ anymore on unforward
v1 -> v2:
- device handle caching in vfio_platform_device is introduced in a
separate patch
- use container_of
---
drivers/vfio/platform/vfio_platform_irq.c | 28 +++++++++++++++++++++++++++-
1 file changed, 27 insertions(+), 1 deletion(-)
diff --git a/drivers/vfio/platform/vfio_platform_irq.c b/drivers/vfio/platform/vfio_platform_irq.c
index cb7d5e9..e486c32 100644
--- a/drivers/vfio/platform/vfio_platform_irq.c
+++ b/drivers/vfio/platform/vfio_platform_irq.c
@@ -224,23 +224,49 @@ out:
static void vfio_platform_irq_bypass_stop(struct irq_bypass_producer *prod)
{
+ disable_irq(prod->irq);
}
static void vfio_platform_irq_bypass_start(struct irq_bypass_producer *prod)
{
+ enable_irq(prod->irq);
}
static int vfio_platform_irq_bypass_add_consumer(
struct irq_bypass_producer *prod,
struct irq_bypass_consumer *cons)
{
- return 0;
+ struct vfio_platform_irq *irq =
+ container_of(prod, struct vfio_platform_irq, producer);
+ int ret;
+
+ /*
+ * if the IRQ is active at irqchip level or masked by VFIO
+ * this means the host IRQ may be under injection and
+ * it is not safe to change the forwarding state at that time.
+ * It is not possible to discriminate VFIO explicit masking
+ * from VFIO auto-masking, leading to possible false detection of
+ * IRQ activity. It is up to the user-space to avoid VFIO explicit
+ * masking situation. In case of failure, no retry is attempted and
+ * bypass setup simply fails.
+ */
+ ret = vfio_platform_irq_is_active(irq);
+ if (ret < 0)
+ return ret;
+ else if (ret == 1)
+ return -EAGAIN;
+
+ return vfio_platform_set_forwarded(irq, true);
}
static void vfio_platform_irq_bypass_del_consumer(
struct irq_bypass_producer *prod,
struct irq_bypass_consumer *cons)
{
+ struct vfio_platform_irq *irq =
+ container_of(prod, struct vfio_platform_irq, producer);
+
+ vfio_platform_set_forwarded(irq, false);
}
static int vfio_set_trigger(struct vfio_platform_device *vdev, int index,
--
1.9.1
From: Marc Zyngier <[email protected]>
So far, the only user of the mapped interrupt facility was the timer:
the physical distributor active state needed to be context-switched
for each vcpu, as the device is shared across all vcpus.
This patch allows to indicate whether a mapped IRQ originates from a
device shared between several VMs (typically the architected timer) or
from a device assigned to a single VM.
A new "shared" flag is added to irq_phys_map and passed to the
mapping function.
the VGIC state machine is adapted to support the non shared mapped IRQs:
- only can be sampled when it is pending
- when queueing the IRQ (programming the LR), the pending state is
removed as for edge sensitive IRQs
- queued state is not modelled. Level state is not modelled
- its injection with high level always is valid since steming from the HW.
Signed-off-by: Eric Auger <[email protected]>
Signed-off-by: Marc Zyngier <[email protected]>
---
v3 -> v4:
- reword the patch title
- rebase on [PATCH v2 0/8] Rework architected timer and forwarded IRQs
handling
- renamed shared_hw into non_shared_mapped_irq;
- handle edge-sensitive unshared mapped IRQs
- vgic_validate_injection rejects mapped unshared edge with
level == 0
---
include/kvm/arm_vgic.h | 3 ++-
virt/kvm/arm/arch_timer.c | 2 +-
virt/kvm/arm/vgic.c | 37 ++++++++++++++++++++++++++++---------
3 files changed, 31 insertions(+), 11 deletions(-)
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 9c747cb..9bf6a30 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -158,6 +158,7 @@ struct irq_phys_map {
u32 virt_irq;
u32 phys_irq;
u32 irq;
+ bool shared;
};
struct irq_phys_map_entry {
@@ -344,7 +345,7 @@ void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
int kvm_vgic_vcpu_active_irq(struct kvm_vcpu *vcpu);
struct irq_phys_map *kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu,
- int virt_irq, int irq);
+ int virt_irq, int irq, bool shared);
int kvm_vgic_unmap_phys_irq(struct kvm_vcpu *vcpu, struct irq_phys_map *map);
#define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
diff --git a/virt/kvm/arm/arch_timer.c b/virt/kvm/arm/arch_timer.c
index 21a0ab2..9eea751 100644
--- a/virt/kvm/arm/arch_timer.c
+++ b/virt/kvm/arm/arch_timer.c
@@ -289,7 +289,7 @@ int kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
* Tell the VGIC that the virtual interrupt is tied to a
* physical interrupt. We do that once per VCPU.
*/
- map = kvm_vgic_map_phys_irq(vcpu, irq->irq, host_vtimer_irq);
+ map = kvm_vgic_map_phys_irq(vcpu, irq->irq, host_vtimer_irq, true);
if (WARN_ON(IS_ERR(map)))
return PTR_ERR(map);
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index bc30d93..dba8eb6 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -410,7 +410,11 @@ void vgic_cpu_irq_clear(struct kvm_vcpu *vcpu, int irq)
static bool vgic_can_sample_irq(struct kvm_vcpu *vcpu, int irq)
{
- return !vgic_irq_is_queued(vcpu, irq);
+ struct irq_phys_map *map = vgic_irq_map_search(vcpu, irq);
+ bool non_shared_mapped_irq = map && !map->shared;
+
+ return !vgic_irq_is_queued(vcpu, irq) ||
+ (non_shared_mapped_irq && vgic_dist_irq_is_pending(vcpu, irq));
}
/**
@@ -1205,11 +1209,14 @@ bool vgic_queue_irq(struct kvm_vcpu *vcpu, u8 sgi_source_id, int irq)
static bool vgic_queue_hwirq(struct kvm_vcpu *vcpu, int irq)
{
+ struct irq_phys_map *map = vgic_irq_map_search(vcpu, irq);
+ bool non_shared_mapped_irq = map && !map->shared;
+
if (!vgic_can_sample_irq(vcpu, irq))
return true; /* level interrupt, already queued */
if (vgic_queue_irq(vcpu, 0, irq)) {
- if (vgic_irq_is_edge(vcpu, irq)) {
+ if (vgic_irq_is_edge(vcpu, irq) || non_shared_mapped_irq) {
vgic_dist_irq_clear_pending(vcpu, irq);
vgic_cpu_irq_clear(vcpu, irq);
} else {
@@ -1292,6 +1299,8 @@ static int process_queued_irq(struct kvm_vcpu *vcpu,
int lr, struct vgic_lr vlr)
{
int pending = 0;
+ struct irq_phys_map *map = vgic_irq_map_search(vcpu, vlr.irq);
+ bool non_shared_mapped_irq = map && !map->shared;
/*
* If the IRQ was EOIed (called from vgic_process_maintenance) or it
@@ -1312,8 +1321,7 @@ static int process_queued_irq(struct kvm_vcpu *vcpu,
vgic_irq_clear_queued(vcpu, vlr.irq);
/* Any additional pending interrupt? */
- if (vgic_irq_is_edge(vcpu, vlr.irq)) {
- BUG_ON(!(vlr.state & LR_HW));
+ if (vgic_irq_is_edge(vcpu, vlr.irq) || non_shared_mapped_irq) {
pending = vgic_dist_irq_is_pending(vcpu, vlr.irq);
} else {
if (vgic_dist_irq_get_level(vcpu, vlr.irq)) {
@@ -1506,18 +1514,23 @@ void vgic_kick_vcpus(struct kvm *kvm)
}
}
-static int vgic_validate_injection(struct kvm_vcpu *vcpu, int irq, int level)
+static int vgic_validate_injection(struct kvm_vcpu *vcpu,
+ struct irq_phys_map *map,
+ int irq, int level)
{
int edge_triggered = vgic_irq_is_edge(vcpu, irq);
/*
* Only inject an interrupt if:
* - edge triggered and we have a rising edge
- * - level triggered and we change level
+ * - level triggered and we change level (except for
+ * mapped unshared IRQs where level is not modelled)
*/
if (edge_triggered) {
int state = vgic_dist_irq_is_pending(vcpu, irq);
return level > state;
+ } else if (map && !map->shared) {
+ return true;
} else {
int state = vgic_dist_irq_get_level(vcpu, irq);
return level != state;
@@ -1545,7 +1558,7 @@ static int vgic_update_irq_pending(struct kvm *kvm, int cpuid,
edge_triggered = vgic_irq_is_edge(vcpu, irq_num);
level_triggered = !edge_triggered;
- if (!vgic_validate_injection(vcpu, irq_num, level)) {
+ if (!vgic_validate_injection(vcpu, map, irq_num, level)) {
ret = false;
goto out;
}
@@ -1718,16 +1731,20 @@ static struct list_head *vgic_get_irq_phys_map_list(struct kvm_vcpu *vcpu,
* @vcpu: The VCPU pointer
* @virt_irq: The virtual irq number
* @irq: The Linux IRQ number
+ * @shared: Indicates if the interrupt has to be context-switched or
+ * if it is private to a VM
*
* Establish a mapping between a guest visible irq (@virt_irq) and a
* Linux irq (@irq). On injection, @virt_irq will be associated with
* the physical interrupt represented by @irq. This mapping can be
* established multiple times as long as the parameters are the same.
+ * If @shared is true, the active state of the interrupt will be
+ * context-switched.
*
* Returns a valid pointer on success, and an error pointer otherwise
*/
struct irq_phys_map *kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu,
- int virt_irq, int irq)
+ int virt_irq, int irq, bool shared)
{
struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
struct list_head *root = vgic_get_irq_phys_map_list(vcpu, virt_irq);
@@ -1761,7 +1778,8 @@ struct irq_phys_map *kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu,
if (map) {
/* Make sure this mapping matches */
if (map->phys_irq != phys_irq ||
- map->irq != irq)
+ map->irq != irq ||
+ map->shared != shared)
map = ERR_PTR(-EINVAL);
/* Found an existing, valid mapping */
@@ -1772,6 +1790,7 @@ struct irq_phys_map *kvm_vgic_map_phys_irq(struct kvm_vcpu *vcpu,
map->virt_irq = virt_irq;
map->phys_irq = phys_irq;
map->irq = irq;
+ map->shared = shared;
list_add_tail_rcu(&entry->entry, root);
--
1.9.1
Up to now irqfd injection was only possible for unmapped IRQ. This
patch adds support for injecting mapped interrupts.
Signed-off-by: Eric Auger <[email protected]>
---
virt/kvm/arm/vgic.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index dba8eb6..e96f79e 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -2476,13 +2476,21 @@ int kvm_irq_map_chip_pin(struct kvm *kvm, unsigned irqchip, unsigned pin)
int kvm_set_irq(struct kvm *kvm, int irq_source_id,
u32 irq, int level, bool line_status)
{
+ struct vgic_dist *dist = &kvm->arch.vgic;
unsigned int spi = irq + VGIC_NR_PRIVATE_IRQS;
+ struct irq_phys_map *map;
+ int vcpu_id;
trace_kvm_set_irq(irq, level, irq_source_id);
BUG_ON(!vgic_initialized(kvm));
- return kvm_vgic_inject_irq(kvm, 0, spi, level);
+ vcpu_id = dist->irq_spi_cpu[irq];
+ map = vgic_irq_map_search(kvm_get_vcpu(kvm, vcpu_id), spi);
+ if (!map)
+ return kvm_vgic_inject_irq(kvm, vcpu_id, spi, level);
+ else
+ return kvm_vgic_inject_mapped_irq(kvm, vcpu_id, map, level);
}
/* MSI not implemented yet */
--
1.9.1
Implements kvm_vgic_[set|unset]_forward.
Handle low-level VGIC programming: physical IRQ/guest IRQ mapping,
list register cleanup, VGIC state machine. Also interacts with
the irqchip.
Signed-off-by: Eric Auger <[email protected]>
---
v3 -> v4:
- use the target vCPU in set/unset_forward
- rebase after removal of vgic_irq_lr_map
- clarify set/unset_forward comments,
- host_irq renamed into irq and virt_irq (SPI ID) now used in
place of SPI index (former guest_irq)
- replaced BUG_ON by WARN_ON
- simplify unset_forward implementation by using vgic_unqueue_irqs
- handle edge mapped unshared IRQs
v2 -> v3:
- on unforward, we do not compute & output the active state anymore.
This means if the unforward happens while the physical IRQ is
active, we will not VFIO mask the IRQ while deactiving it. If a
new physical IRQ hits, the corresponding virtual IRQ might not
be injected (hence lost) due to VGIC state machine.
bypass rfc v2:
- use irq_set_vcpu_affinity API
- use irq_set_irqchip_state instead of chip->irq_eoi
bypass rfc:
- rename kvm_arch_{set|unset}_forward into
kvm_vgic_{set|unset}_forward. Remove __KVM_HAVE_ARCH_HALT_GUEST.
The function is bound to be called by ARM code only.
v4 -> v5:
- fix arm64 compilation issues, ie. also defines
__KVM_HAVE_ARCH_HALT_GUEST for arm64
v3 -> v4:
- code originally located in kvm_vfio_arm.c
- kvm_arch_vfio_{set|unset}_forward renamed into
kvm_arch_{set|unset}_forward
- split into 2 functions (set/unset) since unset does not fail anymore
- unset can be invoked at whatever time. Extra care is taken to handle
transition in VGIC state machine, LR cleanup, ...
v2 -> v3:
- renaming of kvm_arch_set_fwd_state into kvm_arch_vfio_set_forward
- takes a bool arg instead of kvm_fwd_irq_action enum
- removal of KVM_VFIO_IRQ_CLEANUP
- platform device check now happens here
- more precise errors returned
- irq_eoi handled externally to this patch (VGIC)
- correct enable_irq bug done twice
- reword the commit message
- correct check of platform_bus_type
- use raw_spin_lock_irqsave and check the validity of the handler
---
include/kvm/arm_vgic.h | 5 +++
virt/kvm/arm/vgic.c | 118 +++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 123 insertions(+)
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 9bf6a30..8090ab4 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -368,4 +368,9 @@ static inline int vgic_v3_probe(struct device_node *vgic_node,
}
#endif
+int kvm_vgic_set_forward(struct kvm *kvm, unsigned int irq,
+ unsigned int virt_irq);
+void kvm_vgic_unset_forward(struct kvm *kvm, unsigned int irq,
+ unsigned int virt_irq);
+
#endif
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index e96f79e..bd500b4 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -2500,3 +2500,121 @@ int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e,
{
return 0;
}
+
+/**
+ * kvm_vgic_set_forward - Set IRQ forwarding
+ *
+ * @kvm: handle to the VM
+ * @irq: the host linux IRQ
+ * @virt_irq: the guest SPI ID
+ *
+ * This function is supposed to be called only if the IRQ
+ * is not in progress: ie. not active at GIC level and not
+ * currently under injection in the guest. The physical IRQ must
+ * also be disabled and all vCPUs must have been exited and
+ * prevented from being re-entered.
+ */
+int kvm_vgic_set_forward(struct kvm *kvm, unsigned int irq,
+ unsigned int virt_irq)
+{
+ struct vgic_dist *dist = &kvm->arch.vgic;
+ struct kvm_vcpu *vcpu;
+ struct irq_phys_map *map;
+ int cpu_id;
+
+ kvm_debug("%s irq=%d virt_irq=%d\n", __func__, irq, virt_irq);
+
+ cpu_id = dist->irq_spi_cpu[virt_irq - VGIC_NR_PRIVATE_IRQS];
+ vcpu = kvm_get_vcpu(kvm, cpu_id);
+ if (!vcpu)
+ return 0;
+ /*
+ * let's tell the irqchip driver that after this function
+ * returns, a new occurrence of that physical irq will be handled
+ * as a forwarded IRQ, ie. the host will only perform priority
+ * drop but will not deactivate the physical IRQ: guest will
+ */
+ irq_set_vcpu_affinity(irq, vcpu);
+
+ /*
+ * let's program the vgic so that after this function returns
+ * any subsequent virt_irq injection will be considered as
+ * forwarded and LR will be programmed with HWbit set
+ */
+ map = kvm_vgic_map_phys_irq(vcpu, virt_irq, irq, false);
+
+ return !map;
+}
+
+/**
+ * kvm_vgic_unset_forward - Unset IRQ forwarding
+ *
+ * @kvm: handle to the VM
+ * @irq: host Linux IRQ number
+ * @virt_irq: virtual SPI ID
+ *
+ * This function must be called when the host irq is disabled
+ * and all vCPUs have been exited and prevented from being re-entered.
+ */
+void kvm_vgic_unset_forward(struct kvm *kvm,
+ unsigned int irq,
+ unsigned int virt_irq)
+{
+ struct vgic_dist *dist = &kvm->arch.vgic;
+ struct kvm_vcpu *vcpu;
+ int cpu_id;
+ bool active, is_level;
+ struct irq_phys_map *map;
+
+ kvm_debug("%s irq=%d virt_irq=%d\n", __func__, irq, virt_irq);
+
+ spin_lock(&dist->lock);
+
+ cpu_id = dist->irq_spi_cpu[virt_irq - VGIC_NR_PRIVATE_IRQS];
+ vcpu = kvm_get_vcpu(kvm, cpu_id);
+ is_level = !vgic_irq_is_edge(vcpu, virt_irq);
+
+ irq_get_irqchip_state(irq, IRQCHIP_STATE_ACTIVE, &active);
+
+ if (!vcpu)
+ goto out;
+
+ map = vgic_irq_map_search(vcpu, virt_irq);
+ if (!map || map->shared || kvm_vgic_unmap_phys_irq(vcpu, map)) {
+ WARN_ON(1);
+ goto out;
+ }
+
+ if (!active) {
+ /*
+ * The physical IRQ is not active so no virtual IRQ
+ * is under injection, reset the level state which was
+ * not modelled.
+ */
+ vgic_dist_irq_clear_level(vcpu, virt_irq);
+ goto out;
+ }
+
+ vgic_unqueue_irqs(vcpu);
+
+ if (vgic_dist_irq_is_pending(vcpu, virt_irq) ||
+ vgic_irq_is_active(vcpu, virt_irq)) {
+ /*
+ * on next flush, LR for virt_irq will be programmed
+ * with maintenance IRQ. For level sensitive IRQ,
+ * let's set the level which was not modelled up to now
+ */
+ if (is_level)
+ vgic_dist_irq_set_level(vcpu, virt_irq);
+ } else {
+ vgic_dist_irq_clear_level(vcpu, virt_irq);
+ }
+
+out:
+ /* Let's deactivate and unmap the physical IRQ from the vCPU */
+ irq_set_irqchip_state(irq, IRQCHIP_STATE_ACTIVE, false);
+ irq_set_vcpu_affinity(irq, NULL);
+
+ spin_unlock(&dist->lock);
+}
+
--
1.9.1
When disabling a non shared mapped IRQs, we must manually deactivate
the corresponding physical IRQ on top of removing the active state
from the distributor.
Signed-off-by: Eric Auger <[email protected]>
---
virt/kvm/arm/vgic.c | 22 +++++++++++++++++++++-
1 file changed, 21 insertions(+), 1 deletion(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index bd500b4..98ae15f 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -604,14 +604,34 @@ bool vgic_handle_clear_active_reg(struct kvm *kvm,
struct kvm_exit_mmio *mmio,
phys_addr_t offset, int vcpu_id)
{
- u32 *reg;
+ u32 *reg, orig, cleared;
struct vgic_dist *dist = &kvm->arch.vgic;
+ struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, vcpu_id);
+ unsigned int i;
reg = vgic_bitmap_get_reg(&dist->irq_active, vcpu_id, offset);
+ orig = *reg;
vgic_reg_access(mmio, reg, offset,
ACCESS_READ_VALUE | ACCESS_WRITE_CLEARBIT);
if (mmio->is_write) {
+ cleared = orig ^ *reg;
+
+ for (i = 0; i < 32; i++) {
+ struct irq_phys_map *map;
+ unsigned int irq_num;
+
+ if (!(cleared && (1 << i)))
+ continue;
+ irq_num = (offset * 8) + i;
+ map = vgic_irq_map_search(vcpu, irq_num);
+ if (!map || (map && map->shared))
+ continue;
+ irq_set_irqchip_state(map->irq,
+ IRQCHIP_STATE_ACTIVE,
+ false);
+ }
+
vgic_update_state(kvm);
return true;
}
--
1.9.1
This patch implements the clear of a pending non shared mapped IRQ.
In case of an edge IRQ, we deactivate the physical IRQ that will never
be deactivated by the guest. In case of a level sensitive IRQ we check
the level of the input signal. If it is asserted we leave the virtual
IRQ pending. In the opposite, we remove the pending state and deactivate
the IRQ.
Signed-off-by: Eric Auger <[email protected]>
---
virt/kvm/arm/vgic.c | 34 ++++++++++++++++++++++++++++++++--
1 file changed, 32 insertions(+), 2 deletions(-)
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 98ae15f..4be5972 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -550,20 +550,50 @@ bool vgic_handle_clear_pending_reg(struct kvm *kvm,
phys_addr_t offset, int vcpu_id)
{
u32 *level_active;
- u32 *reg, orig;
+ u32 *reg, orig, cleared;
int mode = ACCESS_READ_VALUE | ACCESS_WRITE_CLEARBIT;
+ unsigned int i;
struct vgic_dist *dist = &kvm->arch.vgic;
+ struct kvm_vcpu *vcpu = kvm_get_vcpu(kvm, vcpu_id);
reg = vgic_bitmap_get_reg(&dist->irq_pending, vcpu_id, offset);
orig = *reg;
vgic_reg_access(mmio, reg, offset, mode);
if (mmio->is_write) {
- /* Re-set level triggered level-active interrupts */
level_active = vgic_bitmap_get_reg(&dist->irq_level,
vcpu_id, offset);
+ cleared = orig ^ *reg;
+
+ /* Re-set level triggered level-active interrupts */
reg = vgic_bitmap_get_reg(&dist->irq_pending, vcpu_id, offset);
*reg |= *level_active;
+ for (i = 0; i < 32; i++) {
+ struct irq_phys_map *map;
+ bool phys_pending;
+ unsigned int irq_num;
+
+ if (!(cleared && (1 << i)))
+ continue;
+ irq_num = (offset * 8) + i;
+ map = vgic_irq_map_search(vcpu, irq_num);
+ if (!map || (map && map->shared))
+ continue;
+ /* check whether the signal is asserted */
+ irq_get_irqchip_state(map->irq,
+ IRQCHIP_STATE_PENDING,
+ &phys_pending);
+ if (!vgic_irq_is_edge(vcpu, irq_num) && phys_pending) {
+ vgic_dist_irq_set_pending(vcpu, irq_num);
+ continue;
+ }
+
+ vgic_dist_irq_clear_pending(vcpu, irq_num);
+ irq_set_irqchip_state(map->irq,
+ IRQCHIP_STATE_ACTIVE,
+ false);
+ }
+
/* Ignore writes to SGIs */
if (offset < 2) {
*reg &= ~0xffff;
--
1.9.1
Implement IRQ bypass callbacks for arm/arm64 IRQ forwarding:
- kvm_arch_irq_bypass_add_producer: perform VGIC/irqchip
settings for forwarding
- kvm_arch_irq_bypass_del_producer: same for inverse operation
- kvm_arch_irq_bypass_stop: halt guest execution
- kvm_arch_irq_bypass_start: resume guest execution
and set CONFIG_HAVE_KVM_IRQ_BYPASS for arm/arm64.
Signed-off-by: Eric Auger <[email protected]>
---
v3 -> v4:
- kvm_set/unset_forward now takes the virtual IRQ ID as a parameter
v2 -> v3:
- kvm_arch_irq_bypass_resume renamed into kvm_arch_irq_bypass_start
- kvm_vgic_unset_forward does not take the active bool param anymore
- kvm_arch_irq_bypass_add_producer now returns an error value
- remove kvm_arch_irq_bypass_update
v1 -> v2:
- struct kvm_kernel_irqfd is retrieved with container_of
- function names changed
---
arch/arm/kvm/Kconfig | 1 +
arch/arm/kvm/arm.c | 37 +++++++++++++++++++++++++++++++++++++
arch/arm64/kvm/Kconfig | 1 +
3 files changed, 39 insertions(+)
diff --git a/arch/arm/kvm/Kconfig b/arch/arm/kvm/Kconfig
index 73d7201..4c9a95e 100644
--- a/arch/arm/kvm/Kconfig
+++ b/arch/arm/kvm/Kconfig
@@ -34,6 +34,7 @@ config KVM
select HAVE_KVM_EVENTFD
select HAVE_KVM_IRQFD
select IRQ_BYPASS_MANAGER
+ select HAVE_KVM_IRQ_BYPASS
depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
---help---
Support hosting virtualized guest machines.
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index eab83b2..99517bd 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -27,6 +27,8 @@
#include <linux/mman.h>
#include <linux/sched.h>
#include <linux/kvm.h>
+#include <linux/kvm_irqfd.h>
+#include <linux/irqbypass.h>
#include <trace/events/kvm.h>
#define CREATE_TRACE_POINTS
@@ -1165,6 +1167,41 @@ struct kvm_vcpu *kvm_mpidr_to_vcpu(struct kvm *kvm, unsigned long mpidr)
return NULL;
}
+int kvm_arch_irq_bypass_add_producer(struct irq_bypass_consumer *cons,
+ struct irq_bypass_producer *prod)
+{
+ struct kvm_kernel_irqfd *irqfd =
+ container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+ return kvm_vgic_set_forward(irqfd->kvm, prod->irq,
+ irqfd->gsi + VGIC_NR_PRIVATE_IRQS);
+}
+void kvm_arch_irq_bypass_del_producer(struct irq_bypass_consumer *cons,
+ struct irq_bypass_producer *prod)
+{
+ struct kvm_kernel_irqfd *irqfd =
+ container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+ kvm_vgic_unset_forward(irqfd->kvm, prod->irq,
+ irqfd->gsi + VGIC_NR_PRIVATE_IRQS);
+}
+
+void kvm_arch_irq_bypass_stop(struct irq_bypass_consumer *cons)
+{
+ struct kvm_kernel_irqfd *irqfd =
+ container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+ kvm_arm_halt_guest(irqfd->kvm);
+}
+
+void kvm_arch_irq_bypass_start(struct irq_bypass_consumer *cons)
+{
+ struct kvm_kernel_irqfd *irqfd =
+ container_of(cons, struct kvm_kernel_irqfd, consumer);
+
+ kvm_arm_resume_guest(irqfd->kvm);
+}
+
/**
* Initialize Hyp-mode and memory mappings on all CPUs.
*/
diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig
index 64f460d..f6829af 100644
--- a/arch/arm64/kvm/Kconfig
+++ b/arch/arm64/kvm/Kconfig
@@ -38,6 +38,7 @@ config KVM
select HAVE_KVM_IRQFD
select KVM_ARM_VGIC_V3
select IRQ_BYPASS_MANAGER
+ select HAVE_KVM_IRQ_BYPASS
---help---
Support hosting virtualized guest machines.
We don't support KVM with 16K page tables yet, due to the multiple
--
1.9.1