With an ARM SMMU, interrupt remapping should always be safe from the
SMMU's point of view, as it is properly handled by the GIC.
Signed-off-by: Antonios Motakis <[email protected]>
---
drivers/iommu/arm-smmu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c
index 15ab2af..ff29402 100644
--- a/drivers/iommu/arm-smmu.c
+++ b/drivers/iommu/arm-smmu.c
@@ -1544,7 +1544,7 @@ static int arm_smmu_domain_has_cap(struct iommu_domain *domain,
if (smmu_domain->root_cfg.smmu->features & ARM_SMMU_FEAT_COHERENT_WALK)
caps |= IOMMU_CAP_CACHE_COHERENCY;
- caps |= IOMMU_CAP_NOEXEC;
+ caps |= IOMMU_CAP_NOEXEC | IOMMU_CAP_INTR_REMAP;
return !!(cap & caps);
}
--
1.8.3.2
> -----Original Message-----
> From: [email protected] [mailto:iommu-
> [email protected]] On Behalf Of Antonios Motakis
> Sent: Thursday, June 05, 2014 10:33 PM
> To: [email protected]; [email protected];
> [email protected]
> Cc: [email protected]; [email protected]; open list;
> [email protected]; [email protected]; Yoder Stuart-B08248;
> moderated list:ARM SMMU DRIVER; Antonios Motakis;
> [email protected]; [email protected]
> Subject: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
> IOMMU_CAP_INTR_REMAP
>
> With an ARM SMMU, interrupt remapping should always be safe from the
> SMMU's point of view, as it is properly handled by the GIC.
>
> Signed-off-by: Antonios Motakis <[email protected]>
> ---
> drivers/iommu/arm-smmu.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index
> 15ab2af..ff29402 100644
> --- a/drivers/iommu/arm-smmu.c
> +++ b/drivers/iommu/arm-smmu.c
> @@ -1544,7 +1544,7 @@ static int arm_smmu_domain_has_cap(struct
> iommu_domain *domain,
> if (smmu_domain->root_cfg.smmu->features &
> ARM_SMMU_FEAT_COHERENT_WALK)
> caps |= IOMMU_CAP_CACHE_COHERENCY;
>
> - caps |= IOMMU_CAP_NOEXEC;
> + caps |= IOMMU_CAP_NOEXEC | IOMMU_CAP_INTR_REMAP;
>
> return !!(cap & caps);
> }
Why mention this as an IOMMU capability when IOMMU doesn't support it? Also, wouldn't this depend on the GIC IP version? What are we gaining by adding this IOMMU capability list?
-Varun
On Thu, Jun 05, 2014 at 07:03:12PM +0200, Antonios Motakis wrote:
> With an ARM SMMU, interrupt remapping should always be safe from the
> SMMU's point of view, as it is properly handled by the GIC.
>
> Signed-off-by: Antonios Motakis <[email protected]>
> ---
> drivers/iommu/arm-smmu.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c
> index 15ab2af..ff29402 100644
> --- a/drivers/iommu/arm-smmu.c
> +++ b/drivers/iommu/arm-smmu.c
> @@ -1544,7 +1544,7 @@ static int arm_smmu_domain_has_cap(struct iommu_domain *domain,
> if (smmu_domain->root_cfg.smmu->features & ARM_SMMU_FEAT_COHERENT_WALK)
> caps |= IOMMU_CAP_CACHE_COHERENCY;
>
> - caps |= IOMMU_CAP_NOEXEC;
> + caps |= IOMMU_CAP_NOEXEC | IOMMU_CAP_INTR_REMAP;
>
> return !!(cap & caps);
> }
> --
> 1.8.3.2
>
What does IOMMU_CAP_INTR_REMAP signify exactly? Is there docs/examples
somewhere I can look at? (A quick scan of the Linux souce code doesn't
reveal much, and I'm not sure if this is purely MSI related or what...)
Thanks,
-Christoffer
On Sun, Jun 08, 2014 at 12:31:29PM +0200, Christoffer Dall wrote:
> On Thu, Jun 05, 2014 at 07:03:12PM +0200, Antonios Motakis wrote:
> > With an ARM SMMU, interrupt remapping should always be safe from the
> > SMMU's point of view, as it is properly handled by the GIC.
> >
> > Signed-off-by: Antonios Motakis <[email protected]>
> > ---
> > drivers/iommu/arm-smmu.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c
> > index 15ab2af..ff29402 100644
> > --- a/drivers/iommu/arm-smmu.c
> > +++ b/drivers/iommu/arm-smmu.c
> > @@ -1544,7 +1544,7 @@ static int arm_smmu_domain_has_cap(struct iommu_domain *domain,
> > if (smmu_domain->root_cfg.smmu->features & ARM_SMMU_FEAT_COHERENT_WALK)
> > caps |= IOMMU_CAP_CACHE_COHERENCY;
> >
> > - caps |= IOMMU_CAP_NOEXEC;
> > + caps |= IOMMU_CAP_NOEXEC | IOMMU_CAP_INTR_REMAP;
> >
> > return !!(cap & caps);
> > }
> > --
> > 1.8.3.2
> >
> What does IOMMU_CAP_INTR_REMAP signify exactly? Is there docs/examples
> somewhere I can look at? (A quick scan of the Linux souce code doesn't
> reveal much, and I'm not sure if this is purely MSI related or what...)
The flag was introduced for x86 IOMMUs to detect whether an IOMMU in the
system has and enables interrupt remapping to allow safe device
assignment to KVM guests. Without interrupt remapping a malicious guest
could attack the host with MSIs from the attached device.
How are PCI MSIs implemented on ARM with SMMU and GIC enabled. MSIs are
only memory dma transactions in the end, is it guaranteed on ARM that a
device only sends MSI transactions it is allowed to?
Joerg
On Mon, Jun 16, 2014 at 03:53:44PM +0100, Joerg Roedel wrote:
> On Sun, Jun 08, 2014 at 12:31:29PM +0200, Christoffer Dall wrote:
> > On Thu, Jun 05, 2014 at 07:03:12PM +0200, Antonios Motakis wrote:
> > > With an ARM SMMU, interrupt remapping should always be safe from the
> > > SMMU's point of view, as it is properly handled by the GIC.
> > >
> > > Signed-off-by: Antonios Motakis <[email protected]>
> > > ---
> > > drivers/iommu/arm-smmu.c | 2 +-
> > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c
> > > index 15ab2af..ff29402 100644
> > > --- a/drivers/iommu/arm-smmu.c
> > > +++ b/drivers/iommu/arm-smmu.c
> > > @@ -1544,7 +1544,7 @@ static int arm_smmu_domain_has_cap(struct iommu_domain *domain,
> > > if (smmu_domain->root_cfg.smmu->features & ARM_SMMU_FEAT_COHERENT_WALK)
> > > caps |= IOMMU_CAP_CACHE_COHERENCY;
> > >
> > > - caps |= IOMMU_CAP_NOEXEC;
> > > + caps |= IOMMU_CAP_NOEXEC | IOMMU_CAP_INTR_REMAP;
> > >
> > > return !!(cap & caps);
> > > }
> > > --
> > > 1.8.3.2
> > >
> > What does IOMMU_CAP_INTR_REMAP signify exactly? Is there docs/examples
> > somewhere I can look at? (A quick scan of the Linux souce code doesn't
> > reveal much, and I'm not sure if this is purely MSI related or what...)
>
> The flag was introduced for x86 IOMMUs to detect whether an IOMMU in the
> system has and enables interrupt remapping to allow safe device
> assignment to KVM guests. Without interrupt remapping a malicious guest
> could attack the host with MSIs from the attached device.
>
> How are PCI MSIs implemented on ARM with SMMU and GIC enabled. MSIs are
> only memory dma transactions in the end, is it guaranteed on ARM that a
> device only sends MSI transactions it is allowed to?
MSIs look just like memory accesses made by the device, so the SMMU will
translate them to point at the GIC ITS (doorbell). The ITS then has tables
to work out how to route the MSI.
So, if IOMMU_CAP_INTR_REMAP is simply supposed to indicate that the SMMU can
translate MSIs to point somewhere else, then the ARM SMMU can always do it.
If it's supposed to indicate that the actual MSI payload can be
filtered/routed, then that requires the GIC ITS.
The part I'm unsure of is how VFIO knows where to map the MSIs to. That
requires knowledge of the physical and virtual doorbell pages -- is that
discoverable in the API?
Will
On Mon, Jun 16, 2014 at 04:13:29PM +0100, Will Deacon wrote:
> MSIs look just like memory accesses made by the device, so the SMMU
> will translate them to point at the GIC ITS (doorbell). The ITS then
> has tables to work out how to route the MSI.
>
> So, if IOMMU_CAP_INTR_REMAP is simply supposed to indicate that the
> SMMU can translate MSIs to point somewhere else, then the ARM SMMU can
> always do it. If it's supposed to indicate that the actual MSI
> payload can be filtered/routed, then that requires the GIC ITS.
>
> The part I'm unsure of is how VFIO knows where to map the MSIs to.
> That requires knowledge of the physical and virtual doorbell pages --
> is that discoverable in the API?
VFIO does not care about the actual routing, it only cares that the
device can not send interrupts it is not allowed to send (e.g.
interrupts to vectors used by other devices or, on x86, exception vectors).
If that is guaranteed by the SMMU or the GIC ITS hardware and driver
then it is fine to set this flag.
Joerg
On Mon, Jun 16, 2014 at 04:21:58PM +0100, Joerg Roedel wrote:
> On Mon, Jun 16, 2014 at 04:13:29PM +0100, Will Deacon wrote:
> > MSIs look just like memory accesses made by the device, so the SMMU
> > will translate them to point at the GIC ITS (doorbell). The ITS then
> > has tables to work out how to route the MSI.
> >
> > So, if IOMMU_CAP_INTR_REMAP is simply supposed to indicate that the
> > SMMU can translate MSIs to point somewhere else, then the ARM SMMU can
> > always do it. If it's supposed to indicate that the actual MSI
> > payload can be filtered/routed, then that requires the GIC ITS.
> >
> > The part I'm unsure of is how VFIO knows where to map the MSIs to.
> > That requires knowledge of the physical and virtual doorbell pages --
> > is that discoverable in the API?
>
> VFIO does not care about the actual routing, it only cares that the
> device can not send interrupts it is not allowed to send (e.g.
> interrupts to vectors used by other devices or, on x86, exception vectors).
> If that is guaranteed by the SMMU or the GIC ITS hardware and driver
> then it is fine to set this flag.
Ok, thanks. In which case, I think this is really a combined property of
the SMMU and the interrupt controller, so we might need some extra code
so that the SMMU can check that the interrupt controller for the device
is also capable of interrupt remapping.
Will
On Mon, 2014-06-16 at 17:21 +0200, Joerg Roedel wrote:
> On Mon, Jun 16, 2014 at 04:13:29PM +0100, Will Deacon wrote:
> > MSIs look just like memory accesses made by the device, so the SMMU
> > will translate them to point at the GIC ITS (doorbell). The ITS then
> > has tables to work out how to route the MSI.
> >
> > So, if IOMMU_CAP_INTR_REMAP is simply supposed to indicate that the
> > SMMU can translate MSIs to point somewhere else, then the ARM SMMU can
> > always do it. If it's supposed to indicate that the actual MSI
> > payload can be filtered/routed, then that requires the GIC ITS.
> >
> > The part I'm unsure of is how VFIO knows where to map the MSIs to.
> > That requires knowledge of the physical and virtual doorbell pages --
> > is that discoverable in the API?
>
> VFIO does not care about the actual routing, it only cares that the
> device can not send interrupts it is not allowed to send (e.g.
> interrupts to vectors used by other devices or, on x86, exception vectors).
> If that is guaranteed by the SMMU or the GIC ITS hardware and driver
> then it is fine to set this flag.
Yep, I agree. Thanks,
Alex
On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
> Ok, thanks. In which case, I think this is really a combined property of
> the SMMU and the interrupt controller, so we might need some extra code
> so that the SMMU can check that the interrupt controller for the device
> is also capable of interrupt remapping.
Right, that this is part of IOMMU code has more or less historic reasons
on x86. Interrupt remapping is purely implemented in the IOMMU there, so
on ARM some clue-code between interrupt controler and smmu is needed.
Joerg
Forgive me if this discussion is not relative here, but I thought it is.
How is VFIO restricting devices from writing to MSI/MSI-X,
Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
Regards,
Tirumalesh.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Joerg Roedel
Sent: Monday, June 16, 2014 8:39 AM
To: Will Deacon
Cc: [email protected]; [email protected]; open list; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER; [email protected]; [email protected]; Christoffer Dall
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
> Ok, thanks. In which case, I think this is really a combined property
> of the SMMU and the interrupt controller, so we might need some extra
> code so that the SMMU can check that the interrupt controller for the
> device is also capable of interrupt remapping.
Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
Joerg
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
When I say emulating ITS, I mean translating guest ITS commands to physical ITS commands and placing them in physical queue.
Regards,
Tirumalesh.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Chalamarla, Tirumalesh
Sent: Thursday, June 26, 2014 11:08 AM
To: Joerg Roedel; Will Deacon
Cc: [email protected]; open list; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
Forgive me if this discussion is not relative here, but I thought it is.
How is VFIO restricting devices from writing to MSI/MSI-X, Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
Regards,
Tirumalesh.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Joerg Roedel
Sent: Monday, June 16, 2014 8:39 AM
To: Will Deacon
Cc: [email protected]; [email protected]; open list; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER; [email protected]; [email protected]; Christoffer Dall
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
> Ok, thanks. In which case, I think this is really a combined property
> of the SMMU and the interrupt controller, so we might need some extra
> code so that the SMMU can check that the interrupt controller for the
> device is also capable of interrupt remapping.
Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
Joerg
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Sorry there was a type,
The question is:
How is VFIO restricting software from writing to MSI/MSI-X vectors of the device.
-----Original Message-----
From: Chalamarla, Tirumalesh
Sent: Thursday, June 26, 2014 11:16 AM
To: Chalamarla, Tirumalesh; Joerg Roedel; Will Deacon
Cc: [email protected]; open list; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
When I say emulating ITS, I mean translating guest ITS commands to physical ITS commands and placing them in physical queue.
Regards,
Tirumalesh.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Chalamarla, Tirumalesh
Sent: Thursday, June 26, 2014 11:08 AM
To: Joerg Roedel; Will Deacon
Cc: [email protected]; open list; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER
Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
Forgive me if this discussion is not relative here, but I thought it is.
How is VFIO restricting devices from writing to MSI/MSI-X, Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
Regards,
Tirumalesh.
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Joerg Roedel
Sent: Monday, June 16, 2014 8:39 AM
To: Will Deacon
Cc: [email protected]; [email protected]; open list; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER; [email protected]; [email protected]; Christoffer Dall
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
> Ok, thanks. In which case, I think this is really a combined property
> of the SMMU and the interrupt controller, so we might need some extra
> code so that the SMMU can check that the interrupt controller for the
> device is also capable of interrupt remapping.
Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
Joerg
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
On Thu, 2014-06-26 at 18:41 +0000, Chalamarla, Tirumalesh wrote:
> Sorry there was a type,
>
> The question is:
>
> How is VFIO restricting software from writing to MSI/MSI-X vectors of the device.
All interrupts are configured via ioctl, not MSI config space or the
MSI-X vector table in MMIO space. VFIO protects the MSI config area by
virtualizing it (you can't actually write the physical enable bit or
address/data through VFIO). The MSI-X vector table is protected by
preventing read, write, or mmap access to it. QEMU provides further
virtualization above the basics provided by VFIO. We really can't
guarantee that devices don't have backdoors to configure these though.
See the realtek quirk in QEMU for an example of a device that has such a
backdoor. That's why we require interrupt remapping, so that a device
that does this can only hurt the guest, and require the user to opt-out
if they feel they have a sufficiently trusted guest. Thanks,
Alex
>
> -----Original Message-----
> From: Chalamarla, Tirumalesh
> Sent: Thursday, June 26, 2014 11:16 AM
> To: Chalamarla, Tirumalesh; Joerg Roedel; Will Deacon
> Cc: [email protected]; open list; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER
> Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
>
> When I say emulating ITS, I mean translating guest ITS commands to physical ITS commands and placing them in physical queue.
>
> Regards,
> Tirumalesh.
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On Behalf Of Chalamarla, Tirumalesh
> Sent: Thursday, June 26, 2014 11:08 AM
> To: Joerg Roedel; Will Deacon
> Cc: [email protected]; open list; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER
> Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
>
> Forgive me if this discussion is not relative here, but I thought it is.
>
> How is VFIO restricting devices from writing to MSI/MSI-X, Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
> And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
>
> Regards,
> Tirumalesh.
>
> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On Behalf Of Joerg Roedel
> Sent: Monday, June 16, 2014 8:39 AM
> To: Will Deacon
> Cc: [email protected]; [email protected]; open list; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER; [email protected]; [email protected]; Christoffer Dall
> Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
>
> On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
> > Ok, thanks. In which case, I think this is really a combined property
> > of the SMMU and the interrupt controller, so we might need some extra
> > code so that the SMMU can check that the interrupt controller for the
> > device is also capable of interrupt remapping.
>
> Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
>
>
> Joerg
>
>
> _______________________________________________
> kvmarm mailing list
> [email protected]
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
> _______________________________________________
> kvmarm mailing list
> [email protected]
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Thanks for the clarification Alex, That’s exactly my point, why are we relying on QEMU or something else to emulate the MSI space when we can directly give access to devices using ITS (of course with a small emulation code).
This way we are also benefited from all ITS services like VCPU migration etc.
What about non QEMU VFIO users, for example, if I wanted to use VFIO to assign a device to a user process I don't need to depend on QEMU. I thought this is one of the main goals of vfio to make it independent of hypervisors.
Thanks,
Tirumalesh.
-----Original Message-----
From: Alex Williamson [mailto:[email protected]]
Sent: Thursday, June 26, 2014 12:00 PM
To: Chalamarla, Tirumalesh
Cc: Joerg Roedel; Will Deacon; [email protected]; open list; [email protected]; [email protected]; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Thu, 2014-06-26 at 18:41 +0000, Chalamarla, Tirumalesh wrote:
> Sorry there was a type,
>
> The question is:
>
> How is VFIO restricting software from writing to MSI/MSI-X vectors of the device.
All interrupts are configured via ioctl, not MSI config space or the MSI-X vector table in MMIO space. VFIO protects the MSI config area by virtualizing it (you can't actually write the physical enable bit or address/data through VFIO). The MSI-X vector table is protected by preventing read, write, or mmap access to it. QEMU provides further virtualization above the basics provided by VFIO. We really can't guarantee that devices don't have backdoors to configure these though.
See the realtek quirk in QEMU for an example of a device that has such a backdoor. That's why we require interrupt remapping, so that a device that does this can only hurt the guest, and require the user to opt-out if they feel they have a sufficiently trusted guest. Thanks,
Alex
>
> -----Original Message-----
> From: Chalamarla, Tirumalesh
> Sent: Thursday, June 26, 2014 11:16 AM
> To: Chalamarla, Tirumalesh; Joerg Roedel; Will Deacon
> Cc: [email protected]; open list; [email protected];
> [email protected]; [email protected];
> [email protected]; [email protected]; moderated
> list:ARM SMMU DRIVER
> Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
> IOMMU_CAP_INTR_REMAP
>
> When I say emulating ITS, I mean translating guest ITS commands to physical ITS commands and placing them in physical queue.
>
> Regards,
> Tirumalesh.
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Chalamarla,
> Tirumalesh
> Sent: Thursday, June 26, 2014 11:08 AM
> To: Joerg Roedel; Will Deacon
> Cc: [email protected]; open list; [email protected];
> [email protected]; [email protected];
> [email protected]; [email protected]; moderated
> list:ARM SMMU DRIVER
> Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
> IOMMU_CAP_INTR_REMAP
>
> Forgive me if this discussion is not relative here, but I thought it is.
>
> How is VFIO restricting devices from writing to MSI/MSI-X, Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
> And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
>
> Regards,
> Tirumalesh.
>
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]] On Behalf Of Joerg
> Roedel
> Sent: Monday, June 16, 2014 8:39 AM
> To: Will Deacon
> Cc: [email protected]; [email protected]; open list;
> [email protected]; [email protected];
> moderated list:ARM SMMU DRIVER; [email protected];
> [email protected]; Christoffer Dall
> Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
> IOMMU_CAP_INTR_REMAP
>
> On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
> > Ok, thanks. In which case, I think this is really a combined
> > property of the SMMU and the interrupt controller, so we might need
> > some extra code so that the SMMU can check that the interrupt
> > controller for the device is also capable of interrupt remapping.
>
> Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
>
>
> Joerg
>
>
> _______________________________________________
> kvmarm mailing list
> [email protected]
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
> _______________________________________________
> kvmarm mailing list
> [email protected]
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay? ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?
On Thu, 2014-06-26 at 19:10 +0000, Chalamarla, Tirumalesh wrote:
> Thanks for the clarification Alex, That’s exactly my point, why are we relying on QEMU or something else to emulate the MSI space when we can directly give access to devices using ITS (of course with a small emulation code).
> This way we are also benefited from all ITS services like VCPU migration etc.
I have no idea what ITS is.
> What about non QEMU VFIO users, for example, if I wanted to use VFIO to assign a device to a user process I don't need to depend on QEMU. I thought this is one of the main goals of vfio to make it independent of hypervisors.
Where did QEMU become a requirement? Maybe I'm missing something in the
ARM part of the conversation that got chopped off, but this is exactly
why we have the VFIO/QEMU split that we do. VFIO provides basic
virtualization for config space and restricts access to other areas that
users shouldn't be allowed to change. QEMU is just one example of a
userspace VFIO driver. QEMU takes the decomposed device exposed through
the VFIO ABI and re-creates a PCI device out of it. VFIO itself has no
dependency on QEMU. Thanks,
Alex
> -----Original Message-----
> From: Alex Williamson [mailto:[email protected]]
> Sent: Thursday, June 26, 2014 12:00 PM
> To: Chalamarla, Tirumalesh
> Cc: Joerg Roedel; Will Deacon; [email protected]; open list; [email protected]; [email protected]; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER
> Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
>
> On Thu, 2014-06-26 at 18:41 +0000, Chalamarla, Tirumalesh wrote:
> > Sorry there was a type,
> >
> > The question is:
> >
> > How is VFIO restricting software from writing to MSI/MSI-X vectors of the device.
>
> All interrupts are configured via ioctl, not MSI config space or the MSI-X vector table in MMIO space. VFIO protects the MSI config area by virtualizing it (you can't actually write the physical enable bit or address/data through VFIO). The MSI-X vector table is protected by preventing read, write, or mmap access to it. QEMU provides further virtualization above the basics provided by VFIO. We really can't guarantee that devices don't have backdoors to configure these though.
> See the realtek quirk in QEMU for an example of a device that has such a backdoor. That's why we require interrupt remapping, so that a device that does this can only hurt the guest, and require the user to opt-out if they feel they have a sufficiently trusted guest. Thanks,
>
> Alex
>
> >
> > -----Original Message-----
> > From: Chalamarla, Tirumalesh
> > Sent: Thursday, June 26, 2014 11:16 AM
> > To: Chalamarla, Tirumalesh; Joerg Roedel; Will Deacon
> > Cc: [email protected]; open list; [email protected];
> > [email protected]; [email protected];
> > [email protected]; [email protected]; moderated
> > list:ARM SMMU DRIVER
> > Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
> > IOMMU_CAP_INTR_REMAP
> >
> > When I say emulating ITS, I mean translating guest ITS commands to physical ITS commands and placing them in physical queue.
> >
> > Regards,
> > Tirumalesh.
> >
> > -----Original Message-----
> > From: [email protected]
> > [mailto:[email protected]] On Behalf Of Chalamarla,
> > Tirumalesh
> > Sent: Thursday, June 26, 2014 11:08 AM
> > To: Joerg Roedel; Will Deacon
> > Cc: [email protected]; open list; [email protected];
> > [email protected]; [email protected];
> > [email protected]; [email protected]; moderated
> > list:ARM SMMU DRIVER
> > Subject: RE: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
> > IOMMU_CAP_INTR_REMAP
> >
> > Forgive me if this discussion is not relative here, but I thought it is.
> >
> > How is VFIO restricting devices from writing to MSI/MSI-X, Is all the vector area is mapped by VFIO to trap the accesses. I am asking this because we might need to emulate ITS somewhere either in KVM or VFIO to provide direct access to devices.
> > And I don't see any mentions on that. I think this flag needs to be set by ITS emulation.
> >
> > Regards,
> > Tirumalesh.
> >
> > -----Original Message-----
> > From: [email protected]
> > [mailto:[email protected]] On Behalf Of Joerg
> > Roedel
> > Sent: Monday, June 16, 2014 8:39 AM
> > To: Will Deacon
> > Cc: [email protected]; [email protected]; open list;
> > [email protected]; [email protected];
> > moderated list:ARM SMMU DRIVER; [email protected];
> > [email protected]; Christoffer Dall
> > Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability
> > IOMMU_CAP_INTR_REMAP
> >
> > On Mon, Jun 16, 2014 at 04:25:26PM +0100, Will Deacon wrote:
> > > Ok, thanks. In which case, I think this is really a combined
> > > property of the SMMU and the interrupt controller, so we might need
> > > some extra code so that the SMMU can check that the interrupt
> > > controller for the device is also capable of interrupt remapping.
> >
> > Right, that this is part of IOMMU code has more or less historic reasons on x86. Interrupt remapping is purely implemented in the IOMMU there, so on ARM some clue-code between interrupt controler and smmu is needed.
> >
> >
> > Joerg
> >
> >
> > _______________________________________________
> > kvmarm mailing list
> > [email protected]
> > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
> > _______________________________________________
> > kvmarm mailing list
> > [email protected]
> > https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
>
>
>
On Thu, Jun 26, 2014 at 08:36:24PM +0100, Alex Williamson wrote:
> On Thu, 2014-06-26 at 19:10 +0000, Chalamarla, Tirumalesh wrote:
> > Thanks for the clarification Alex, That’s exactly my point, why are we
> > relying on QEMU or something else to emulate the MSI space when we can
> > directly give access to devices using ITS (of course with a small
> > emulation code). This way we are also benefited from all ITS services
> > like VCPU migration etc.
>
> I have no idea what ITS is.
ITS is the MSI doorbell for GICv3 (ARM's latest interrupt controller).
I agree that we will need an ITS emulation if we want to use MSIs in the
guest, and I believe that Marc (CC'd) had already started thinking about
that.
> > What about non QEMU VFIO users, for example, if I wanted to use VFIO to
> > assign a device to a user process I don't need to depend on QEMU. I
> > thought this is one of the main goals of vfio to make it independent of
> > hypervisors.
>
> Where did QEMU become a requirement? Maybe I'm missing something in the
> ARM part of the conversation that got chopped off, but this is exactly
> why we have the VFIO/QEMU split that we do. VFIO provides basic
> virtualization for config space and restricts access to other areas that
> users shouldn't be allowed to change. QEMU is just one example of a
> userspace VFIO driver. QEMU takes the decomposed device exposed through
> the VFIO ABI and re-creates a PCI device out of it. VFIO itself has no
> dependency on QEMU. Thanks,
I also don't understand the QEMU part here. The MSI emulation would be in
the kernel, just like the GICv2 emulation that we already have. For
userspace drivers, wouldn't you just use eventfd rather than bother with
emulating MSIs?
Finally, the interrupt remapping part is about the SMMU preventing MSI
writes to arbitrary portions of the host address space. The ITS is about
routing interrupts to CPUs.
Will
Marc,
What is your opinion on ITS emulation . is it should be part of KVM or VFIO.
Also this code needs to depend on ITS host driver a lot, Host ITS driver needs to have an interface for this code to use.
Thanks,
Tirumalesh
-----Original Message-----
From: Will Deacon [mailto:[email protected]]
Sent: Friday, June 27, 2014 1:47 AM
To: Alex Williamson
Cc: Chalamarla, Tirumalesh; Joerg Roedel; [email protected]; open list; [email protected]; [email protected]; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER; [email protected]
Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
On Thu, Jun 26, 2014 at 08:36:24PM +0100, Alex Williamson wrote:
> On Thu, 2014-06-26 at 19:10 +0000, Chalamarla, Tirumalesh wrote:
> > Thanks for the clarification Alex, That’s exactly my point, why are
> > we relying on QEMU or something else to emulate the MSI space when
> > we can directly give access to devices using ITS (of course with a
> > small emulation code). This way we are also benefited from all ITS
> > services like VCPU migration etc.
>
> I have no idea what ITS is.
ITS is the MSI doorbell for GICv3 (ARM's latest interrupt controller).
I agree that we will need an ITS emulation if we want to use MSIs in the guest, and I believe that Marc (CC'd) had already started thinking about that.
> > What about non QEMU VFIO users, for example, if I wanted to use VFIO to
> > assign a device to a user process I don't need to depend on QEMU. I
> > thought this is one of the main goals of vfio to make it independent of
> > hypervisors.
>
> Where did QEMU become a requirement? Maybe I'm missing something in
> the ARM part of the conversation that got chopped off, but this is
> exactly why we have the VFIO/QEMU split that we do. VFIO provides
> basic virtualization for config space and restricts access to other
> areas that users shouldn't be allowed to change. QEMU is just one
> example of a userspace VFIO driver. QEMU takes the decomposed device
> exposed through the VFIO ABI and re-creates a PCI device out of it.
> VFIO itself has no dependency on QEMU. Thanks,
I also don't understand the QEMU part here. The MSI emulation would be in the kernel, just like the GICv2 emulation that we already have. For userspace drivers, wouldn't you just use eventfd rather than bother with emulating MSIs?
Finally, the interrupt remapping part is about the SMMU preventing MSI writes to arbitrary portions of the host address space. The ITS is about routing interrupts to CPUs.
Will
????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay? ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?
On Fri, Jun 27 2014 at 10:57:28 PM, "Chalamarla, Tirumalesh" <[email protected]> wrote:
> Marc,
>
> What is your opinion on ITS emulation . is it should be part
> of KVM or VFIO.
Making any sort of emulation part of VFIO sounds quite wrong. That's not
what VFIO is about, at all. Emulation belongs to the hypervisor, and
nowhere else.
> Also this code needs to depend on ITS host driver a lot, Host
> ITS driver needs to have an interface for this code to use.
You can share the command interface as some form of library, but that's
about it. There is no more relationship between the ITS driver and the
ITS emulation as there is between the GIC driver and its emulation
counterpart.
M.
> Thanks,
> Tirumalesh
>
> -----Original Message-----
> From: Will Deacon [mailto:[email protected]]
> Sent: Friday, June 27, 2014 1:47 AM
> To: Alex Williamson
> Cc: Chalamarla, Tirumalesh; Joerg Roedel; [email protected]; open list; [email protected]; [email protected]; [email protected]; [email protected]; moderated list:ARM SMMU DRIVER; [email protected]
> Subject: Re: [RFC PATCH v6 04/20] iommu/arm-smmu: add capability IOMMU_CAP_INTR_REMAP
>
> On Thu, Jun 26, 2014 at 08:36:24PM +0100, Alex Williamson wrote:
>> On Thu, 2014-06-26 at 19:10 +0000, Chalamarla, Tirumalesh wrote:
>> > Thanks for the clarification Alex, That’s exactly my point, why are
>> > we relying on QEMU or something else to emulate the MSI space when
>> > we can directly give access to devices using ITS (of course with a
>> > small emulation code). This way we are also benefited from all ITS
>> > services like VCPU migration etc.
>>
>> I have no idea what ITS is.
>
> ITS is the MSI doorbell for GICv3 (ARM's latest interrupt controller).
>
> I agree that we will need an ITS emulation if we want to use MSIs in
>> the guest, and I believe that Marc (CC'd) had already started
>> thinking about that.
>
>
>> > What about non QEMU VFIO users, for example, if I wanted to use VFIO to
>> > assign a device to a user process I don't need to depend on QEMU. I
>> > thought this is one of the main goals of vfio to make it independent of
>> > hypervisors.
>>
>> Where did QEMU become a requirement? Maybe I'm missing something in
>> the ARM part of the conversation that got chopped off, but this is
>> exactly why we have the VFIO/QEMU split that we do. VFIO provides
>> basic virtualization for config space and restricts access to other
>> areas that users shouldn't be allowed to change. QEMU is just one
>> example of a userspace VFIO driver. QEMU takes the decomposed device
>> exposed through the VFIO ABI and re-creates a PCI device out of it.
>> VFIO itself has no dependency on QEMU. Thanks,
>
> I also don't understand the QEMU part here. The MSI emulation would be
>> in the kernel, just like the GICv2 emulation that we already
>> have. For userspace drivers, wouldn't you just use eventfd rather
>> than bother with emulating MSIs?
>
> Finally, the interrupt remapping part is about the SMMU preventing MSI
>> writes to arbitrary portions of the host address space. The ITS is
>> about routing interrupts to CPUs.
>
> Will
--
Jazz is not dead. It just smells funny.