LinuxLists.cc - [PATCH 0/6] iommufd: Add iommu capability reporting

[permalink] [raw]

Subject: RE: [PATCH 1/6] iommu: Add new iommu op to get iommu hardware information

> From: Liu, Yi L <[email protected]>
> Sent: Thursday, February 9, 2023 12:17 PM
> @@ -223,6 +224,11 @@ struct iommu_iotlb_gather {
> /**
> * struct iommu_ops - iommu ops and capabilities
> * @capable: check capability
> + * @hw_info: IOMMU hardware capabilities. The type of the returned data
> is

is it 'info' or 'capability'?

> + * defined in include/uapi/linux/iommufd.h. The data buffer is
> + * allocated in the IOMMU driver and the caller should free it
> + * after use. Return the data buffer if success, or ERR_PTR on
> + * failure.
> * @domain_alloc: allocate iommu domain
> * @probe_device: Add device to iommu driver handling
> * @release_device: Remove device from iommu driver handling
> @@ -252,6 +258,7 @@ struct iommu_iotlb_gather {
> */
> struct iommu_ops {
> bool (*capable)(struct device *dev, enum iommu_cap);
> + void *(*hw_info)(struct device *dev, u32 *length);
>
> /* Domain allocation and freeing by the iommu driver */
> struct iommu_domain *(*domain_alloc)(unsigned
> iommu_domain_type);
> @@ -280,6 +287,7 @@ struct iommu_ops {
> void (*remove_dev_pasid)(struct device *dev, ioasid_t pasid);
>
> const struct iommu_domain_ops *default_domain_ops;
> + enum iommu_device_data_type driver_type;

the enum is called 'device_data_type' while the field is called 'driver_type'.

btw this new field is not documented above.

2023-02-10 07:33:02

[permalink] [raw]

Subject: RE: [PATCH 2/6] iommu/vt-d: Implement hw_info for iommu capability query

> From: Liu, Yi L <[email protected]>
> Sent: Thursday, February 9, 2023 12:17 PM
>
> From: Lu Baolu <[email protected]>
>
> To support nested translation in the userspace, it should check the
> underlying hardware information for the capabilities.

remove this sentence. that belongs to next patch.

> +
> +/**
> + * struct iommu_device_info_vtd - Intel VT-d device info
> + *
> + * @flags: Must be set to 0
> + * @__reserved: Must be 0
> + * @cap_reg: Value of Intel VT-d capability register defined in chapter
> + * 11.4.2 of Intel VT-d spec.
> + * @ecap_reg: Value of Intel VT-d capability register defined in chapter
> + * 11.4.3 of Intel VT-d spec.

let's be specific with section name e.g. "11.4.2 Capability Register" so if
the chapter index is changed later people can still know where to look
at.

> + *
> + * Intel hardware iommu capability.

duplicated with the first line "Intel VT-d device info"

2023-02-10 07:55:56

[permalink] [raw]

Subject: RE: [PATCH 3/6] iommufd: Add IOMMU_DEVICE_GET_INFO

> From: Liu, Yi L <[email protected]>
> Sent: Thursday, February 9, 2023 12:17 PM
>
> +int iommufd_device_get_info(struct iommufd_ucmd *ucmd)
> +{
> + struct iommu_device_info *cmd = ucmd->cmd;
> + struct iommufd_object *dev_obj;
> + struct device *dev;
> + const struct iommu_ops *ops;
> + void *data;
> + unsigned int length, data_len;
> + int rc;
> +
> + if (cmd->flags || cmd->__reserved || !cmd->data_len)
> + return -EOPNOTSUPP;

do we want !cmd->data_len being a way to check how many bytes are
required in a following call to get the vendor info?

> +
> + /*
> + * Zero the trailing bytes for userspace if the buffer is bigger
> + * than the data size kernel actually has.
> + */

"Zero the trailing bytes if the user buffer ..."

> +
> +/**
> + * struct iommu_device_info - ioctl(IOMMU_DEVICE_GET_INFO)
> + * @size: sizeof(struct iommu_device_info)
> + * @flags: Must be 0
> + * @dev_id: The device being attached to the IOMMU

iommufd

> + * @data_len: Input the type specific data buffer length in bytes

also is an output field

> + *
> + * Query the hardware iommu capability for given device which has been
> + * bound to iommufd. @data_len is set to be the size of the buffer to
> + * type specific data and the data will be filled. Trailing bytes are

"@data_len is the size of the buffer which captures iommu type specific data"

> + * zeroed if the user buffer is larger than the data kernel has.
> + *
> + * The type specific data would be used to sync capability between the
> + * vIOMMU and the hardware IOMMU, also for the availabillity checking of
> + * iommu hardware features like dirty page tracking in I/O page table.

It's fine to report format information related to stage-1 page table
which userspace manages.

but IMHO this should not be an interface to report which capability is
supported by iommufd. Having hardware supporting dirty bit
doesn't mean the underlying iommu driver provides necessary support
to iommufd dirty tracking.

We either don't state this way if following what this series does which
simply dumps all raw iommu hw info to userspace, or explicitly require
iommu driver to only report information as required when a feature
is supported by iommufd.

> + *
> + * The @out_device_type will be filled if the ioctl succeeds. It would

s/will be/is/

> + * be used to decode the data filled in the buffer pointed by @data_ptr.

s/would be/is/

> + */
> +struct iommu_device_info {
> + __u32 size;
> + __u32 flags;
> + __u32 dev_id;
> + __u32 data_len;
> + __aligned_u64 data_ptr;

moving forward iommu hw cap is not the only information reported
via this interface, e.g. it can be also used to report pasid mode.

from this angle it's better to rename above two fields to be iommu
specific, e.g.:

__u32 iommu_data_len;
__aligned_u64 iommu_data_ptr;

> + __u32 out_device_type;
> + __u32 __reserved;
> +};
> +#define IOMMU_DEVICE_GET_INFO _IO(IOMMUFD_TYPE,
> IOMMUFD_CMD_DEVICE_GET_INFO)
> #endif
> --
> 2.34.1

2023-02-10 11:16:28

by Joao Martins

[permalink] [raw]

Subject: Re: [PATCH 3/6] iommufd: Add IOMMU_DEVICE_GET_INFO

On 10/02/2023 07:55, Tian, Kevin wrote:
>> From: Liu, Yi L <[email protected]>
>> Sent: Thursday, February 9, 2023 12:17 PM
>> + * zeroed if the user buffer is larger than the data kernel has.
>> + *
>> + * The type specific data would be used to sync capability between the
>> + * vIOMMU and the hardware IOMMU, also for the availabillity checking of
>> + * iommu hardware features like dirty page tracking in I/O page table.
>
> It's fine to report format information related to stage-1 page table
> which userspace manages.
>
> but IMHO this should not be an interface to report which capability is
> supported by iommufd. Having hardware supporting dirty bit
> doesn't mean the underlying iommu driver provides necessary support
> to iommufd dirty tracking.
>

+1

In fact this shouldn't really be a way to check any capability as we are dumping
straight away the IOMMU hardware registers. By dumping raw IOMMU hardware data,
forces the application to understand IOMMU hardware specific formats. Maybe
that's OK for hw nesting as there's a lot of info you need to know for the
vIOMMU pgtables, pasid and etc so both are tightly coupled. But it is a bit
disconnected from what really the software (IOMMUFD) and driver can use, without
getting onto assumptions.

[Calling it IOMMU_DEVICE_GET_HW_INFO would be bit more obvious if you're not
asking IOMMUFD support]

For capability checking, I think this really should be returning capabilities
that both IOMMU driver supports ... and that IOMMUFD understands and marshalled
on a format of its own defined by IOMMUFD. Maybe using IOMMU_CAP or some other
thing within the kernel (now that's per-device), or even simpler. That's at
least had written initially for the dirty tracking series.

2023-02-10 20:58:45

[permalink] [raw]

Subject: Re: [PATCH 3/6] iommufd: Add IOMMU_DEVICE_GET_INFO

On Fri, Feb 10, 2023 at 11:10:34AM +0000, Joao Martins wrote:
> On 10/02/2023 07:55, Tian, Kevin wrote:
> >> From: Liu, Yi L <[email protected]>
> >> Sent: Thursday, February 9, 2023 12:17 PM
> >> + * zeroed if the user buffer is larger than the data kernel has.
> >> + *
> >> + * The type specific data would be used to sync capability between the
> >> + * vIOMMU and the hardware IOMMU, also for the availabillity checking of
> >> + * iommu hardware features like dirty page tracking in I/O page table.
> >
> > It's fine to report format information related to stage-1 page table
> > which userspace manages.
> >
> > but IMHO this should not be an interface to report which capability is
> > supported by iommufd. Having hardware supporting dirty bit
> > doesn't mean the underlying iommu driver provides necessary support
> > to iommufd dirty tracking.
> >
>
> +1
>
> In fact this shouldn't really be a way to check any capability as we are dumping
> straight away the IOMMU hardware registers. By dumping raw IOMMU hardware data,
> forces the application to understand IOMMU hardware specific formats. Maybe
> that's OK for hw nesting as there's a lot of info you need to know for the
> vIOMMU pgtables, pasid and etc so both are tightly coupled. But it is a bit
> disconnected from what really the software (IOMMUFD) and driver can use, without
> getting onto assumptions.
>
> [Calling it IOMMU_DEVICE_GET_HW_INFO would be bit more obvious if you're not
> asking IOMMUFD support]
>
> For capability checking, I think this really should be returning capabilities
> that both IOMMU driver supports ... and that IOMMUFD understands and marshalled
> on a format of its own defined by IOMMUFD. Maybe using IOMMU_CAP or some other
> thing within the kernel (now that's per-device), or even simpler. That's at
> least had written initially for the dirty tracking series.

Yes, the design of IOMMU_DEVICE_GET_INFO is already split. The HW
specific structure should only contain things related to operating the
HW specific paths (ie other HW specific structures in other APIs)

The enclosing struct should have general information about operating
the device through the abstract API - like dirty tracking.

But it may be that HW will individually report dirty tracking
capabilties related to special page table types - ie can the S2 page
table do dirty tracking

Jason

2023-02-10 21:00:02

[permalink] [raw]

Subject: Re: [PATCH 3/6] iommufd: Add IOMMU_DEVICE_GET_INFO

On Fri, Feb 10, 2023 at 07:55:42AM +0000, Tian, Kevin wrote:
> > From: Liu, Yi L <[email protected]>
> > Sent: Thursday, February 9, 2023 12:17 PM
> >
> > +int iommufd_device_get_info(struct iommufd_ucmd *ucmd)
> > +{
> > + struct iommu_device_info *cmd = ucmd->cmd;
> > + struct iommufd_object *dev_obj;
> > + struct device *dev;
> > + const struct iommu_ops *ops;
> > + void *data;
> > + unsigned int length, data_len;
> > + int rc;
> > +
> > + if (cmd->flags || cmd->__reserved || !cmd->data_len)
> > + return -EOPNOTSUPP;
>
> do we want !cmd->data_len being a way to check how many bytes are
> required in a following call to get the vendor info?

There is no reason, if the userspace doesn't have a struct large
enough then it also doesn't know what the extra bytes should even be
used for. No reason to read the.

> > +struct iommu_device_info {
> > + __u32 size;
> > + __u32 flags;
> > + __u32 dev_id;
> > + __u32 data_len;
> > + __aligned_u64 data_ptr;
>
> moving forward iommu hw cap is not the only information reported
> via this interface, e.g. it can be also used to report pasid mode.
>
> from this angle it's better to rename above two fields to be iommu
> specific, e.g.:
>
> __u32 iommu_data_len;
> __aligned_u64 iommu_data_ptr;

maybe call it hw info

Jason

2023-02-10 22:45:10

by Alex Williamson

[permalink] [raw]

Subject: Re: [PATCH 2/6] iommu/vt-d: Implement hw_info for iommu capability query

On Wed, 8 Feb 2023 20:16:38 -0800
Yi Liu <[email protected]> wrote:

> From: Lu Baolu <[email protected]>
>
> To support nested translation in the userspace, it should check the
> underlying hardware information for the capabilities.
>
> Add intel_iommu_hw_info() to report cap_reg and ecap_reg information.
>
> Signed-off-by: Lu Baolu <[email protected]>
> Signed-off-by: Yi Liu <[email protected]>
> Signed-off-by: Nicolin Chen <[email protected]>
> Signed-off-by: Yi Sun <[email protected]>
> ---
> drivers/iommu/intel/iommu.c | 19 +++++++++++++++++++
> drivers/iommu/intel/iommu.h | 1 +
> include/uapi/linux/iommufd.h | 21 +++++++++++++++++++++
> 3 files changed, 41 insertions(+)
>
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 59df7e42fd53..929f600cc350 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -4760,8 +4760,26 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid)
> intel_pasid_tear_down_entry(iommu, dev, pasid, false);
> }
>
> +static void *intel_iommu_hw_info(struct device *dev, u32 *length)
> +{
> + struct device_domain_info *info = dev_iommu_priv_get(dev);
> + struct intel_iommu *iommu = info->iommu;
> + struct iommu_device_info_vtd *vtd;
> +
> + vtd = kzalloc(sizeof(*vtd), GFP_KERNEL);
> + if (!vtd)
> + return ERR_PTR(-ENOMEM);
> +
> + vtd->cap_reg = iommu->cap;
> + vtd->ecap_reg = iommu->ecap;

Just a friendly reminder that these registers are already exposed to
userspace under /sys/class/iommu/ and each device has an iommu link
back to their iommu device there. This series doesn't really stand on
its own without some discussion of why that interface is not sufficient
for this use case. Thanks,

Alex

2023-02-11 00:15:58

[permalink] [raw]

Subject: Re: [PATCH 2/6] iommu/vt-d: Implement hw_info for iommu capability query

On Fri, Feb 10, 2023 at 03:44:10PM -0700, Alex Williamson wrote:
> On Wed, 8 Feb 2023 20:16:38 -0800
> Yi Liu <[email protected]> wrote:
>
> > From: Lu Baolu <[email protected]>
> >
> > To support nested translation in the userspace, it should check the
> > underlying hardware information for the capabilities.
> >
> > Add intel_iommu_hw_info() to report cap_reg and ecap_reg information.
> >
> > Signed-off-by: Lu Baolu <[email protected]>
> > Signed-off-by: Yi Liu <[email protected]>
> > Signed-off-by: Nicolin Chen <[email protected]>
> > Signed-off-by: Yi Sun <[email protected]>
> > ---
> > drivers/iommu/intel/iommu.c | 19 +++++++++++++++++++
> > drivers/iommu/intel/iommu.h | 1 +
> > include/uapi/linux/iommufd.h | 21 +++++++++++++++++++++
> > 3 files changed, 41 insertions(+)
> >
> > diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> > index 59df7e42fd53..929f600cc350 100644
> > --- a/drivers/iommu/intel/iommu.c
> > +++ b/drivers/iommu/intel/iommu.c
> > @@ -4760,8 +4760,26 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid)
> > intel_pasid_tear_down_entry(iommu, dev, pasid, false);
> > }
> >
> > +static void *intel_iommu_hw_info(struct device *dev, u32 *length)
> > +{
> > + struct device_domain_info *info = dev_iommu_priv_get(dev);
> > + struct intel_iommu *iommu = info->iommu;
> > + struct iommu_device_info_vtd *vtd;
> > +
> > + vtd = kzalloc(sizeof(*vtd), GFP_KERNEL);
> > + if (!vtd)
> > + return ERR_PTR(-ENOMEM);
> > +
> > + vtd->cap_reg = iommu->cap;
> > + vtd->ecap_reg = iommu->ecap;
>
> Just a friendly reminder that these registers are already exposed to
> userspace under /sys/class/iommu/ and each device has an iommu link
> back to their iommu device there.

I think in cases of mdevs w/ PASID (eg SIOV) it is not general to get
from vfio sysfs to the sysfs path for the iommu.

> This series doesn't really stand on its own without some discussion
> of why that interface is not sufficient for this use case.

IMHO I don't really like the idea of mixing iommufd with sysfs, it
should stand on its own.

In particular there is no generic way to go from a iommufd dev_id to
any sysfs path, userspace would need prior unique knowledge about the
subsystem that is being bound to iommufd first.

So, I think some of those explanations would be good in the commit
message?

I would also add explanation about what userspace is supposed to do
with these bits when it operates the nesting feature.

Jason

2023-02-11 03:38:15

[permalink] [raw]

Subject: Re: [PATCH 1/6] iommu: Add new iommu op to get iommu hardware information

On 2023/2/10 15:28, Tian, Kevin wrote:
>> From: Liu, Yi L <[email protected]>
>> Sent: Thursday, February 9, 2023 12:17 PM
>> @@ -223,6 +224,11 @@ struct iommu_iotlb_gather {
>> /**
>> * struct iommu_ops - iommu ops and capabilities
>> * @capable: check capability
>> + * @hw_info: IOMMU hardware capabilities. The type of the returned data
>> is
>
> is it 'info' or 'capability'?
>
>> + * defined in include/uapi/linux/iommufd.h. The data buffer is
>> + * allocated in the IOMMU driver and the caller should free it
>> + * after use. Return the data buffer if success, or ERR_PTR on
>> + * failure.
>> * @domain_alloc: allocate iommu domain
>> * @probe_device: Add device to iommu driver handling
>> * @release_device: Remove device from iommu driver handling
>> @@ -252,6 +258,7 @@ struct iommu_iotlb_gather {
>> */
>> struct iommu_ops {
>> bool (*capable)(struct device *dev, enum iommu_cap);
>> + void *(*hw_info)(struct device *dev, u32 *length);
>>
>> /* Domain allocation and freeing by the iommu driver */
>> struct iommu_domain *(*domain_alloc)(unsigned
>> iommu_domain_type);
>> @@ -280,6 +287,7 @@ struct iommu_ops {
>> void (*remove_dev_pasid)(struct device *dev, ioasid_t pasid);
>>
>> const struct iommu_domain_ops *default_domain_ops;
>> + enum iommu_device_data_type driver_type;
>
> the enum is called 'device_data_type' while the field is called 'driver_type'.

The enum is named from uAPI's p-o-v and driver_type is from IOMMU's.

> btw this new field is not documented above.

Yes. Will do that.

Best regards,
baolu

2023-02-11 03:42:56

[permalink] [raw]

Subject: Re: [PATCH 1/6] iommu: Add new iommu op to get iommu hardware information

2023-02-11 03:45:55

[permalink] [raw]

Subject: Re: [PATCH 2/6] iommu/vt-d: Implement hw_info for iommu capability query

On 2023/2/10 15:32, Tian, Kevin wrote:
>> From: Liu, Yi L <[email protected]>
>> Sent: Thursday, February 9, 2023 12:17 PM
>>
>> From: Lu Baolu <[email protected]>
>>
>> To support nested translation in the userspace, it should check the
>> underlying hardware information for the capabilities.
>
> remove this sentence. that belongs to next patch.
>
>> +
>> +/**
>> + * struct iommu_device_info_vtd - Intel VT-d device info
>> + *
>> + * @flags: Must be set to 0
>> + * @__reserved: Must be 0
>> + * @cap_reg: Value of Intel VT-d capability register defined in chapter
>> + * 11.4.2 of Intel VT-d spec.
>> + * @ecap_reg: Value of Intel VT-d capability register defined in chapter
>> + * 11.4.3 of Intel VT-d spec.
>
> let's be specific with section name e.g. "11.4.2 Capability Register" so if
> the chapter index is changed later people can still know where to look
> at.
>
>> + *
>> + * Intel hardware iommu capability.
>
> duplicated with the first line "Intel VT-d device info"

Above all agreed.

Best regards,
baolu

2023-02-13 01:54:24

[permalink] [raw]

Subject: RE: [PATCH 1/6] iommu: Add new iommu op to get iommu hardware information

> From: Baolu Lu <[email protected]>
> Sent: Saturday, February 11, 2023 11:43 AM
>
> On 2023/2/10 15:28, Tian, Kevin wrote:
> >> From: Liu, Yi L<[email protected]>
> >> Sent: Thursday, February 9, 2023 12:17 PM
> >> @@ -223,6 +224,11 @@ struct iommu_iotlb_gather {
> >> /**
> >> * struct iommu_ops - iommu ops and capabilities
> >> * @capable: check capability
> >> + * @hw_info: IOMMU hardware capabilities. The type of the returned
> data
> >> is
> > is it 'info' or 'capability'?
>
> hw_info. IOMMU core does not care about specific content, so it is not
> necessary to define it as capability or anything else.
>
> Perhaps we need to change the comments a bit, say, "IOMMU hardware
> information"?
>

yes, that is probably more consistent.

2023-02-13 02:04:29

[permalink] [raw]

Subject: RE: [PATCH 3/6] iommufd: Add IOMMU_DEVICE_GET_INFO

> From: Jason Gunthorpe <[email protected]>
> Sent: Saturday, February 11, 2023 5:00 AM
>
>
> > > +struct iommu_device_info {
> > > + __u32 size;
> > > + __u32 flags;
> > > + __u32 dev_id;
> > > + __u32 data_len;
> > > + __aligned_u64 data_ptr;
> >
> > moving forward iommu hw cap is not the only information reported
> > via this interface, e.g. it can be also used to report pasid mode.
> >
> > from this angle it's better to rename above two fields to be iommu
> > specific, e.g.:
> >
> > __u32 iommu_data_len;
> > __aligned_u64 iommu_data_ptr;
>
> maybe call it hw info
>

that is fine given we already have 'struct iommu_device_info'.

probably this cmd should be named as IOMMU_DEVIEC_GET_IOMMU_INFO
to be more accurate.

2023-02-13 02:36:50

by Binbin Wu

[permalink] [raw]

Subject: Re: [PATCH 1/6] iommu: Add new iommu op to get iommu hardware information

On 2/9/2023 12:16 PM, Yi Liu wrote:
> From: Lu Baolu <[email protected]>
>
> Introduce a new iommu op get

get -> to get

> the IOMMU hardware capabilities for iommufd.
> This information will be used by any vIOMMU driver which is owned by
> userspace.
>
> This op chooses to make the special parameters opaque to the core. This
> suits the current usage model where accessing any of the IOMMU device
> special parameters does require a userspace driver that matches the kernel
> driver. If a need for common parameters, implemented similarly by several
> drivers, arises then there is room in the design to grow a generic parameter
> set as well. No warpper

warpper -> wrapper

> API is added as it is supposed to be used by
> iommufd only.
>
> Signed-off-by: Lu Baolu <[email protected]>
> Signed-off-by: Yi Liu <[email protected]>
> Signed-off-by: Nicolin Chen <[email protected]>
> ---
> include/linux/iommu.h | 8 ++++++++
> include/uapi/linux/iommufd.h | 6 ++++++
> 2 files changed, 14 insertions(+)
>
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> index cb586d054c57..97b398d19fd2 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -15,6 +15,7 @@
> #include <linux/of.h>
> #include <linux/ioasid.h>
> #include <uapi/linux/iommu.h>
> +#include <uapi/linux/iommufd.h>
>
> #define IOMMU_READ (1 << 0)
> #define IOMMU_WRITE (1 << 1)
> @@ -223,6 +224,11 @@ struct iommu_iotlb_gather {
> /**
> * struct iommu_ops - iommu ops and capabilities
> * @capable: check capability
> + * @hw_info: IOMMU hardware capabilities. The type of the returned data is
> + * defined in include/uapi/linux/iommufd.h. The data buffer is
> + * allocated in the IOMMU driver and the caller should free it
> + * after use. Return the data buffer if success, or ERR_PTR on
> + * failure.
> * @domain_alloc: allocate iommu domain
> * @probe_device: Add device to iommu driver handling
> * @release_device: Remove device from iommu driver handling
> @@ -252,6 +258,7 @@ struct iommu_iotlb_gather {
> */
> struct iommu_ops {
> bool (*capable)(struct device *dev, enum iommu_cap);
> + void *(*hw_info)(struct device *dev, u32 *length);
>
> /* Domain allocation and freeing by the iommu driver */
> struct iommu_domain *(*domain_alloc)(unsigned iommu_domain_type);
> @@ -280,6 +287,7 @@ struct iommu_ops {
> void (*remove_dev_pasid)(struct device *dev, ioasid_t pasid);
>
> const struct iommu_domain_ops *default_domain_ops;
> + enum iommu_device_data_type driver_type;

How to understand the name "iommu_device_data_type"?
Is it just refer to the driver types or it has a more generic meaning?

> unsigned long pgsize_bitmap;
> struct module *owner;
> };
> diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h
> index 98ebba80cfa1..2309edb55028 100644
> --- a/include/uapi/linux/iommufd.h
> +++ b/include/uapi/linux/iommufd.h
> @@ -344,4 +344,10 @@ struct iommu_vfio_ioas {
> __u16 __reserved;
> };
> #define IOMMU_VFIO_IOAS _IO(IOMMUFD_TYPE, IOMMUFD_CMD_VFIO_IOAS)
> +
> +/**
> + * enum iommu_device_data_type - IOMMU hardware Data types
> + */
> +enum iommu_device_data_type {
> +};
> #endif

2023-02-13 03:09:53

by Binbin Wu

[permalink] [raw]

Subject: Re: [PATCH 2/6] iommu/vt-d: Implement hw_info for iommu capability query

On 2/9/2023 12:16 PM, Yi Liu wrote:
> From: Lu Baolu <[email protected]>
>
> To support nested translation in the userspace, it should check the
> underlying hardware information for the capabilities.
>
> Add intel_iommu_hw_info() to report cap_reg and ecap_reg information.
>
> Signed-off-by: Lu Baolu <[email protected]>
> Signed-off-by: Yi Liu <[email protected]>
> Signed-off-by: Nicolin Chen <[email protected]>
> Signed-off-by: Yi Sun <[email protected]>
> ---
> drivers/iommu/intel/iommu.c | 19 +++++++++++++++++++
> drivers/iommu/intel/iommu.h | 1 +
> include/uapi/linux/iommufd.h | 21 +++++++++++++++++++++
> 3 files changed, 41 insertions(+)
>
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 59df7e42fd53..929f600cc350 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -4760,8 +4760,26 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid)
> intel_pasid_tear_down_entry(iommu, dev, pasid, false);
> }
>
> +static void *intel_iommu_hw_info(struct device *dev, u32 *length)
> +{
> + struct device_domain_info *info = dev_iommu_priv_get(dev);
> + struct intel_iommu *iommu = info->iommu;
> + struct iommu_device_info_vtd *vtd;
> +
> + vtd = kzalloc(sizeof(*vtd), GFP_KERNEL);
> + if (!vtd)
> + return ERR_PTR(-ENOMEM);
> +
> + vtd->cap_reg = iommu->cap;
> + vtd->ecap_reg = iommu->ecap;
> + *length = sizeof(*vtd);
> +
> + return vtd;
> +}
> +
> const struct iommu_ops intel_iommu_ops = {
> .capable = intel_iommu_capable,
> + .hw_info = intel_iommu_hw_info,
> .domain_alloc = intel_iommu_domain_alloc,
> .probe_device = intel_iommu_probe_device,
> .probe_finalize = intel_iommu_probe_finalize,
> @@ -4774,6 +4792,7 @@ const struct iommu_ops intel_iommu_ops = {
> .def_domain_type = device_def_domain_type,
> .remove_dev_pasid = intel_iommu_remove_dev_pasid,
> .pgsize_bitmap = SZ_4K,
> + .driver_type = IOMMU_DEVICE_DATA_INTEL_VTD,
> #ifdef CONFIG_INTEL_IOMMU_SVM
> .page_response = intel_svm_page_response,
> #endif
> diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
> index 06e61e474856..2e70265d4ceb 100644
> --- a/drivers/iommu/intel/iommu.h
> +++ b/drivers/iommu/intel/iommu.h
> @@ -22,6 +22,7 @@
> #include <linux/ioasid.h>
> #include <linux/bitfield.h>
> #include <linux/xarray.h>
> +#include <uapi/linux/iommufd.h>
>
> #include <asm/cacheflush.h>
> #include <asm/iommu.h>
> diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h
> index 2309edb55028..fda75c8450ee 100644
> --- a/include/uapi/linux/iommufd.h
> +++ b/include/uapi/linux/iommufd.h
> @@ -347,7 +347,28 @@ struct iommu_vfio_ioas {
>
> /**
> * enum iommu_device_data_type - IOMMU hardware Data types
> + * @IOMMU_DEVICE_DATA_INTEL_VTD: Intel VT-d iommu data type
> */
> enum iommu_device_data_type {
> + IOMMU_DEVICE_DATA_INTEL_VTD = 1,
> +};
> +
> +/**
> + * struct iommu_device_info_vtd - Intel VT-d device info
> + *
> + * @flags: Must be set to 0

Could you add more description about the usage of flags here?

> + * @__reserved: Must be 0
> + * @cap_reg: Value of Intel VT-d capability register defined in chapter
> + * 11.4.2 of Intel VT-d spec.
> + * @ecap_reg: Value of Intel VT-d capability register defined in chapter
> + * 11.4.3 of Intel VT-d spec.
> + *
> + * Intel hardware iommu capability.
> + */
> +struct iommu_device_info_vtd {
> + __u32 flags;
> + __u32 __reserved;
> + __aligned_u64 cap_reg;
> + __aligned_u64 ecap_reg;
> };
> #endif

2023-02-13 08:46:58

[permalink] [raw]

Subject: Re: [PATCH 1/6] iommu: Add new iommu op to get iommu hardware information

On 2023/2/13 10:36, Binbin Wu wrote:
>
> On 2/9/2023 12:16 PM, Yi Liu wrote:
>> From: Lu Baolu <[email protected]>
>>
>> Introduce a new iommu op get
>
> get -> to get
>
>
>> the IOMMU hardware capabilities for iommufd.
>> This information will be used by any vIOMMU driver which is owned by
>> userspace.
>>
>> This op chooses to make the special parameters opaque to the core. This
>> suits the current usage model where accessing any of the IOMMU device
>> special parameters does require a userspace driver that matches the
>> kernel
>> driver. If a need for common parameters, implemented similarly by several
>> drivers, arises then there is room in the design to grow a generic
>> parameter
>> set as well. No warpper
>
> warpper -> wrapper

Thanks, will update.

>
>> API is added as it is supposed to be used by
>> iommufd only.
>>
>> Signed-off-by: Lu Baolu <[email protected]>
>> Signed-off-by: Yi Liu <[email protected]>
>> Signed-off-by: Nicolin Chen <[email protected]>
>> ---
>> include/linux/iommu.h        | 8 ++++++++
>> include/uapi/linux/iommufd.h | 6 ++++++
>> 2 files changed, 14 insertions(+)
>>
>> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
>> index cb586d054c57..97b398d19fd2 100644
>> --- a/include/linux/iommu.h
>> +++ b/include/linux/iommu.h
>> @@ -15,6 +15,7 @@
>> #include <linux/of.h>
>> #include <linux/ioasid.h>
>> #include <uapi/linux/iommu.h>
>> +#include <uapi/linux/iommufd.h>
>> #define IOMMU_READ    (1 << 0)
>> #define IOMMU_WRITE    (1 << 1)
>> @@ -223,6 +224,11 @@ struct iommu_iotlb_gather {
>> /**
>>    * struct iommu_ops - iommu ops and capabilities
>>    * @capable: check capability
>> + * @hw_info: IOMMU hardware capabilities. The type of the returned
>> data is
>> + *           defined in include/uapi/linux/iommufd.h. The data buffer is
>> + *           allocated in the IOMMU driver and the caller should free it
>> + *           after use. Return the data buffer if success, or ERR_PTR on
>> + *          failure.
>>    * @domain_alloc: allocate iommu domain
>>    * @probe_device: Add device to iommu driver handling
>>    * @release_device: Remove device from iommu driver handling
>> @@ -252,6 +258,7 @@ struct iommu_iotlb_gather {
>>    */
>> struct iommu_ops {
>>       bool (*capable)(struct device *dev, enum iommu_cap);
>> +    void *(*hw_info)(struct device *dev, u32 *length);
>>       /* Domain allocation and freeing by the iommu driver */
>>       struct iommu_domain *(*domain_alloc)(unsigned iommu_domain_type);
>> @@ -280,6 +287,7 @@ struct iommu_ops {
>>       void (*remove_dev_pasid)(struct device *dev, ioasid_t pasid);
>>       const struct iommu_domain_ops *default_domain_ops;
>> +    enum iommu_device_data_type driver_type;
>
>
> How to understand the name "iommu_device_data_type"?

This is to tell userspace which type of IOMMU the returned information
comes from, Intel VT-d, ARM or others...

> Is it just refer to the driver types or it has a more generic meaning?

IOMMU driver sets this field to distinguish different IOMMU hardware.

Best regards,
baolu

2023-02-13 08:49:22

[permalink] [raw]

Subject: Re: [PATCH 2/6] iommu/vt-d: Implement hw_info for iommu capability query

On 2023/2/13 11:09, Binbin Wu wrote:
>
> On 2/9/2023 12:16 PM, Yi Liu wrote:
>> From: Lu Baolu <[email protected]>
>>
>> To support nested translation in the userspace, it should check the
>> underlying hardware information for the capabilities.
>>
>> Add intel_iommu_hw_info() to report cap_reg and ecap_reg information.
>>
>> Signed-off-by: Lu Baolu <[email protected]>
>> Signed-off-by: Yi Liu <[email protected]>
>> Signed-off-by: Nicolin Chen <[email protected]>
>> Signed-off-by: Yi Sun <[email protected]>
>> ---
>> drivers/iommu/intel/iommu.c | 19 +++++++++++++++++++
>> drivers/iommu/intel/iommu.h | 1 +
>> include/uapi/linux/iommufd.h | 21 +++++++++++++++++++++
>> 3 files changed, 41 insertions(+)
>>
>> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>> index 59df7e42fd53..929f600cc350 100644
>> --- a/drivers/iommu/intel/iommu.c
>> +++ b/drivers/iommu/intel/iommu.c
>> @@ -4760,8 +4760,26 @@ static void intel_iommu_remove_dev_pasid(struct
>> device *dev, ioasid_t pasid)
>>       intel_pasid_tear_down_entry(iommu, dev, pasid, false);
>> }
>> +static void *intel_iommu_hw_info(struct device *dev, u32 *length)
>> +{
>> +    struct device_domain_info *info = dev_iommu_priv_get(dev);
>> +    struct intel_iommu *iommu = info->iommu;
>> +    struct iommu_device_info_vtd *vtd;
>> +
>> +    vtd = kzalloc(sizeof(*vtd), GFP_KERNEL);
>> +    if (!vtd)
>> +        return ERR_PTR(-ENOMEM);
>> +
>> +    vtd->cap_reg = iommu->cap;
>> +    vtd->ecap_reg = iommu->ecap;
>> +    *length = sizeof(*vtd);
>> +
>> +    return vtd;
>> +}
>> +
>> const struct iommu_ops intel_iommu_ops = {
>>       .capable        = intel_iommu_capable,
>> +    .hw_info        = intel_iommu_hw_info,
>>       .domain_alloc        = intel_iommu_domain_alloc,
>>       .probe_device        = intel_iommu_probe_device,
>>       .probe_finalize        = intel_iommu_probe_finalize,
>> @@ -4774,6 +4792,7 @@ const struct iommu_ops intel_iommu_ops = {
>>       .def_domain_type    = device_def_domain_type,
>>       .remove_dev_pasid    = intel_iommu_remove_dev_pasid,
>>       .pgsize_bitmap        = SZ_4K,
>> +    .driver_type        = IOMMU_DEVICE_DATA_INTEL_VTD,
>> #ifdef CONFIG_INTEL_IOMMU_SVM
>>       .page_response        = intel_svm_page_response,
>> #endif
>> diff --git a/drivers/iommu/intel/iommu.h b/drivers/iommu/intel/iommu.h
>> index 06e61e474856..2e70265d4ceb 100644
>> --- a/drivers/iommu/intel/iommu.h
>> +++ b/drivers/iommu/intel/iommu.h
>> @@ -22,6 +22,7 @@
>> #include <linux/ioasid.h>
>> #include <linux/bitfield.h>
>> #include <linux/xarray.h>
>> +#include <uapi/linux/iommufd.h>
>> #include <asm/cacheflush.h>
>> #include <asm/iommu.h>
>> diff --git a/include/uapi/linux/iommufd.h b/include/uapi/linux/iommufd.h
>> index 2309edb55028..fda75c8450ee 100644
>> --- a/include/uapi/linux/iommufd.h
>> +++ b/include/uapi/linux/iommufd.h
>> @@ -347,7 +347,28 @@ struct iommu_vfio_ioas {
>> /**
>>    * enum iommu_device_data_type - IOMMU hardware Data types
>> + * @IOMMU_DEVICE_DATA_INTEL_VTD: Intel VT-d iommu data type
>>    */
>> enum iommu_device_data_type {
>> +    IOMMU_DEVICE_DATA_INTEL_VTD = 1,
>> +};
>> +
>> +/**
>> + * struct iommu_device_info_vtd - Intel VT-d device info
>> + *
>> + * @flags: Must be set to 0
>
> Could you add more description about the usage of flags here?

This is a Reserved 0 fields for other coming features.

Best regards,
baolu

2023-02-13 14:51:52

[permalink] [raw]

Subject: Re: [PATCH 2/6] iommu/vt-d: Implement hw_info for iommu capability query

On Wed, Feb 08, 2023 at 08:16:38PM -0800, Yi Liu wrote:
> From: Lu Baolu <[email protected]>
>
> To support nested translation in the userspace, it should check the
> underlying hardware information for the capabilities.
>
> Add intel_iommu_hw_info() to report cap_reg and ecap_reg information.
>
> Signed-off-by: Lu Baolu <[email protected]>
> Signed-off-by: Yi Liu <[email protected]>
> Signed-off-by: Nicolin Chen <[email protected]>
> Signed-off-by: Yi Sun <[email protected]>
> ---
> drivers/iommu/intel/iommu.c | 19 +++++++++++++++++++
> drivers/iommu/intel/iommu.h | 1 +
> include/uapi/linux/iommufd.h | 21 +++++++++++++++++++++
> 3 files changed, 41 insertions(+)

This should come after the next patch in the series

Jason

2023-02-22 21:08:18