Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Cc:     baolu.lu@linux.intel.com, David Airlie <airlied@linux.ie>,
        linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org,
        Jonathan Hunter <jonathanh@nvidia.com>,
        Alex Williamson <alex.williamson@redhat.com>,
        Thierry Reding <thierry.reding@gmail.com>,
        Jacob jun Pan <jacob.jun.pan@intel.com>,
        Daniel Vetter <daniel@ffwll.ch>
Subject: Re: [PATCH 6/7] iommu: Use right way to retrieve iommu_ops
To:     Robin Murphy <robin.murphy@arm.com>,
        Joerg Roedel <joro@8bytes.org>,
        Jason Gunthorpe <jgg@nvidia.com>,
        Christoph Hellwig <hch@infradead.org>,
        Ben Skeggs <bskeggs@redhat.com>,
        Kevin Tian <kevin.tian@intel.com>,
        Ashok Raj <ashok.raj@intel.com>, Will Deacon <will@kernel.org>
References: <20220124071103.2097118-1-baolu.lu@linux.intel.com>
 <20220124071103.2097118-7-baolu.lu@linux.intel.com>
 <363b8f7d-5459-0d19-c1ac-a2c6bce9d26f@arm.com>
From:   Lu Baolu <baolu.lu@linux.intel.com>
Message-ID: <8d4fea6f-aa21-3703-a254-2594004c60a1@linux.intel.com>
Date:   Tue, 25 Jan 2022 11:54:15 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
MIME-Version: 1.0
In-Reply-To: <363b8f7d-5459-0d19-c1ac-a2c6bce9d26f@arm.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 1/25/22 8:20 AM, Robin Murphy wrote:
> On 2022-01-24 07:11, Lu Baolu wrote:
>> The common iommu_ops is hooked to both device and domain. When a helper
>> has both device and domain pointer, the way to get the iommu_ops looks
>> messy in iommu core. This sorts out the way to get iommu_ops. The device
>> related helpers go through device pointer, while the domain related ones
>> go through domain pointer.
>>
>> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
>> ---
>>   include/linux/iommu.h |  8 ++++++++
>>   drivers/iommu/iommu.c | 25 ++++++++++++++-----------
>>   2 files changed, 22 insertions(+), 11 deletions(-)
>>
>> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
>> index aa5486243892..111b3e9c79bb 100644
>> --- a/include/linux/iommu.h
>> +++ b/include/linux/iommu.h
>> @@ -385,6 +385,14 @@ static inline void iommu_iotlb_gather_init(struct 
>> iommu_iotlb_gather *gather)
>>       };
>>   }
>> +static inline const struct iommu_ops *dev_iommu_ops_get(struct device 
>> *dev)
>> +{
>> +    if (dev && dev->iommu && dev->iommu->iommu_dev)
>> +        return dev->iommu->iommu_dev->ops;
>> +
>> +    return NULL;
> 
> This probably warrants at least a WARN, but it's arguable to just assume 
> that valid ops must be installed if iommu_probe_device() has succeeded. 
> The device ops are essentially for internal use within the IOMMU 
> subsystem itself, so we should be able to trust ourselves not to misuse 
> the helper.

I agree that we could add a WARN() here. The expectation is that every
device going through the IOMMU interfaces or helpers should have been
probed by iommu_probe_device().

> 
>> +}
>> +
>>   #define IOMMU_BUS_NOTIFY_PRIORITY        0
>>   #define IOMMU_GROUP_NOTIFY_ADD_DEVICE        1 /* Device added */
>>   #define IOMMU_GROUP_NOTIFY_DEL_DEVICE        2 /* Pre Device removed */
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index 5230c6d90ece..6631e2ea44df 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -764,6 +764,7 @@ EXPORT_SYMBOL_GPL(iommu_group_set_name);
>>   static int iommu_create_device_direct_mappings(struct iommu_group 
>> *group,
>>                              struct device *dev)
>>   {
>> +    const struct iommu_ops *ops = dev_iommu_ops_get(dev);
>>       struct iommu_domain *domain = group->default_domain;
>>       struct iommu_resv_region *entry;
>>       struct list_head mappings;
>> @@ -785,8 +786,8 @@ static int 
>> iommu_create_device_direct_mappings(struct iommu_group *group,
>>           dma_addr_t start, end, addr;
>>           size_t map_size = 0;
>> -        if (domain->ops->apply_resv_region)
>> -            domain->ops->apply_resv_region(dev, domain, entry);
>> +        if (ops->apply_resv_region)
>> +            ops->apply_resv_region(dev, domain, entry);
> 
> Strictly I think this was a domain op, as it was about reserving the 
> IOVA range in the given DMA domain. Also taking the domain as an 
> argument is a bit of a giveaway. However it's now just dead code either 
> way since there are no remaining implementations, and no reason for any 
> new ones.

This callback is a dead code. I will cleanup it.

$ git grep apply_resv_region
drivers/iommu/iommu.c:          if (ops->apply_resv_region)
drivers/iommu/iommu.c:                  ops->apply_resv_region(dev, 
domain, entry);
include/linux/iommu.h: * @apply_resv_region: Temporary helper call-back 
for iova reserved ranges
include/linux/iommu.h:  void (*apply_resv_region)(struct device *dev,

> 
>>           start = ALIGN(entry->start, pg_size);
>>           end   = ALIGN(entry->start + entry->length, pg_size);
>> @@ -831,8 +832,10 @@ static int 
>> iommu_create_device_direct_mappings(struct iommu_group *group,
>>   static bool iommu_is_attach_deferred(struct iommu_domain *domain,
>>                        struct device *dev)
>>   {
>> -    if (domain->ops->is_attach_deferred)
>> -        return domain->ops->is_attach_deferred(domain, dev);
>> +    const struct iommu_ops *ops = dev_iommu_ops_get(dev);
>> +
>> +    if (ops->is_attach_deferred)
>> +        return ops->is_attach_deferred(domain, dev);
> 
> Similarly if this takes a domain as its first argument then it's de 
> facto a domain method. However, I'd concur that logically it *is* a 
> device op, so let's drop that (unused) domain argument if we're cleaning 
> up.
> 
> Maybe there's even an argument for factoring this out to a standard flag 
> in dev_iommu rather than an op at all?

Make it part of dev_iommu looks more attractive. Let me check how many
efforts will it take. If a lot of changes required, maybe we can remove
@domain in this series and then switch it to a dev_iommu flag in a
separated series.

> 
> The others covered here look OK - we can blame PCI for page response 
> being weirdly device-centric - however could we also convert all the 
> feasible instances of dev->bus->iommu_ops to dev_iommu_ops() as well?

Sure.

> (Subtly implying that I'm also not a fan of having "_get" in the name 
> for a non-refcounted lookup...) Obviously iommu_probe_device() and 
> iommmu_domain_alloc() still need bus ops at this point, but I'm working 
> on that... :)

Thanks and glad to know that.

> 
> Thanks,
> Robin.

Best regards,
baolu