Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH] iommu: check for the deferred attach when attaching a
 device
From:   lijiang <lijiang@redhat.com>
To:     linux-kernel@vger.kernel.org
Cc:     joro@8bytes.org, will@kernel.org, iommu@lists.linux-foundation.org,
        jroedel@suse.de, jsnitsel@redhat.com,
        "Lendacky, Thomas" <thomas.lendacky@amd.com>
References: <20201226053959.4222-1-lijiang@redhat.com>
 <33b6f925-71e6-5d9e-74c3-3e1eaf13398e@redhat.com>
Message-ID: <b385db3b-4506-6d75-49e1-e11064e65d6a@redhat.com>
Date:   Tue, 5 Jan 2021 15:52:06 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.9.1
MIME-Version: 1.0
In-Reply-To: <33b6f925-71e6-5d9e-74c3-3e1eaf13398e@redhat.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk

在 2021年01月05日 11:55, lijiang 写道:
> Hi,
> 
> Also add Joerg to cc list.
> 

Also add more people to cc list, Jerry Snitselaar and Tom Lendacky.

Thanks.

> Thanks.
> Lianbo
> 在 2020年12月26日 13:39, Lianbo Jiang 写道:
>> Currently, because domain attach allows to be deferred from iommu
>> driver to device driver, and when iommu initializes, the devices
>> on the bus will be scanned and the default groups will be allocated.
>>
>> Due to the above changes, some devices could be added to the same
>> group as below:
>>
>> [    3.859417] pci 0000:01:00.0: Adding to iommu group 16
>> [    3.864572] pci 0000:01:00.1: Adding to iommu group 16
>> [    3.869738] pci 0000:02:00.0: Adding to iommu group 17
>> [    3.874892] pci 0000:02:00.1: Adding to iommu group 17
>>
>> But when attaching these devices, it doesn't allow that a group has
>> more than one device, otherwise it will return an error. This conflicts
>> with the deferred attaching. Unfortunately, it has two devices in the
>> same group for my side, for example:
>>
>> [    9.627014] iommu_group_device_count(): device name[0]:0000:01:00.0
>> [    9.633545] iommu_group_device_count(): device name[1]:0000:01:00.1
>> ...
>> [   10.255609] iommu_group_device_count(): device name[0]:0000:02:00.0
>> [   10.262144] iommu_group_device_count(): device name[1]:0000:02:00.1
>>
>> Finally, which caused the failure of tg3 driver when tg3 driver calls
>> the dma_alloc_coherent() to allocate coherent memory in the tg3_test_dma().
>>
>> [    9.660310] tg3 0000:01:00.0: DMA engine test failed, aborting
>> [    9.754085] tg3: probe of 0000:01:00.0 failed with error -12
>> [    9.997512] tg3 0000:01:00.1: DMA engine test failed, aborting
>> [   10.043053] tg3: probe of 0000:01:00.1 failed with error -12
>> [   10.288905] tg3 0000:02:00.0: DMA engine test failed, aborting
>> [   10.334070] tg3: probe of 0000:02:00.0 failed with error -12
>> [   10.578303] tg3 0000:02:00.1: DMA engine test failed, aborting
>> [   10.622629] tg3: probe of 0000:02:00.1 failed with error -12
>>
>> In addition, the similar situations also occur in other drivers such
>> as the bnxt_en driver. That can be reproduced easily in kdump kernel
>> when SME is active.
>>
>> Add a check for the deferred attach in the iommu_attach_device() and
>> allow to attach the deferred device regardless of how many devices
>> are in a group.
>>
>> Signed-off-by: Lianbo Jiang <lijiang@redhat.com>
>> ---
>>  drivers/iommu/iommu.c | 5 ++++-
>>  1 file changed, 4 insertions(+), 1 deletion(-)
>>
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index ffeebda8d6de..dccab7b133fb 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -1967,8 +1967,11 @@ int iommu_attach_device(struct iommu_domain *domain, struct device *dev)
>>  	 */
>>  	mutex_lock(&group->mutex);
>>  	ret = -EINVAL;
>> -	if (iommu_group_device_count(group) != 1)
>> +	if (!iommu_is_attach_deferred(domain, dev) &&
>> +	    iommu_group_device_count(group) != 1) {
>> +		dev_err_ratelimited(dev, "Group has more than one device\n");
>>  		goto out_unlock;
>> +	}
>>  
>>  	ret = __iommu_attach_group(domain, group);
>>  
>>