Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 25 Aug 2022 09:22:08 +0200
From:   Alexander Gordeev <agordeev@linux.ibm.com>
To:     Matthew Rosato <mjrosato@linux.ibm.com>
Cc:     Pierre Morel <pmorel@linux.ibm.com>, iommu@lists.linux.dev,
        linux-s390@vger.kernel.org, schnelle@linux.ibm.com,
        borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com,
        gerald.schaefer@linux.ibm.com, svens@linux.ibm.com,
        joro@8bytes.org, will@kernel.org, robin.murphy@arm.com,
        jgg@nvidia.com, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] iommu/s390: Fix race with release_device ops
Message-ID: <YwcjIPcT1b0uyyFn@li-4a3a4a4c-28e5-11b2-a85c-a8d192c6f089.ibm.com>
References: <20220823203059.81919-1-mjrosato@linux.ibm.com>
 <a6e42442-d9cb-0d63-bb71-da78a5669a51@linux.ibm.com>
 <04644ee5-2386-1f3d-c1a3-fc4227570cf7@linux.ibm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <04644ee5-2386-1f3d-c1a3-fc4227570cf7@linux.ibm.com>
Precedence: bulk

On Wed, Aug 24, 2022 at 04:25:19PM -0400, Matthew Rosato wrote:
> >> @@ -90,15 +90,39 @@ static int s390_iommu_attach_device(struct iommu_domain *domain,
> >> ????? struct zpci_dev *zdev = to_zpci_dev(dev);
> >> ????? struct s390_domain_device *domain_device;
> >> ????? unsigned long flags;
> >> -??? int cc, rc;
> >> +??? int cc, rc = 0;
> >> ? ????? if (!zdev)
> >> ????????? return -ENODEV;
> >> ? +??? /* First check compatibility */
> >> +??? spin_lock_irqsave(&s390_domain->list_lock, flags);
> >> +??? /* First device defines the DMA range limits */
> >> +??? if (list_empty(&s390_domain->devices)) {
> >> +??????? domain->geometry.aperture_start = zdev->start_dma;
> >> +??????? domain->geometry.aperture_end = zdev->end_dma;
> >> +??????? domain->geometry.force_aperture = true;
> >> +??? /* Allow only devices with identical DMA range limits */
> >> +??? } else if (domain->geometry.aperture_start != zdev->start_dma ||
> >> +?????????? domain->geometry.aperture_end != zdev->end_dma) {
> >> +??????? rc = -EINVAL;
> >> +??? }
> >> +??? spin_unlock_irqrestore(&s390_domain->list_lock, flags);
> >> +??? if (rc)
> >> +??????? return rc;
> >> +
> >> ????? domain_device = kzalloc(sizeof(*domain_device), GFP_KERNEL);
> >> ????? if (!domain_device)
> >> ????????? return -ENOMEM;
> >> ? +??? /* Leave now if the device has already been released */
> >> +??? spin_lock_irqsave(&zdev->dma_domain_lock, flags);
> >> +??? if (!dev_iommu_priv_get(dev)) {
> >> +??????? spin_unlock_irqrestore(&zdev->dma_domain_lock, flags);
> >> +??????? kfree(domain_device);
> >> +??????? return 0;
> >> +??? }
> >> +
> >> ????? if (zdev->dma_table && !zdev->s390_domain) {
> >> ????????? cc = zpci_dma_exit_device(zdev);
> >> ????????? if (cc) {
> > 
> > Am I wrong? It seems to me that zpci_dma_exit_device here is called with the spin_lock locked but this function zpci_dma_exit_device calls vfree which may sleep.
> > 
> 
> Oh, good point, I just enabled lockdep to verify that.
> 
> I think we could just replace this with a mutex instead, it's not a performance path.  I've been running tests successfully today with this patch modified to instead use a mutex for dma_domain_lock.

But your original version uses irq-savvy spinlocks.
Are there data that need to be protected against interrupts?

Thanks!