Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 14 Jun 2023 10:19:49 -0700
From:   Jacob Pan <jacob.jun.pan@linux.intel.com>
To:     Baolu Lu <baolu.lu@linux.intel.com>
Cc:     LKML <linux-kernel@vger.kernel.org>, iommu@lists.linux.dev,
        Jason Gunthorpe <jgg@nvidia.com>,
        Joerg Roedel <joro@8bytes.org>,
        Robin Murphy <robin.murphy@arm.com>,
        Jean-Philippe Brucker <jean-philippe@linaro.com>,
        dmaengine@vger.kernel.org, vkoul@kernel.org,
        Will Deacon <will@kernel.org>,
        David Woodhouse <dwmw2@infradead.org>,
        Raj Ashok <ashok.raj@intel.com>,
        "Tian, Kevin" <kevin.tian@intel.com>, Yi Liu <yi.l.liu@intel.com>,
        "Yu, Fenghua" <fenghua.yu@intel.com>,
        Dave Jiang <dave.jiang@intel.com>,
        Tony Luck <tony.luck@intel.com>,
        "Zanussi, Tom" <tom.zanussi@intel.com>, rex.zhang@intel.com,
        xiaochen.shen@intel.com, narayan.ranganathan@intel.com,
        jacob.jun.pan@linux.intel.com
Subject: Re: [PATCH v8 2/7] iommu: Move global PASID allocation from SVA to
 core
Message-ID: <20230614101949.1164f33f@jacob-builder>
In-Reply-To: <b5a6a7ad-bfad-b550-f4be-f5ec0679f3d2@linux.intel.com>
References: <20230602182212.150825-1-jacob.jun.pan@linux.intel.com>
        <20230602182212.150825-3-jacob.jun.pan@linux.intel.com>
        <08830c11-5528-0c42-0bc3-89c3796611fe@linux.intel.com>
        <b5a6a7ad-bfad-b550-f4be-f5ec0679f3d2@linux.intel.com>
Organization: OTC
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

Hi Baolu,

On Tue, 13 Jun 2023 11:06:03 +0800, Baolu Lu <baolu.lu@linux.intel.com>
wrote:

> On 6/10/23 8:13 PM, Baolu Lu wrote:
> > On 6/3/23 2:22 AM, Jacob Pan wrote: =20
> >> +ioasid_t iommu_alloc_global_pasid_dev(struct device *dev)
> >> +{
> >> +=C2=A0=C2=A0=C2=A0 int ret;
> >> +=C2=A0=C2=A0=C2=A0 ioasid_t max;
> >> +
> >> +=C2=A0=C2=A0=C2=A0 max =3D dev->iommu->max_pasids;
> >> +=C2=A0=C2=A0=C2=A0 /*
> >> +=C2=A0=C2=A0=C2=A0=C2=A0 * max_pasids is set up by vendor driver base=
d on number of PASID=20
> >> bits
> >> +=C2=A0=C2=A0=C2=A0=C2=A0 * supported but the IDA allocation is inclus=
ive.
> >> +=C2=A0=C2=A0=C2=A0=C2=A0 */
> >> +=C2=A0=C2=A0=C2=A0 ret =3D ida_alloc_range(&iommu_global_pasid_ida,=20
> >> IOMMU_FIRST_GLOBAL_PASID, max - 1, GFP_KERNEL);
> >> +=C2=A0=C2=A0=C2=A0 if (ret < 0)
> >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return IOMMU_PASID_INVALID;
> >> +
> >> +=C2=A0=C2=A0=C2=A0 return ret;
> >> +}
> >> +EXPORT_SYMBOL_GPL(iommu_alloc_global_pasid_dev); =20
> >=20
> > "dev->iommu->max_pasids =3D=3D 0" indicates no pasid support on the dev=
ice.
> > The code should return IOMMU_PASID_INVALID explicitly. Perhaps we can
> > make this function like this:
> >=20
> > ioasid_t iommu_alloc_global_pasid_dev(struct device *dev)
> > {
> >  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int ret;
> >=20
> >  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (!dev->iommu->max_pasids)
> >  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0 return IOMMU_PASID_INVALID;
> >=20
> >  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 /*
> >  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * max_pasids is set u=
p by vendor driver based on number of=20
> > PASID bits
> >  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 * supported but the I=
DA allocation is inclusive.
> >  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */
> >  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ret =3D ida_alloc_range(&io=
mmu_global_pasid_ida,=20
> > IOMMU_FIRST_GLOBAL_PASID,
> >  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=
=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=
=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 dev->iommu->max_pasids - 1, GFP_KERNEL);
> >=20
> >  =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return ret < 0 ? IOMMU_PASI=
D_INVALID : ret;
> > }
> > EXPORT_SYMBOL_GPL(iommu_alloc_global_pasid_dev);
> >=20
> > Other change in this series looks good to me.
> >=20
> > I hope I can queue this series including above change as part of my VT-d
> > update for v6.5 to Joerg if no objection.
> >=20
> > Let's try to re-enable this key feature of Intel idxd driver in v6.5. =
=20
>=20
> This series didn't pass my test.
>=20
> The first time when I run "idxd_ktest.sh -c 1 -t 1 -i 100 -m shared", it
> passed. But when I run it again, the idxd hardware operation resulted in
> timed-out issues.
>=20
Hmm, not sure what happened. Are you using the out of tree idxd_ktest
kernel module or the dmaengine test sysfs?

e.g.
echo "Config params for DMA test"                  =20
echo $1 > /sys/module/dmatest/parameters/iterations=20
#echo 1 > /sys/module/dmatest/parameters/noverify  =20
echo "" > /sys/module/dmatest/parameters/channel   =20
echo 1 > /sys/module/dmatest/parameters/run        =20
sleep 2                                            =20
echo 0 > /sys/module/dmatest/parameters/run        =20
echo "Completed!"                                  =20

It passed my test many iterations for shared
in-kernel DSA test, will get your tree and test again.


Thanks,

Jacob