Date:   Mon, 13 Mar 2023 17:42:25 -0500
From:   Bjorn Helgaas <helgaas@kernel.org>
To:     Sathyanarayanan Kuppuswamy 
        <sathyanarayanan.kuppuswamy@linux.intel.com>
Cc:     Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com>,
        linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev,
        joro@8bytes.org, bhelgaas@google.com, robin.murphy@arm.com,
        will@kernel.org, jean-philippe@linaro.org,
        darren@os.amperecomputing.com, scott@os.amperecomputing.com
Subject: Re: [PATCH v2 1/2] PCI/ATS: Add a helper function to configure ATS
 STU of a PF
Message-ID: <20230313224225.GA1551878@bhelgaas>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <6339c50a-8dfe-f3a2-63d7-504abd4e62f0@linux.intel.com>
Precedence: bulk

On Mon, Mar 13, 2023 at 03:30:30PM -0700, Sathyanarayanan Kuppuswamy wrote:
> On 3/13/23 2:12 PM, Bjorn Helgaas wrote:
> > On Mon, Feb 27, 2023 at 08:21:36PM -0800, Ganapatrao Kulkarni wrote:
> >> As per PCI specification (PCI Express Base Specification Revision
> >> 6.0, Section 10.5) both PF and VFs of a PCI EP are permitted to be enabled
> >> independently for ATS capability, however the STU(Smallest Translation
> >> Unit) is shared between PF and VFs. For VFs, it is hardwired to Zero and
> >> the associated PF's value applies to VFs.
> >>
> >> In the current code, the STU is being configured while enabling the PF ATS.
> >> Hence, it is not able to enable ATS for VFs, if it is not enabled on the
> >> associated PF already.
> >>
> >> Adding a function pci_ats_stu_configure(), which can be called to
> >> configure the STU during PF enumeration.
> >> Latter enumerations of VFs can successfully enable ATS independently.

> >> + * pci_ats_stu_configure - Configure STU of a PF.
> >> + * @dev: the PCI device
> >> + * @ps: the IOMMU page shift
> >> + *
> >> + * Returns 0 on success, or negative on failure.
> >> + */
> >> +int pci_ats_stu_configure(struct pci_dev *dev, int ps)
> >> +{
> >> +	u16 ctrl;
> >> +
> >> +	if (dev->ats_enabled || dev->is_virtfn)
> >> +		return 0;
> 
> We don't have any checks for the PF case here. That means you can
> re-configure the STU as many times as you want until ATS is enabled
> in PF. So, if there are active VFs which uses this STU, can PF
> re-configure the STU at will?

Really good question!  I withdraw my ack until this is resolved.

I don't think we want PFs changing STU behind the back of VFs.

Bjorn