Date:   Thu, 16 Feb 2023 10:47:29 +0000
From:   Jean-Philippe Brucker <jean-philippe@linaro.org>
To:     Leon Romanovsky <leon@kernel.org>
Cc:     Bjorn Helgaas <helgaas@kernel.org>,
        Ganapatrao Kulkarni <gankulkarni@os.amperecomputing.com>,
        linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,
        bhelgaas@google.com, darren@os.amperecomputing.com,
        scott@os.amperecomputing.com, Will Deacon <will@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>,
        Joerg Roedel <joro@8bytes.org>,
        linux-arm-kernel@lists.infradead.org, iommu@lists.linux.dev
Subject: Re: [PATCH] PCI/ATS:  Allow to enable ATS on VFs even if it is not
 enabled on PF
Message-ID: <Y+4JwV843tZWGxih@myrica>
References: <Y+ksmNWJdWNkGAU9@unreal>
 <20230215205726.GA3213227@bhelgaas>
 <Y+3al/a3HPrvfNgh@unreal>
 <Y+3fa/3HC1vsLRXa@unreal>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y+3fa/3HC1vsLRXa@unreal>
Precedence: bulk

On Thu, Feb 16, 2023 at 09:46:51AM +0200, Leon Romanovsky wrote:
> On Thu, Feb 16, 2023 at 09:26:15AM +0200, Leon Romanovsky wrote:
> > On Wed, Feb 15, 2023 at 02:57:26PM -0600, Bjorn Helgaas wrote:
> > > [+cc Will, Robin, Joerg for arm-smmu-v3 page size question]
> > > 
> > > On Sun, Feb 12, 2023 at 08:14:48PM +0200, Leon Romanovsky wrote:
> > > > On Wed, Feb 08, 2023 at 10:43:21AM -0800, Ganapatrao Kulkarni wrote:
> > > > > As per PCIe specification(section 10.5), If a VF implements an
> > > > > ATS capability, its associated PF must implement an ATS capability.
> > > > > The ATS Capabilities in VFs and their associated PFs are permitted to
> > > > > be enabled independently.
> > > > > Also, it states that the Smallest Translation Unit (STU) for VFs must be
> > > > > hardwired to Zero and the associated PF's value applies to VFs STU.
> > > > > 
> > > > > The current code allows to enable ATS on VFs only if it is already
> > > > > enabled on associated PF, which is not necessary as per the specification.
> > > > > 
> > > > > It is only required to have valid STU programmed on PF to enable
> > > > > ATS on VFs. Adding code to write the first VFs STU to a PF's STU
> > > > > when PFs ATS is not enabled.
> > > >
> > > > Can you please add here quotes from the spec and its version? I don't see
> > > > anything like this in my version of PCIe specification.

In PCIe r6.0, 10.5.1 ATS Extended Capability:

"The ATS Capabilities in VFs and their associated PFs are permitted to be
enabled independently."

> For VFs, this field must be hardwired to Zero. The associated PF's value applies.
> Default value is 0 0000b"

And this sentence indicates that the PF's STU should be configured
appropriately in order to use ATS in the VF.

So a driver is permitted to enable the VF ATS capability without enabling
the PF ATS cap, though the STU value of the PF cap still applies. But the
first sentence is weak ("permitted" instead of "required"), so as Joerg
said, some device implementations may still require to enable the PF cap
in order to enable the VF cap.

Maybe we could have a list of vendor:device IDs which allow enabling the
VF cap independently?

Thanks,
Jean