Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 23 Jun 2023 08:06:28 +0200
From:   Lukas Wunner <lukas@wunner.de>
To:     Felix Kuehling <felix.kuehling@amd.com>
Cc:     Smita Koralahalli <Smita.KoralahalliChannabasappa@amd.com>,
        linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
        Bjorn Helgaas <bhelgaas@google.com>, oohall@gmail.com,
        Mahesh J Salgaonkar <mahesh@linux.ibm.com>,
        Kuppuswamy Sathyanarayanan 
        <sathyanarayanan.kuppuswamy@linux.intel.com>,
        Yazen Ghannam <yazen.ghannam@amd.com>,
        Fontenot Nathan <Nathan.Fontenot@amd.com>,
        Jay Cornwall <Jay.Cornwall@amd.com>
Subject: Re: [PATCH v3 2/2] PCI: pciehp: Clear the optional capabilities in
 DEVCTL2 on a hot-plug
Message-ID: <20230623060628.GA28334@wunner.de>
References: <20230621185152.105320-1-Smita.KoralahalliChannabasappa@amd.com>
 <20230621185152.105320-3-Smita.KoralahalliChannabasappa@amd.com>
 <20230622063105.GA624@wunner.de>
 <d64f8de4-cac1-ccca-33fe-1fda418837e2@amd.com>
 <20230622214247.GB11993@wunner.de>
 <8bb20976-97ec-0c5c-adc8-183896de6768@amd.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <8bb20976-97ec-0c5c-adc8-183896de6768@amd.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Precedence: bulk

On Fri, Jun 23, 2023 at 05:59:55AM +0200, Felix Kuehling wrote:
> On 2023-06-22 23:42, Lukas Wunner wrote:
> > On Thu, Jun 22, 2023 at 02:02:12PM -0700, Smita Koralahalli wrote:
> > > Would it be fair to just reuse pci_enable_atomic_ops_to_root() for
> > > Atomic_Ops configuration?
> > 
> > Hm, that's a good question.  I'm not an expert on that corner of
> > the PCI core.
> > 
> > But indeed what you could try is amend that function to not only
> > *set* PCI_EXP_DEVCTL2_ATOMIC_REQ if it's supported, but to also
> > *clear* it if it's not supported.
> > 
> > And you'd have to call pci_enable_atomic_ops_to_root() on enumeration,
> > e.g. from pci_init_capabilities().
> > 
> > That should obviate the need to call pci_enable_atomic_ops_to_root()
> > from drivers, so you could probably remove the call from all the
> > drivers which currently call it (amdgpu, infiniband, mellanox),
> > in one separate patch per driver.
> > 
> > An then you could drop the EXPORT clause for pci_enable_atomic_ops_to_root()
> > and make it private to the PCI core.
> 
> Then our driver would need an alternative way to determine whether atomic
> capabilities are enabled for a device. We currently use the return value
> from pci_enable_atomic_ops_to_root to determine this.

Just read PCI_EXP_DEVCTL2 and check whether PCI_EXP_DEVCTL2_ATOMIC_REQ
is set.  (I.e. has been set by the PCI core on device enumeration.)

Problem solved, I guess?

Thanks,

Lukas