Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Wed, 1 Jun 2022 13:17:38 -0700
From:   Ira Weiny <ira.weiny@intel.com>
To:     Lukas Wunner <lukas@wunner.de>
Cc:     Jonathan Cameron <Jonathan.Cameron@huawei.com>,
        Dan Williams <dan.j.williams@intel.com>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Christoph Hellwig <hch@infradead.org>,
        Alison Schofield <alison.schofield@intel.com>,
        Vishal Verma <vishal.l.verma@intel.com>,
        Ben Widawsky <ben.widawsky@intel.com>,
        linux-kernel@vger.kernel.org, linux-cxl@vger.kernel.org,
        linux-pci@vger.kernel.org
Subject: Re: [PATCH V8 03/10] PCI: Create PCI library functions in support of
 DOE mailboxes.
Message-ID: <YpfJYujerG2hM4vU@iweiny-desk3>
References: <20220414203237.2198665-1-ira.weiny@intel.com>
 <20220414203237.2198665-4-ira.weiny@intel.com>
 <20220530190657.GA14765@wunner.de>
 <20220531113350.0000421e@Huawei.com>
 <YpbWCYujYDEkMm1B@iweiny-desk3>
 <20220601071808.GA19924@wunner.de>
 <Ypee328j+l6ZdbUT@iweiny-desk3>
 <20220601175647.GA21509@wunner.de>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220601175647.GA21509@wunner.de>
Precedence: bulk

On Wed, Jun 01, 2022 at 07:56:47PM +0200, Lukas Wunner wrote:
> On Wed, Jun 01, 2022 at 10:16:15AM -0700, Ira Weiny wrote:
> > On Wed, Jun 01, 2022 at 09:18:08AM +0200, Lukas Wunner wrote:
> > > You only need to re-check the Data Object Ready bit on the last-but-one
> > > dword in case the function was reset concurrently.  Per sec. 6.30.2,
> > > "An FLR to a Function must result in the aborting of any DOE transfer
> > > in progress."
> > 
> > I think I disagree.  Even if we do that and an FLR comes before the last read
> > the last read could be 0's.
> 
> PCIe r6.0, Table 7-316 says:
> 
>   "If there is no additional data object ready for transfer, the
>    DOE instance must clear this bit after the entire data object has been
>    transferred, as indicated by software writing to the DOE Read Data
>    Mailbox Register after reading the final DW of the data object."
> 
> Remember that you *read* a dword from the mailbox and then acknowledge
> reception to the mailbox by *writing* a dword to the mailbox.
> 
> So you check that the Data Object Ready bit is set before acknowledging
> the final dword with a register write.  That's race-free.

Ok.

> 
> (I realize me talking about the "last-but-one dword" above was quite
> unclear, sorry about that.)
> 

Ah yes.  Ok, I'll put in a check before the final write.

Ira