Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 25 Aug 2022 20:06:26 +0300
From:   Serge Semin <fancer.lancer@gmail.com>
To:     Bjorn Helgaas <helgaas@kernel.org>
Cc:     Serge Semin <Sergey.Semin@baikalelectronics.ru>,
        Gustavo Pimentel <gustavo.pimentel@synopsys.com>,
        Vinod Koul <vkoul@kernel.org>, Rob Herring <robh@kernel.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        Jingoo Han <jingoohan1@gmail.com>, Frank Li <Frank.Li@nxp.com>,
        Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>,
        Lorenzo Pieralisi <lpieralisi@kernel.org>,
        Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= <kw@linux.com>,
        Alexey Malahov <Alexey.Malahov@baikalelectronics.ru>,
        Pavel Parkhomenko <Pavel.Parkhomenko@baikalelectronics.ru>,
        linux-pci@vger.kernel.org, dmaengine@vger.kernel.org,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH RESEND v5 24/24] PCI: dwc: Add DW eDMA engine support
Message-ID: <20220825170626.fnqmw636tqgrqdqc@mobilestation>
References: <20220825051614.kfify5fbqlhurvdn@mobilestation>
 <20220825160443.GA2854084@bhelgaas>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220825160443.GA2854084@bhelgaas>
Precedence: bulk

On Thu, Aug 25, 2022 at 11:04:43AM -0500, Bjorn Helgaas wrote:
> On Thu, Aug 25, 2022 at 08:16:14AM +0300, Serge Semin wrote:
> > On Wed, Aug 24, 2022 at 01:17:54PM -0500, Bjorn Helgaas wrote:
> > > On Wed, Aug 24, 2022 at 09:13:19PM +0300, Serge Semin wrote:
> > > > On Wed, Aug 24, 2022 at 11:51:18AM -0500, Bjorn Helgaas wrote:
> > > > > On Mon, Aug 22, 2022 at 09:53:32PM +0300, Serge Semin wrote:
> > > 
> > > > > > +	val = dw_pcie_readl_dbi(pci, PCIE_DMA_VIEWPORT_BASE + PCIE_DMA_CTRL);
> > > > > > +	if (val == 0xFFFFFFFF && pci->edma.reg_base) {
> > > > > > +		pci->edma.mf = EDMA_MF_EDMA_UNROLL;
> > > > > > +
> > > > > > +		val = dw_pcie_readl_dma(pci, PCIE_DMA_CTRL);
> > > > > > +	} else if (val != 0xFFFFFFFF) {
> > > > > 
> > > > 
> > > > > Consider PCI_POSSIBLE_ERROR() as an annotation about the meaning of
> > > > > 0xFFFFFFFF and something to grep for.
> > > > 
> > > > In this case FFs don't mean an error but a special value, which
> > > > indicates that the eDMA is mapped via the unrolled CSRs space. The
> > > > similar approach has been implemented for the iATU legacy/unroll setup
> > > > auto-detection. So I don't see much reasons to have it grepped, so as
> > > > to have a macro-based parametrization since the special value will
> > > > unluckily change while having the explicit literal utilized gives a
> > > > better understanding of the way the algorithm works.
> > 
> > > If 0xFFFFFFFF is the result of a successful PCIe Memory Read,
> > 
> > Right. It is.
> > 
> > > and not
> > > something synthesized by the host bridge when it handles an
> > > Unsupported Request completion,
> > 
> > No it isn't. To be clear 0xFFs don't indicate some PCIe bus/controller
> > malfunction, but they are a result of reading the
> > DMA_CTRL_VIEWPORT_OFF register which doesn't exist. The manual
> > explicitly says: "Note - When register does not exist, value is fixed
> > to 32'hFFFF_FFFF". The register doesn't exist if either eDMA is
> > unavailable or the eDMA CSRs are mapped via the unrolled state.
> 

> OK.  I don't think that's worded very well in the manual.  A register
> that does not exist does not have a value, and attempts to read it
> should fail.

No. The manual explicitly says that this particular CSR
(DMA_CTRL_VIEWPORT_OFF) value is tied to 32'hFFFF_FFFF if the register
doesn't exist. There is no such text mentioned for any other
non-existing CSR.

> If they want to say the register always exists and
> contains 0xFFFFFFFF for versions earlier than X, that would make
> sense.  Wouldn't be the first time a manual is ambiguous ;)

They say, that the register doesn't exist if either eDMA isn't
available or it's mapped via the unrolled CSR space. There is no
reference to the IP-core version.

Anyway basically you are right they indeed imply that the register
always exists.

> 
> If the device itself, i.e., not the Root Complex, is fabricating this
> 0xFFFFFFFF value, reading it should not cause any AER or other error
> status bits to be set.
> 
> If the Root Complex fabricates 0xFFFFFFFF upon receipt of a Completion
> with Unsupported Request status, I would expect bits like Received
> Master Abort to be set in the Root Port's Secondary Status register.

The device CSRs are accessed by means of the controller DBI-interface.
Even though the whole CSRs space do look as the extended PCIe config
space, the DBI-based access isn't tracked by the standard PCIe
capabilities like AER. So in case of the eDMA auto-detection procedure
introduced in this patch (and iATU auto-detection, which is already
available in the DW PCIe driver ) we won't have any bus error status
raised.

-Sergey