Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:8 as permitted sender) client-ip=2620:137:e000::3:8;
Date:   Wed, 11 Oct 2023 13:50:13 +0300
From:   Serge Semin <fancer.lancer@gmail.com>
To:     Kory Maincent <kory.maincent@bootlin.com>
Cc:     Manivannan Sadhasivam <mani@kernel.org>,
        Gustavo Pimentel <gustavo.pimentel@synopsys.com>,
        Vinod Koul <vkoul@kernel.org>,
        Cai Huoqing <cai.huoqing@linux.dev>,
        Thomas Petazzoni <thomas.petazzoni@bootlin.com>,
        dmaengine@vger.kernel.org, linux-kernel@vger.kernel.org,
        Herve Codina <herve.codina@bootlin.com>
Subject: Re: [PATCH v3 5/6] dmaengine: dw-edma: HDMA: Add sync read before
 starting the DMA transfer in remote setup
Message-ID: <o4h5rrfsfu5sipovexqrjl3jatfkh6ht3fycwv5vzh6bhxdygj@3jy3ns7ciglj>
References: <20231011-b4-feature_hdma_mainline-v3-0-24ee0c979c6f@bootlin.com>
 <20231011-b4-feature_hdma_mainline-v3-5-24ee0c979c6f@bootlin.com>
 <6adlujxc4cnrxbl5fbqpg5fishq7jvk6w6chgjyktbwcxd2dvi@w4ayo5ooh7fq>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <6adlujxc4cnrxbl5fbqpg5fishq7jvk6w6chgjyktbwcxd2dvi@w4ayo5ooh7fq>
Precedence: bulk

On Wed, Oct 11, 2023 at 01:47:49PM +0300, Serge Semin wrote:
> On Wed, Oct 11, 2023 at 10:11:44AM +0200, Kory Maincent wrote:
> > The Linked list element and pointer are not stored in the same memory as
> > the HDMA controller register. If the doorbell register is toggled before
> > the full write of the linked list a race condition error can appears.
> 
> s/can appears/may occur

It should have been: s/can appears/will occur

-Serge(y)

> 
> > In remote setup we can only use a readl to the memory to assured the full
> > write has occurred.
> 
> s/assured/assure
> 
> > 
> > Fixes: e74c39573d35 ("dmaengine: dw-edma: Add support for native HDMA")
> > Signed-off-by: Kory Maincent <kory.maincent@bootlin.com>
> 
> Reviewed-by: Serge Semin <fancer.lancer@gmail.com>
> 
> -Serge(y)
> 
> > ---
> > 
> > Changes in v2:
> > - Move the sync read in a function.
> > - Add commments
> > ---
> >  drivers/dma/dw-edma/dw-hdma-v0-core.c | 17 +++++++++++++++++
> >  1 file changed, 17 insertions(+)
> > 
> > diff --git a/drivers/dma/dw-edma/dw-hdma-v0-core.c b/drivers/dma/dw-edma/dw-hdma-v0-core.c
> > index 04b0bcb6ded9..13b6aec6a6de 100644
> > --- a/drivers/dma/dw-edma/dw-hdma-v0-core.c
> > +++ b/drivers/dma/dw-edma/dw-hdma-v0-core.c
> > @@ -222,6 +222,20 @@ static void dw_hdma_v0_core_write_chunk(struct dw_edma_chunk *chunk)
> >  	dw_hdma_v0_write_ll_link(chunk, i, control, chunk->ll_region.paddr);
> >  }
> >  
> > +static void dw_hdma_v0_sync_ll_data(struct dw_edma_chunk *chunk)
> > +{
> > +	/*
> > +	 * In case of remote HDMA engine setup, the DW PCIe RP/EP internals
> > +	 * configuration registers and Application memory are normally accessed
> > +	 * over different buses. Ensure LL-data reaches the memory before the
> > +	 * doorbell register is toggled by issuing the dummy-read from the remote
> > +	 * LL memory in a hope that the posted MRd TLP will return only after the
> > +	 * last MWr TLP is completed
> > +	 */
> > +	if (!(chunk->chan->dw->chip->flags & DW_EDMA_CHIP_LOCAL))
> > +		readl(chunk->ll_region.vaddr.io);
> > +}
> > +
> >  static void dw_hdma_v0_core_start(struct dw_edma_chunk *chunk, bool first)
> >  {
> >  	struct dw_edma_chan *chan = chunk->chan;
> > @@ -252,6 +266,9 @@ static void dw_hdma_v0_core_start(struct dw_edma_chunk *chunk, bool first)
> >  	/* Set consumer cycle */
> >  	SET_CH_32(dw, chan->dir, chan->id, cycle_sync,
> >  		  HDMA_V0_CONSUMER_CYCLE_STAT | HDMA_V0_CONSUMER_CYCLE_BIT);
> > +
> > +	dw_hdma_v0_sync_ll_data(chunk);
> > +
> >  	/* Doorbell */
> >  	SET_CH_32(dw, chan->dir, chan->id, doorbell, HDMA_V0_DOORBELL_START);
> >  }
> > 
> > -- 
> > 2.25.1
> >