Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 4 May 2022 13:47:20 +0530
From:   Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
To:     Loic Poulain <loic.poulain@linaro.org>
Cc:     mhi@lists.linux.dev, linux-arm-msm@vger.kernel.org,
        linux-kernel@vger.kernel.org, quic_hemantk@quicinc.com,
        quic_bbhatt@quicinc.com
Subject: Re: [PATCH 5/5] bus: mhi: host: Remove redundant dma_wmb() before
 ctx wp update
Message-ID: <20220504081720.GB5446@thinkpad>
References: <20220502104144.91806-1-manivannan.sadhasivam@linaro.org>
 <20220502104144.91806-6-manivannan.sadhasivam@linaro.org>
 <CAMZdPi_i60TqszUL+=ocMn-4veyoGRQoOGD_B4YiEpz_uWE+ZQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAMZdPi_i60TqszUL+=ocMn-4veyoGRQoOGD_B4YiEpz_uWE+ZQ@mail.gmail.com>
Precedence: bulk

Hi Loic,

On Wed, May 04, 2022 at 09:21:20AM +0200, Loic Poulain wrote:
> Hi Mani,
> 
> On Mon, 2 May 2022 at 12:42, Manivannan Sadhasivam
> <manivannan.sadhasivam@linaro.org> wrote:
> >
> > The endpoint device will only read the context wp when the host rings
> > the doorbell.
> 
> Are we sure about this statement? what if we update ctxt_wp while the
> device is still processing the previous ring? is it going to continue
> processing the new ctxt_wp or wait for a new doorbell interrupt? what
> about burst mode in which we don't ring at all (ring_db is no-op)?
> 

Good point. I think my statement was misleading. But still this scenario won't
happen as per my undestanding. Please see below.

> > And moreover the doorbell write is using writel(). This
> > guarantess that the prior writes will be completed before ringing
> > doorbell.
> 
> Yes but the barrier is to ensure that descriptor/ring content is
> updated before we actually pass it to device ownership, it's not about
> ordering with the doorbell write, but the memory coherent ones.
> 

I see a clear data dependency between writing the ring element and updating the
context pointer. For instance,

```
struct mhi_ring_element *mhi_tre;

mhi_tre = ring->wp;
/* Populate mhi_tre */
...

/* Increment wp */
ring->wp += el_size;

/* Update ctx wp */
ring->ctx_wp = ring->iommu_base + (ring->wp - ring->base);
```

This is analogous to:

```
Read PTR A;
Update PTR A;
Increment PTR A;
Write PTR A to PTR B;
```

Here, because of the data dependency due to "ring->wp", the CPU or compiler
won't be ordering the instructions. I think that's one of the reason we never
hit any issue due to this.

Thanks,
Mani

> >
> > So there is no need of an additional dma_wmb() to order the coherent
> > memory writes w.r.t each other. Even if the writes gets reordered, it
> > won't affect the endpoint device.
> >
> > Cc: Loic Poulain <loic.poulain@linaro.org>
> > Signed-off-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> > ---
> >  drivers/bus/mhi/host/main.c | 5 -----
> >  1 file changed, 5 deletions(-)
> >
> > diff --git a/drivers/bus/mhi/host/main.c b/drivers/bus/mhi/host/main.c
> > index 966ffc2458b9..6706a82d3aa8 100644
> > --- a/drivers/bus/mhi/host/main.c
> > +++ b/drivers/bus/mhi/host/main.c
> > @@ -138,11 +138,6 @@ void mhi_ring_chan_db(struct mhi_controller *mhi_cntrl,
> >
> >         db = ring->iommu_base + (ring->wp - ring->base);
> >
> > -       /*
> > -        * Writes to the new ring element must be visible to the hardware
> > -        * before letting h/w know there is new element to fetch.
> > -        */
> > -       dma_wmb();
> >         *ring->ctxt_wp = cpu_to_le64(db);
> >
> >         mhi_chan->db_cfg.ring_db(mhi_cntrl, &mhi_chan->db_cfg,
> > --
> > 2.25.1
> >
> 
> Regards,
> Loic