Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <CABb+yY0cW1GZHVmwEr19JRdJTmsAxw9uq83QV_aq-tdPJO5_Fg@mail.gmail.com>
 <20200604092052.GD8814@bogus> <CABb+yY27Ngb0C-onkU2qyt=uKgG4iVrcv8hGkC+anypQbTRA1w@mail.gmail.com>
 <20200605045645.GD12397@bogus> <CABb+yY2YZ99NjHYNi0=KLGFDsVUeJmqiJD3E25Chwk-THJV4iw@mail.gmail.com>
 <20200605085830.GA32372@bogus> <CABb+yY2TR7tuMx6u8yah6mO2GwZ5SWYOO80EQRL-i=ybgn=Wog@mail.gmail.com>
 <20200610093334.yznxl2esv5ht27ns@vireshk-i7> <20200611100027.GB18781@bogus>
 <CABb+yY1OwVB+uk+0w+D-4Xy46iYn8tPZYBZ0qjrzQFLA6SaTvw@mail.gmail.com>
 <20200612052853.nds4iycie6ldjnnr@vireshk-i7> <CAK8P3a0nVOR7YYSZaKmzm3WsUZLgOqL7yZq+f0Dfnn2=16AkLA@mail.gmail.com>
In-Reply-To: <CAK8P3a0nVOR7YYSZaKmzm3WsUZLgOqL7yZq+f0Dfnn2=16AkLA@mail.gmail.com>
From:   Jassi Brar <jassisinghbrar@gmail.com>
Date:   Tue, 8 Sep 2020 22:23:33 -0500
Message-ID: <CABb+yY3HDs219C8Wcc+CJRLukV4OvCKhqevU-9jN4bCU6SC3rQ@mail.gmail.com>
Subject: Re: [RFC] dt-bindings: mailbox: add doorbell support to ARM MHU
To:     Arnd Bergmann <arnd@arndb.de>
Cc:     Viresh Kumar <viresh.kumar@linaro.org>,
        Sudeep Holla <sudeep.holla@arm.com>,
        Rob Herring <robh@kernel.org>,
        Frank Rowand <frowand.list@gmail.com>,
        Bjorn Andersson <bjorn.andersson@linaro.org>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
        Devicetree List <devicetree@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, Sep 8, 2020 at 4:15 AM Arnd Bergmann <arnd@arndb.de> wrote:
>
> Picking up the old thread again after and getting pinged by multiple
> colleagues about it (thanks!) reading through the history.
>
> On Fri, Jun 12, 2020 at 7:29 AM Viresh Kumar <viresh.kumar@linaro.org> wrote:
> >
> > On 11-06-20, 19:34, Jassi Brar wrote:
> > > In the first post in this thread, Viresh lamented that mailbox
> > > introduces "a few ms" delay in the scheduler path.
> > > Your own tests show that is certainly not the case -- average is the
> > > same as proposed virtual channels 50-100us, the best case is 3us vs
> > > 53us for virtual channels.
> >
> > Hmmm, I am not sure where is the confusion here Jassi. There are two
> > things which are very very different from each other.
> >
> > - Time taken by the mailbox framework (and remote for acknowledging
> >   it) for completion of a single request, this can be 3us to 100s of
> >   us. This is clear for everyone. THIS IS NOT THE PROBLEM.
> >
> > - Delay introduced by few of such requests on the last one, i.e. 5
> >   normal requests followed by an important one (like DVFS), the last
> >   one needs to wait for the first 5 to finish first. THIS IS THE
> >   PROBLEM.
>
> Earlier, Jassi also commented "Linux does not provide real-time
> guarantees", which to me is what actually causes the issue here:
>
> Linux having timeouts when communicating to the firmware means
> that it relies on the hardware and firmware having real-time behavior
> even when not providing real-time guarantees to its processes.
>
The timeout used in SCMI is simply based on how long the Juno (?)
platform takes to reply in most cases.
Talking proper code-design, the timeout (if at all) shouldn't even be
a hardcoded value, but instead taken from the platform.

> When comparing the two usage models, it's clear that the minimum
> latency for a message delivery is always at least the time time
> to process an interrupt, plus at least one expensive MMIO read
> and one less expensive posted MMIO write for an Ack. If we
> have a doorbell plus out-of-band message, we need an extra
> DMA barrier and a read from coherent memory, both of which can
> be noticeable. As soon as messages are queued in the current
> model, the maximum latency increases by a potentially unbounded
> number of round-trips, while in the doorbell model that problem
> does not exist, so I agree that we need to handle both modes
> in the kernel deal with all existing hardware as well as firmware
> that requires low-latency communication.
>
From the test case Sudeep last shared, the scmi usage on mhu doesn't
not even hit any bottleneck ... the test "failed" because of the too
small hardcoded timeout value. Otherwise the current code actually
shows better numbers.
We need some synthetic tests to bring the limitation to the surface. I
agree that there may be such a test case, however fictitious. For that
reason, I am ok with the doorbell mode.

> The only questions that I see in need of being answered are:
>
> 1. Should the binding use just different "#mbox-cells" values or
>    also different "compatible" strings to tell that difference?
> 2. Should one driver try to handle both modes or should there
>    be two drivers?
>
> It sounds like Jassi strongly prefers separate drivers, which
> would make separate compatible strings the more practical
> approach. While the argument can be made that a single
> piece of hardware should only have one DT description,
> the counter-argument would be that the behavior described
> by the DT here is made up by both the hardware and the
> firmware behind it, and they are in fact different.
>
I totally agree with one compat-string for one hardware. However, as
you said, unlike other device classes, the mailbox driver runs the
sumtotal of hardware and the remote firmware behaviour. Also the
implementations wouldn't share much, so I think a separate file+dt
will be better.  But I wanna get rid of this toothache that flares up
every season, so whatever.

Cheers!