Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Thu, 8 Jul 2021 13:23:53 +0530
From:   Viresh Kumar <viresh.kumar@linaro.org>
To:     Ionela Voinescu <ionela.voinescu@arm.com>,
        Kevin Wangtao <kevin.wangtao@hisilicon.com>,
        Leo Yan <leo.yan@linaro.org>,
        Jassi Brar <jaswinder.singh@linaro.org>
Cc:     zhongkaihua@huawei.com, Dmitry Osipenko <digetx@gmail.com>,
        Viresh Kumar <vireshk@kernel.org>, Nishanth Menon <nm@ti.com>,
        Stephen Boyd <sboyd@kernel.org>, linux-pm@vger.kernel.org,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Rafael Wysocki <rjw@rjwysocki.net>,
        Sibi Sankar <sibis@codeaurora.org>,
        linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH 03/13] opp: Keep track of currently programmed OPP
Message-ID: <20210708075353.ivsuc4y47i6bbhgz@vireshk-i7>
References: <cover.1611227342.git.viresh.kumar@linaro.org>
 <96b57316a2a307a5cc5ff7302b3cd0084123a2ed.1611227342.git.viresh.kumar@linaro.org>
 <20210707102410.GA4357@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210707102410.GA4357@arm.com>
User-Agent: NeoMutt/20180716-391-311a52
Precedence: bulk

On 07-07-21, 11:24, Ionela Voinescu wrote:
> Now comes the interesting part: what seems to fix it is a call to
> clk_get_rate(opp_table->clk) in _set_opp(), which is what basically
> happened before this patch, as _find_current_opp() was always called.
> I do not need to do anything with the returned frequency.

Wow, thanks for narrowing it down this far :)

I had a quick look and this is what I think is the problem here.

This platform uses mailbox API to send its frequency change requests to another
processor.  And the way it is written currently, I don't see any guarantee
whatsoever which say

  "once clk_set_rate() returns, the frequency would have already changed".

And this may exactly be the thing you are able to hit, luckily because of this
patchset :)

As a quick way of checking if that is right or not, this may make it work:

diff --git a/drivers/mailbox/hi3660-mailbox.c b/drivers/mailbox/hi3660-mailbox.c
index 395ddc250828..9856c1c84dcf 100644
--- a/drivers/mailbox/hi3660-mailbox.c
+++ b/drivers/mailbox/hi3660-mailbox.c
@@ -201,6 +201,9 @@ static int hi3660_mbox_send_data(struct mbox_chan *chan, void *msg)

        /* Trigger data transferring */
        writel(BIT(mchan->ack_irq), base + MBOX_SEND_REG);
+
+       hi3660_mbox_check_state(chan);
+
        return 0;
 }

-------------------------8<-------------------------

As a proper fix, something like this (not even compile tested) is required I
believe as I don't see the clients would know if the transfer is over. Cc'ing
mailbox guys to see what can be done.

diff --git a/drivers/clk/hisilicon/clk-hi3660-stub.c b/drivers/clk/hisilicon/clk-hi3660-stub.c
index 3a653d54bee0..c1e62ea4cf01 100644
--- a/drivers/clk/hisilicon/clk-hi3660-stub.c
+++ b/drivers/clk/hisilicon/clk-hi3660-stub.c
@@ -89,7 +89,6 @@ static int hi3660_stub_clk_set_rate(struct clk_hw *hw, unsigned long rate,
                stub_clk->msg[0], stub_clk->msg[1]);
 
        mbox_send_message(stub_clk_chan.mbox, stub_clk->msg);
-       mbox_client_txdone(stub_clk_chan.mbox, 0);
 
        stub_clk->rate = rate;
        return 0;
@@ -131,7 +130,7 @@ static int hi3660_stub_clk_probe(struct platform_device *pdev)
        /* Use mailbox client without blocking */
        stub_clk_chan.cl.dev = dev;
        stub_clk_chan.cl.tx_done = NULL;
-       stub_clk_chan.cl.tx_block = false;
+       stub_clk_chan.cl.tx_block = true;
        stub_clk_chan.cl.knows_txdone = false;
 
        /* Allocate mailbox channel */
diff --git a/drivers/mailbox/hi3660-mailbox.c b/drivers/mailbox/hi3660-mailbox.c
index 395ddc250828..8f6b787c0aba 100644
--- a/drivers/mailbox/hi3660-mailbox.c
+++ b/drivers/mailbox/hi3660-mailbox.c
@@ -1,5 +1,5 @@
 // SPDX-License-Identifier: GPL-2.0
-// Copyright (c) 2017-2018 HiSilicon Limited.
+// Copyright (c) 2017-2018 Hisilicon Limited.
 // Copyright (c) 2017-2018 Linaro Limited.
 
 #include <linux/bitops.h>
@@ -83,7 +83,7 @@ static struct hi3660_mbox *to_hi3660_mbox(struct mbox_controller *mbox)
        return container_of(mbox, struct hi3660_mbox, controller);
 }
 
-static int hi3660_mbox_check_state(struct mbox_chan *chan)
+static bool hi3660_mbox_last_tx_done(struct mbox_chan *chan)
 {
        unsigned long ch = (unsigned long)chan->con_priv;
        struct hi3660_mbox *mbox = to_hi3660_mbox(chan->mbox);
@@ -94,20 +94,20 @@ static int hi3660_mbox_check_state(struct mbox_chan *chan)
 
        /* Mailbox is ready to use */
        if (readl(base + MBOX_MODE_REG) & MBOX_STATE_READY)
-               return 0;
+               return true;
 
        /* Wait for acknowledge from remote */
        ret = readx_poll_timeout_atomic(readl, base + MBOX_MODE_REG,
                        val, (val & MBOX_STATE_ACK), 1000, 300000);
        if (ret) {
                dev_err(mbox->dev, "%s: timeout for receiving ack\n", __func__);
-               return ret;
+               return false;
        }
 
        /* clear ack state, mailbox will get back to ready state */
        writel(BIT(mchan->ack_irq), base + MBOX_ICLR_REG);
 
-       return 0;
+       return true;
 }
 
 static int hi3660_mbox_unlock(struct mbox_chan *chan)
@@ -182,10 +182,6 @@ static int hi3660_mbox_send_data(struct mbox_chan *chan, void *msg)
        unsigned int i;
        int ret;
 
-       ret = hi3660_mbox_check_state(chan);
-       if (ret)
-               return ret;
-
        /* Clear mask for destination interrupt */
        writel_relaxed(~BIT(mchan->dst_irq), base + MBOX_IMASK_REG);
 
@@ -207,6 +203,7 @@ static int hi3660_mbox_send_data(struct mbox_chan *chan, void *msg)
 static const struct mbox_chan_ops hi3660_mbox_ops = {
        .startup        = hi3660_mbox_startup,
        .send_data      = hi3660_mbox_send_data,
+       .last_tx_done   = hi3660_mbox_last_tx_done,
 };
 
 static struct mbox_chan *hi3660_mbox_xlate(struct mbox_controller *controller,
@@ -259,6 +256,7 @@ static int hi3660_mbox_probe(struct platform_device *pdev)
        mbox->controller.num_chans = MBOX_CHAN_MAX;
        mbox->controller.ops = &hi3660_mbox_ops;
        mbox->controller.of_xlate = hi3660_mbox_xlate;
+       mbox->controller.txdone_poll = true;
 
        /* Initialize mailbox channel data */
        chan = mbox->chan;

-- 
viresh