Received-SPF: pass (google.com: domain of linux-kernel+bounces-189042-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Fri, 24 May 2024 12:23:01 -0700
From: Bjorn Andersson <quic_bjorande@quicinc.com>
To: Krzysztof Kozlowski <krzk@kernel.org>
CC: Chris Lew <quic_clew@quicinc.com>, Bjorn Andersson <andersson@kernel.org>,
        Baolin Wang <baolin.wang@linux.alibaba.com>,
        Peter Zijlstra
	<peterz@infradead.org>, Ingo Molnar <mingo@redhat.com>,
        Will Deacon
	<will@kernel.org>, Waiman Long <longman@redhat.com>,
        Boqun Feng
	<boqun.feng@gmail.com>, Jonathan Corbet <corbet@lwn.net>,
        Mathieu Poirier
	<mathieu.poirier@linaro.org>,
        Rob Herring <robh@kernel.org>,
        "Krzysztof
 Kozlowski" <krzk+dt@kernel.org>,
        Conor Dooley <conor+dt@kernel.org>,
        Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>,
        Konrad Dybcio
	<konrad.dybcio@linaro.org>,
        <linux-remoteproc@vger.kernel.org>, <linux-kernel@vger.kernel.org>,
        <linux-doc@vger.kernel.org>, <linux-arm-msm@vger.kernel.org>,
        <devicetree@vger.kernel.org>
Subject: Re: [PATCH 5/7] dt-bindings: remoteproc: qcom,pas: Add hwlocks
Message-ID: <ZlDpFV8bL8/lwGOP@hu-bjorande-lv.qualcomm.com>
References: <20240516-hwspinlock-bust-v1-0-47a90a859238@quicinc.com>
 <20240516-hwspinlock-bust-v1-5-47a90a859238@quicinc.com>
 <3521519f-34b8-472d-be37-f0e64bba24fc@kernel.org>
 <a944418a-1699-44fa-bdfc-2e57129adea1@quicinc.com>
 <c9882ba0-bbbf-44ec-9606-ebe68bcb8866@kernel.org>
 <ZkzzY311XiRigJPt@hu-bjorande-lv.qualcomm.com>
 <92dcd555-69b1-4111-92dd-debe5107d526@kernel.org>
 <Zk4wab/NZOOZ3hA6@hu-bjorande-lv.qualcomm.com>
 <aed37430-7e87-4516-86da-3997c01a8aa8@kernel.org>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <aed37430-7e87-4516-86da-3997c01a8aa8@kernel.org>

On Thu, May 23, 2024 at 08:15:54AM +0200, Krzysztof Kozlowski wrote:
> On 22/05/2024 19:50, Bjorn Andersson wrote:
> >>>>>
> >>>>> We did consider tying this to the SMEM instance, but the entitiy 
> >>>>> relating to firmware is the remoteproc instance.
> >>>>
> >>>> I still do not understand why you have to add hwlock to remoteproc, even
> >>>> though it is not directly used. Your driver problem looks like lack of
> >>>> proper driver architecture - you want to control the locks not from the
> >>>> layer took the lock, but one layer up. Sorry, no, fix the driver
> >>>> architecture.
> >>>>
> >>>
> >>> No, it is the firmware's reference to the lock that is represented in
> >>> the remoteproc node, while SMEM deals with Linux's reference to the lock.
> >>>
> >>> This reference would be used to release the lock - on behalf of the
> >>> firmware - in the event that the firmware held it when it
> >>> stopped/crashed.
> >>
> >> I understood, but the remoteproc driver did not acquire the hardware
> >> lock. It was taken by smem, if I got it correctly, so you should poke
> >> smem to bust the spinlock.
> >>
> > 
> > The remoteproc instance is the closest representation of the entity that
> > took the lock (i.e. the firmware). SMEM here is just another consumer of
> > the same lock.
> > 
> >> The hwlock is not a property of remote proc, because remote proc does
> >> not care, right? Other device cares... and now for every smem user you
> >> will add new binding property?
> >>
> > 
> > Right, the issue seen relates to SMEM, because the remote processor (not
> > the remoteproc driver) took the lock.
> > 
> >> No, you are adding a binding based on your driver solution.
> > 
> > Similar to how hwspinlocks are used in other platforms (e.g. TI) the
> > firmware could take multiple locks, e.g. to synchronize access to other
> > shared memory mechanism (i.e. not SMEM). While I am not aware of such
> > use case today, my expectation was that in such case we just list all
> > the hwlocks related to the firmware and bust those from the remoteproc
> > instance.
> > 
> > Having to export APIs from each one of such drivers and make the
> > remoteproc identify the relevant instances and call those APIs does
> > indeed seem inconvenient.
> > SMEM is special here because it's singleton, but this would not
> > necessarily be true for other cases.
> 
> I don't think that exporting such API is unreasonable, but quite
> opposite - expected. The remote processor crashed, so the remoteproc
> driver is supposed to call some sort of smem_cleanup() or
> smem_cleanup_on_crash() and call would bust/release the lock. That way
> lock handling is encapsulated entirely in one driver which already takes
> and releases the lock.
> 

I don't agree.

SMEM does indeed acquire and release the same, shared, lock. But the
SMEM driver instance on our side is not involved in taking the lock for
the firmware.

There exist an equivalent SMEM driver instance in the firmware that
crashed and that's the thing that needs to be released.


We're also not tearing down, or cleaning up anything in our SMEM
instance. It is simply "when remoteproc id N died, check if N is holding
the lock and if so force a release of the lock - so that others can grab
it".

> Just like freeing any memory. remoteproc driver does not free other
> driver's memory only because processor crashed.
> 

That's a good comparison. Because when the firmware running on the
remote processor crashes, it is indeed the job of the remoteproc driver
to clean up the memory allocated to run the remote processor.

Regards,
Bjorn