Received-SPF: pass (google.com: domain of linux-kernel+bounces-129774-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Date: Wed, 3 Apr 2024 14:03:21 +0100
From: Sudeep Holla <sudeep.holla@arm.com>
To: Jens Wiklander <jens.wiklander@linaro.org>
Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
	Marc Bonnici <marc.bonnici@arm.com>,
	Olivier Deprez <Olivier.Deprez@arm.com>,
	Lorenzo Pieralisi <lpieralisi@kernel.org>,
	Bertrand Marquis <Bertrand.Marquis@arm.com>
Subject: Re: [PATCH v2] firmware: arm_ffa: support running as a guest in a vm
Message-ID: <Zg1TmTcqRbzla3nN@bogus>
References: <20240325081335.2326979-1-jens.wiklander@linaro.org>
 <ZgLrQ7FtDy3INX8F@bogus>
 <CAHUa44HuuPE_cs3i4XEvHpSV+T0koCqBPo66uOmYyQ1=Rx=NWw@mail.gmail.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <CAHUa44HuuPE_cs3i4XEvHpSV+T0koCqBPo66uOmYyQ1=Rx=NWw@mail.gmail.com>

On Wed, Mar 27, 2024 at 10:23:35AM +0100, Jens Wiklander wrote:
> On Tue, Mar 26, 2024 at 4:35 PM Sudeep Holla <sudeep.holla@arm.com> wrote:
> >
> > On Mon, Mar 25, 2024 at 09:13:35AM +0100, Jens Wiklander wrote:
> > > Add support for running the driver in a guest to a hypervisor. The main
> > > difference is introducing notification pending interrupt and that
> > > FFA_NOTIFICATION_BITMAP_CREATE doesn't need to be called.
> > >
> > > The guest may need to use a notification pending interrupt instead of or
> > > in addition to the schedule receiver interrupt.
> >
> > The above statement makes me worry a bit that we are still not on the same
> > page about NPI vs SRI. NPI need not exist in addition to SRI. And in v1
> > you did mention you have SRI in the guest as well. Then why do we need
> > NPI in addition to that. As part of SRI, the callback  ffa_self_notif_handle
> > gets registered and will be called as part of SRI handling. What you
> > do in  notif_pend_irq_handler(), exactly what ffa_self_notif_handle()
> > already does.
> 
> That's my understanding of what an NPI handler should do to be able to
> receive per-vCPU notifications.
> 
> >
> > I am still struggling to understand the usecase here. If you just have
> > NPI and no SRI when running the driver in the VM, then it aligns with
> > my understanding of possible use-case(not the one you mentioned in v1:
> > where FF-A driver in VM will have SRI as OPTEE is the secondary scheduler)
> 
> OP-TEE is not a secondary scheduler. OP-TEE (the SP) is scheduled as
> usual by the normal world using direct request. OP-TEE doesn't receive
> FF-A notifications and I'm not sure it will ever be needed.
>

Sorry for my poor choice of words yet again. I meant VM kernel(running
as NS virtual endpoint) with OPTEE driver running in it as secondary
scheduler. IIUC, there will be another instance of OPTEE driver in the
primary scheduler endpoint(i.e. host kernel) and it will take care of
running SRI handler ?

> >
> > If we are supporting NPI or SRI, I think we can see if we can further
> > simplify this change, but I want to get to an agreement with usage model
> > before we dig into implementation details in this patch.
>
> The spec doesn't as far as I know explicitly make NPI and SRI mutually
> exclusive, it doesn't make sense to use both in all configurations.
> I'm trying to be as dynamic as possible when configuring the NPI and
> SRI handlers.
>

Fair enough

> If the kernel is a physical endpoint, it's easy, it only uses SRI and
> the SPMC will not give an NPI when asked.
>

Agreed.

> If the kernel is a virtual endpoint it might be more complicated since
> a VM may need to act as a secondary scheduler. That's not fully
> supported in this patch, since it can only schedule itself. SRI is not
> used in my current configuration. If a hypervisor injects an SRI I
> expect it to filter what's returned by FFA_NOTIFICATION_INFO_GET for
> this VM so it doesn't interfere with notifications for other VMs.
>

OK

> In my current configuration, the hypervisor uses NPI to signal pending
> notifications to the guest. I do not need a secondary scheduler since
> OP-TEE doesn't receive notifications. At a later time, we may have SPs
> that need to be scheduled, but that's not a problem I'm trying to
> solve here.

Understood. I will take a look at the patch with the above information.

--
Regards,
Sudeep