Received-SPF: pass (google.com: domain of linux-kernel+bounces-120610-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Precedence: bulk
MIME-Version: 1.0
References: <20240325081335.2326979-1-jens.wiklander@linaro.org> <ZgLrQ7FtDy3INX8F@bogus>
In-Reply-To: <ZgLrQ7FtDy3INX8F@bogus>
From: Jens Wiklander <jens.wiklander@linaro.org>
Date: Wed, 27 Mar 2024 10:23:35 +0100
Message-ID: <CAHUa44HuuPE_cs3i4XEvHpSV+T0koCqBPo66uOmYyQ1=Rx=NWw@mail.gmail.com>
Subject: Re: [PATCH v2] firmware: arm_ffa: support running as a guest in a vm
To: Sudeep Holla <sudeep.holla@arm.com>
Cc: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, 
	Marc Bonnici <marc.bonnici@arm.com>, Olivier Deprez <Olivier.Deprez@arm.com>, 
	Lorenzo Pieralisi <lpieralisi@kernel.org>, Bertrand Marquis <Bertrand.Marquis@arm.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Mar 26, 2024 at 4:35=E2=80=AFPM Sudeep Holla <sudeep.holla@arm.com>=
 wrote:
>
> On Mon, Mar 25, 2024 at 09:13:35AM +0100, Jens Wiklander wrote:
> > Add support for running the driver in a guest to a hypervisor. The main
> > difference is introducing notification pending interrupt and that
> > FFA_NOTIFICATION_BITMAP_CREATE doesn't need to be called.
> >
> > The guest may need to use a notification pending interrupt instead of o=
r
> > in addition to the schedule receiver interrupt.
>
> The above statement makes me worry a bit that we are still not on the sam=
e
> page about NPI vs SRI. NPI need not exist in addition to SRI. And in v1
> you did mention you have SRI in the guest as well. Then why do we need
> NPI in addition to that. As part of SRI, the callback  ffa_self_notif_han=
dle
> gets registered and will be called as part of SRI handling. What you
> do in  notif_pend_irq_handler(), exactly what ffa_self_notif_handle()
> already does.

That's my understanding of what an NPI handler should do to be able to
receive per-vCPU notifications.

>
> I am still struggling to understand the usecase here. If you just have
> NPI and no SRI when running the driver in the VM, then it aligns with
> my understanding of possible use-case(not the one you mentioned in v1:
> where FF-A driver in VM will have SRI as OPTEE is the secondary scheduler=
)

OP-TEE is not a secondary scheduler. OP-TEE (the SP) is scheduled as
usual by the normal world using direct request. OP-TEE doesn't receive
FF-A notifications and I'm not sure it will ever be needed.

>
> If we are supporting NPI or SRI, I think we can see if we can further
> simplify this change, but I want to get to an agreement with usage model
> before we dig into implementation details in this patch.

The spec doesn't as far as I know explicitly make NPI and SRI mutually
exclusive, it doesn't make sense to use both in all configurations.
I'm trying to be as dynamic as possible when configuring the NPI and
SRI handlers.

If the kernel is a physical endpoint, it's easy, it only uses SRI and
the SPMC will not give an NPI when asked.

If the kernel is a virtual endpoint it might be more complicated since
a VM may need to act as a secondary scheduler. That's not fully
supported in this patch, since it can only schedule itself. SRI is not
used in my current configuration. If a hypervisor injects an SRI I
expect it to filter what's returned by FFA_NOTIFICATION_INFO_GET for
this VM so it doesn't interfere with notifications for other VMs.

In my current configuration, the hypervisor uses NPI to signal pending
notifications to the guest. I do not need a secondary scheduler since
OP-TEE doesn't receive notifications. At a later time, we may have SPs
that need to be scheduled, but that's not a problem I'm trying to
solve here.

Thanks,
Jens