Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Feedback-ID: i1568416f:Fastmail
Date:   Tue, 4 Jul 2023 13:43:46 +0200
From:   Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= 
        <marmarek@invisiblethingslab.com>
To:     Juergen Gross <jgross@suse.com>
Cc:     Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
        Petr Pavlu <petr.pavlu@suse.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        vikram.garhwal@amd.com
Subject: Re: [PATCH 2/2] xen/virtio: Avoid use of the dom0 backend in dom0
Message-ID: <ZKQF8qq8Oh5E+Fxx@mail-itl>
References: <20230621131214.9398-1-petr.pavlu@suse.com>
 <20230621131214.9398-3-petr.pavlu@suse.com>
 <15e31609-6c45-7372-76ee-0adf7a64fe88@epam.com>
 <alpine.DEB.2.22.394.2306281745010.3936094@ubuntu-linux-20-04-desktop>
 <b21398eb-2fb2-4fca-dd90-d2c81d8df1c4@epam.com>
 <alpine.DEB.2.22.394.2306291502150.3936094@ubuntu-linux-20-04-desktop>
 <ZKPO5WbFGblXU5hX@MacBook-Air-de-Roger.local>
 <e5eda26f-9a31-dc3a-d97f-33d2efa20efa@suse.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
        protocol="application/pgp-signature"; boundary="bKEul5l8IcOXfCYH"
Content-Disposition: inline
In-Reply-To: <e5eda26f-9a31-dc3a-d97f-33d2efa20efa@suse.com>
Precedence: bulk


--bKEul5l8IcOXfCYH
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 4 Jul 2023 13:43:46 +0200
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Juergen Gross <jgross@suse.com>
Cc: Roger Pau =?utf-8?B?TW9ubsOp?= <roger.pau@citrix.com>,
	Stefano Stabellini <sstabellini@kernel.org>,
	Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
	Petr Pavlu <petr.pavlu@suse.com>,
	"xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	vikram.garhwal@amd.com
Subject: Re: [PATCH 2/2] xen/virtio: Avoid use of the dom0 backend in dom0

Hi,

FWIW, I have ran into this issue some time ago too. I run Xen on top of
KVM and then passthrough some of the virtio devices (network one
specifically) into a (PV) guest. So, I hit both cases, the dom0 one and
domU one. As a temporary workaround I needed to disable
CONFIG_XEN_VIRTIO completely (just disabling
CONFIG_XEN_VIRTIO_FORCE_GRANT was not enough to fix it).
With that context in place, the actual response below.

On Tue, Jul 04, 2023 at 12:39:40PM +0200, Juergen Gross wrote:
> On 04.07.23 09:48, Roger Pau Monn=C3=A9 wrote:
> > On Thu, Jun 29, 2023 at 03:44:04PM -0700, Stefano Stabellini wrote:
> > > On Thu, 29 Jun 2023, Oleksandr Tyshchenko wrote:
> > > > On 29.06.23 04:00, Stefano Stabellini wrote:
> > > > > I think we need to add a second way? It could be anything that ca=
n help
> > > > > us distinguish between a non-grants-capable virtio backend and a
> > > > > grants-capable virtio backend, such as:
> > > > > - a string on xenstore
> > > > > - a xen param
> > > > > - a special PCI configuration register value
> > > > > - something in the ACPI tables
> > > > > - the QEMU machine type
> > > >=20
> > > >=20
> > > > Yes, I remember there was a discussion regarding that. The point is=
 to
> > > > choose a solution to be functional for both PV and HVM *and* to be =
able
> > > > to support a hotplug. IIRC, the xenstore could be a possible candid=
ate.
> > >=20
> > > xenstore would be among the easiest to make work. The only downside is
> > > the dependency on xenstore which otherwise virtio+grants doesn't have.
> >=20
> > I would avoid introducing a dependency on xenstore, if nothing else we
> > know it's a performance bottleneck.
> >=20
> > We would also need to map the virtio device topology into xenstore, so
> > that we can pass different options for each device.
>=20
> This aspect (different options) is important. How do you want to pass vir=
tio
> device configuration parameters from dom0 to the virtio backend domain? Y=
ou
> probably need something like Xenstore (a virtio based alternative like vi=
rtiofs
> would work, too) for that purpose.
>=20
> Mapping the topology should be rather easy via the PCI-Id, e.g.:
>=20
> /local/domain/42/device/virtio/0000:00:1c.0/backend

While I agree this would probably be the simplest to implement, I don't
like introducing xenstore dependency into virtio frontend either.
Toolstack -> backend communication is probably easier to solve, as it's
much more flexible (could use qemu cmdline, QMP, other similar
mechanisms for non-qemu backends etc).

> > > Vikram is working on virtio with grants support in QEMU as we speak.
> > > Maybe we could find a way to add a flag in QEMU so that we can detect=
 at
> > > runtime if a given virtio device support grants or not.
> >=20
> > Isn't there a way for the device to expose capabilities already?  For
> > example how does a virtio-blk backend expose support for indirect
> > descriptors?
>=20
> Those capabilities are defined in the virtio spec [1]. Adding the backend
> domid would be possible, but it probably wouldn't be that easy (requires
> changing the virtio spec by either expanding an existing config area or by
> adding a new one). I'm not sure handling in the specific frontends is
> generic enough for being able to have a central place where the backend
> domid could be retrieved, without requiring any change of the frontends.

IMHO the proper solution is to extend the spec. I don't have much
experience with virtio code, but reading the spec it looks like new
config area will be better for compatibility/uniform handling in a
frontent-agnostic way. Since it will definitely take time, some
transitional solution (maybe even xenstore...) might be warranted.

> Juergen
>=20
> [1]: http://docs.oasis-open.org/virtio/virtio/v1.2/virtio-v1.2.html


--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--bKEul5l8IcOXfCYH
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmSkBfIACgkQ24/THMrX
1yyXUgf+N/PLUJAoNhlNjh4kow5XxRzM4Afaw1bwhZYI6546o2bL/yRgGWcBqNue
nNZzUUmNMsNvEi0zy/1rWQppCaDZKPhgtb33Ty9oC9iSBeziB9LQk8A8yOMiQ0st
hazKZOfD2GBRpfULOfUCOj6+9AQWadmnHAbKT0gJMSARp3shkfN7kZoL4xczD9tn
2JKsIIBERXc/MpMh7gQwM0F6EBf00Zqi6Qvss8/IKtNuUSI9u7xQLOWrxscUXKCc
OoDe7b4jhgRpRR3BZoiON6khBEKBt59UcZR+mQll0h9XeTH7n0MyaZO5SASVdD3m
L1RNV4X+MQ8fUplS1LMJtKBg3j9pBA==
=CQd8
-----END PGP SIGNATURE-----

--bKEul5l8IcOXfCYH--