Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20201023092650.GB29066@infradead.org> <VI1PR04MB4960E9ECD7310B8CA1E053DC92190@VI1PR04MB4960.eurprd04.prod.outlook.com>
 <20201027062802.GC207971@kroah.com> <VI1PR04MB4960C0E76374B2775D99A82192160@VI1PR04MB4960.eurprd04.prod.outlook.com>
 <20201027151106.e4skr6dsbwvo4al6@axis.com> <VI1PR04MB49603783EF3FD3F3635FCDAF92170@VI1PR04MB4960.eurprd04.prod.outlook.com>
 <93bd1c60ea4d910489a7592200856eaf8022ced0.camel@intel.com>
 <AM0PR04MB4947F01860DE953B8496FA8892170@AM0PR04MB4947.eurprd04.prod.outlook.com>
 <CAK8P3a1JRx32VfFcwFpK0i6F5MQMCK-yCKw8=d_R08Y3iQ7wLQ@mail.gmail.com>
 <CAK8P3a3u06ZHdAb_n3byTqfxAvy_wi48X1g0N4ODuH2uEM0xLA@mail.gmail.com> <20201029100727.trbppgbusd5vogpz@axis.com>
In-Reply-To: <20201029100727.trbppgbusd5vogpz@axis.com>
From:   Arnd Bergmann <arnd@kernel.org>
Date:   Thu, 29 Oct 2020 12:53:09 +0100
Message-ID: <CAK8P3a3T2Riy-vfd8RcdWeeER4usc2m78rkmx4Q_8N3zGA6r_A@mail.gmail.com>
Subject: Re: [PATCH V3 2/4] misc: vop: do not allocate and reassign the used ring
To:     Vincent Whitchurch <vincent.whitchurch@axis.com>
Cc:     Sherry Sun <sherry.sun@nxp.com>,
        "Dutt, Sudeep" <sudeep.dutt@intel.com>,
        dl-linux-imx <linux-imx@nxp.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "hch@infradead.org" <hch@infradead.org>,
        "kishon@ti.com" <kishon@ti.com>,
        "lorenzo.pieralisi@arm.com" <lorenzo.pieralisi@arm.com>,
        "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        "Dixit, Ashutosh" <ashutosh.dixit@intel.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, Oct 29, 2020 at 11:07 AM Vincent Whitchurch
<vincent.whitchurch@axis.com> wrote:
>
> On Wed, Oct 28, 2020 at 04:50:36PM +0100, Arnd Bergmann wrote:
> > I think we should try to do something on top of the PCIe endpoint subsystem
> > to make it work across arbitrary combinations of host and device
> > implementations,
> > and provide a superset of what the MIC driver, (out-of-tree) Bluefield endpoint
> > driver, and the NTB subsystem as well as a couple of others used to do,
> > each of them tunneling block/network/serial/... over a PCIe link of some
> > sort, usually with virtio.
>
> VOP is not PCIe-specific (as demonstrated by the vop-loopback patches I
> posted a while ago [1]), and it would be a shame for a replacement to be
> tied to the PCIe endpoint subsystem.  There are many SOCs out there
> which have multiple Linux-capable processors without cache-coherency
> between them.  VOP is (or should I say was since I guess it's being
> deleted) the closest we have in mainline to easily get generic virtio
> (and not just rpmsg) running between these kind of Linux instances.  If
> a new replacement framework were to be PCIe-exclusive then we'd have to
> invent one more framework for non-PCIe links to do pretty much the same
> thing.
>
> [1] https://lore.kernel.org/lkml/20190403104746.16063-1-vincent.whitchurch@axis.com/

Right, sorry I forgot about that. I think this means we should keep having
an abstraction between VOP (under whichever name) and the lower levels,
and be aware that it might run on any number of these:

- PCIe endpoint, with the endpoint controlling the virtio configuration
- PCIe endpoint, with the host (the side that has the pci_driver) controlling
  the virtio configuration
- NTB connections
- your  loopback mode
- Virtio tunnels between VM guests (see https://www.linaro.org/projects/#STR)
- Intel MIC (to be removed, but it would be wrong to make assumptions that
  cannot be made on that type of hardware)
- ...

The existing VOP codebase does look like a reasonable start, though
I think we need to discuss whether the ioctl interface should be
replaced with a configfs interface, and what other changes would
be needed to make it support the generalized hardware case.

       Arnd