Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: FiL2a7yHDTqBRkS4vHB9fHZLgRgUPVzaV68RQfOBlSdzV22M13BkAaaGyeN1M+iN1IUAhUjFbm
 pE6+NAfUMrGg==
IronPort-SDR: rD8RyL6Rjdj3yDeg4jtsl23revPuXfiJoM/iWQyvyxNMPYhdf81mDfOBm0cxeYPlzvyRlPrdJt
 32POz5+A5pmg==
From:   "Wu, Hao" <hao.wu@intel.com>
To:     "Xu, Yilun" <yilun.xu@intel.com>,
        "mdf@kernel.org" <mdf@kernel.org>,
        "linux-fpga@vger.kernel.org" <linux-fpga@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
CC:     "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        "trix@redhat.com" <trix@redhat.com>,
        "lgoncalv@redhat.com" <lgoncalv@redhat.com>
Subject: RE: [PATCH v3 2/3] fpga: dfl: add the userspace I/O device support
 for DFL devices
Thread-Topic: [PATCH v3 2/3] fpga: dfl: add the userspace I/O device support
 for DFL devices
Thread-Index: AQHW1DhiFXQvRNubJU6hP8U7m1i7W6n8ewGg
Date:   Fri, 18 Dec 2020 08:05:27 +0000
Message-ID: <DM6PR11MB381997F5E297408A68BF5F2185C30@DM6PR11MB3819.namprd11.prod.outlook.com>
References: <1608183881-18692-1-git-send-email-yilun.xu@intel.com>
 <1608183881-18692-3-git-send-email-yilun.xu@intel.com>
In-Reply-To: <1608183881-18692-3-git-send-email-yilun.xu@intel.com>
Accept-Language: en-US
Content-Language: en-US
dlp-product: dlpe-windows
dlp-reaction: no-action
dlp-version: 11.5.1.3
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: e0966a67-f4ef-48b4-7e5f-08d8a32bae7d
X-MS-Exchange-CrossTenant-originalarrivaltime: 18 Dec 2020 08:05:27.7947
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: AncBp/a08JiyXzWLsf8q4gOcPeqn+hlA4tAe2S0YYTXuQ8xURQeVfdVYLRW2OUNeAuVSsy/r/uVoM2+xUp/sYQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR1101MB2076
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> Subject: [PATCH v3 2/3] fpga: dfl: add the userspace I/O device support f=
or
> DFL devices
>=20
> This patch supports the DFL drivers be written in userspace. This is
> realized by exposing the userspace I/O device interfaces.
>=20
> The driver leverages the uio_pdrv_genirq, it adds the uio_pdrv_genirq
> platform device with the DFL device's resources, and let the generic UIO
> platform device driver provide support to userspace access to kernel
> interrupts and memory locations.
>=20
> The driver matches DFL devices in a different way. It has no device id
> table, instead it matches any DFL device which could not be handled by
> other DFL drivers.

Looks like we want to build UIO driver as the default/generic driver for DF=
L,
it seems fine but my concern is that UIO has its own limitation, if some da=
y,
dfl device is extended, but UIO has limitation, then we may need to select=
=20
another one as the default driver.. or we can just match them using=20
id_table as we know UIO meets the requirement for those DFL devices?

>=20
> Signed-off-by: Xu Yilun <yilun.xu@intel.com>
> ---
> v2: switch to the new matching algorithem. It matches DFL devices which
>      could not be handled by other DFL drivers.
>     refacor the code about device resources filling.
>     fix some comments.
> v3: split the dfl.c changes out of this patch.
>     some minor fixes
> ---
>  drivers/fpga/Kconfig        |  10 ++++
>  drivers/fpga/Makefile       |   1 +
>  drivers/fpga/dfl-uio-pdev.c | 110
> ++++++++++++++++++++++++++++++++++++++++++++
>  3 files changed, 121 insertions(+)
>  create mode 100644 drivers/fpga/dfl-uio-pdev.c
>=20
> diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig
> index 5d7f0ae..7a88af9 100644
> --- a/drivers/fpga/Kconfig
> +++ b/drivers/fpga/Kconfig
> @@ -202,6 +202,16 @@ config FPGA_DFL_NIOS_INTEL_PAC_N3000
>  	  the card. It also instantiates the SPI master (spi-altera) for
>  	  the card's BMC (Board Management Controller).
>=20
> +config FPGA_DFL_UIO_PDEV
> +	tristate "FPGA DFL Driver for Userspace I/O platform devices"
> +	depends on FPGA_DFL && UIO_PDRV_GENIRQ
> +	help
> +	  Enable this to allow some DFL drivers be written in userspace. It
> +	  adds the uio_pdrv_genirq platform device with the DFL feature's
> +	  resources, and lets the generic UIO platform device driver provide
> +	  support for userspace access to kernel interrupts and memory
> +	  locations.

If we consider this as a default driver for everybody in DFL, then we could
consider build it into the core, otherwise it always requires to be loaded
manually, right?

> +
>  config FPGA_DFL_PCI
>  	tristate "FPGA DFL PCIe Device Driver"
>  	depends on PCI && FPGA_DFL
> diff --git a/drivers/fpga/Makefile b/drivers/fpga/Makefile
> index 18dc9885..8847fe0 100644
> --- a/drivers/fpga/Makefile
> +++ b/drivers/fpga/Makefile
> @@ -45,6 +45,7 @@ dfl-afu-objs :=3D dfl-afu-main.o dfl-afu-region.o dfl-a=
fu-
> dma-region.o
>  dfl-afu-objs +=3D dfl-afu-error.o
>=20
>  obj-$(CONFIG_FPGA_DFL_NIOS_INTEL_PAC_N3000)	+=3D dfl-n3000-nios.o
> +obj-$(CONFIG_FPGA_DFL_UIO_PDEV)		+=3D dfl-uio-pdev.o
>=20
>  # Drivers for FPGAs which implement DFL
>  obj-$(CONFIG_FPGA_DFL_PCI)		+=3D dfl-pci.o
> diff --git a/drivers/fpga/dfl-uio-pdev.c b/drivers/fpga/dfl-uio-pdev.c
> new file mode 100644
> index 0000000..8c57233
> --- /dev/null
> +++ b/drivers/fpga/dfl-uio-pdev.c
> @@ -0,0 +1,110 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * DFL driver for Userspace I/O platform devices
> + *
> + * Copyright (C) 2020 Intel Corporation, Inc.
> + */
> +#include <linux/dfl.h>
> +#include <linux/errno.h>
> +#include <linux/kernel.h>
> +#include <linux/module.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include <linux/uio_driver.h>
> +
> +#include "dfl.h"
> +
> +#define DRIVER_NAME "dfl-uio-pdev"
> +
> +static struct dfl_driver dfl_uio_pdev_driver;
> +
> +static int check_for_other_drv_match(struct device_driver *drv, void *da=
ta)
> +{
> +	struct dfl_driver *ddrv =3D to_dfl_drv(drv);
> +	struct dfl_device *ddev =3D data;
> +
> +	/* skip myself */
> +	if (ddrv =3D=3D &dfl_uio_pdev_driver)
> +		return 0;
> +
> +	return dfl_match_device(ddev, ddrv);
> +}
> +
> +static int dfl_uio_pdev_match(struct dfl_device *ddev)
> +{
> +	/*
> +	 * If any other driver wants the device, leave the device to this other
> +	 * driver.
> +	 */
> +	if (bus_for_each_drv(&dfl_bus_type, NULL, ddev,
> check_for_other_drv_match))
> +		return 0;
> +
> +	return 1;
> +}
> +
> +static int dfl_uio_pdev_probe(struct dfl_device *ddev)
> +{
> +	struct device *dev =3D &ddev->dev;
> +	struct platform_device_info pdevinfo =3D { 0 };
> +	struct uio_info uio_pdata =3D { 0 };
> +	struct platform_device *uio_pdev;
> +	struct resource *res;
> +	int i;
> +
> +	pdevinfo.name =3D "uio_pdrv_genirq";
> +
> +	res =3D kcalloc(ddev->num_irqs + 1, sizeof(*res), GFP_KERNEL);
> +	if (!res)
> +		return -ENOMEM;
> +
> +	res[0].parent =3D &ddev->mmio_res;
> +	res[0].flags =3D IORESOURCE_MEM;
> +	res[0].start =3D ddev->mmio_res.start;
> +	res[0].end =3D ddev->mmio_res.end;
> +
> +	/* then add irq resource */
> +	for (i =3D 0; i < ddev->num_irqs; i++) {
> +		res[i + 1].flags =3D IORESOURCE_IRQ;
> +		res[i + 1].start =3D ddev->irqs[i];
> +		res[i + 1].end =3D ddev->irqs[i];
> +	}

How many interrupts UIO could support? or we need some
warning or just even return error here?

Thanks
Hao

> +
> +	uio_pdata.name =3D DRIVER_NAME;
> +	uio_pdata.version =3D "0";
> +
> +	pdevinfo.res =3D res;
> +	pdevinfo.num_res =3D ddev->num_irqs + 1;
> +	pdevinfo.parent =3D &ddev->dev;
> +	pdevinfo.id =3D PLATFORM_DEVID_AUTO;
> +	pdevinfo.data =3D &uio_pdata;
> +	pdevinfo.size_data =3D sizeof(uio_pdata);
> +
> +	uio_pdev =3D platform_device_register_full(&pdevinfo);
> +	if (!IS_ERR(uio_pdev))
> +		dev_set_drvdata(dev, uio_pdev);
> +
> +	kfree(res);
> +
> +	return PTR_ERR_OR_ZERO(uio_pdev);
> +}
> +
> +static void dfl_uio_pdev_remove(struct dfl_device *ddev)
> +{
> +	struct platform_device *uio_pdev =3D dev_get_drvdata(&ddev->dev);
> +
> +	platform_device_unregister(uio_pdev);
> +}
> +
> +static struct dfl_driver dfl_uio_pdev_driver =3D {
> +	.drv	=3D {
> +		.name       =3D DRIVER_NAME,
> +	},
> +	.match	=3D dfl_uio_pdev_match,
> +	.probe	=3D dfl_uio_pdev_probe,
> +	.remove	=3D dfl_uio_pdev_remove,
> +};
> +module_dfl_driver(dfl_uio_pdev_driver);
> +
> +MODULE_DESCRIPTION("DFL driver for Userspace I/O platform devices");
> +MODULE_AUTHOR("Intel Corporation");
> +MODULE_LICENSE("GPL v2");
> --
> 2.7.4