Received-SPF: pass (google.com: domain of linux-kernel+bounces-143706-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Date: Sat, 13 Apr 2024 13:25:00 +0200
From: Greg KH <gregkh@linuxfoundation.org>
To: Vamsi Krishna Attunuru <vattunuru@marvell.com>
Cc: "arnd@arndb.de" <arnd@arndb.de>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Jerin Jacob <jerinj@marvell.com>
Subject: Re: [EXTERNAL] Re: [PATCH v5 1/1] misc: mrvl-cn10k-dpi: add Octeon
 CN10K DPI administrative driver
Message-ID: <2024041310-bundle-patio-8a22@gregkh>
References: <MW4PR18MB5244C76290A15737DC94FFDBA6042@MW4PR18MB5244.namprd18.prod.outlook.com>
 <20240412121005.1825881-1-vattunuru@marvell.com>
 <2024041250-nursing-tidy-db7e@gregkh>
 <MW4PR18MB5244FA7231C64F8A7928B83EA6042@MW4PR18MB5244.namprd18.prod.outlook.com>
 <2024041258-reminder-widely-00c0@gregkh>
 <MW4PR18MB52447D1E7F0CD0D1CA6BAACCA6042@MW4PR18MB5244.namprd18.prod.outlook.com>
 <2024041351-endowment-underrate-4b8d@gregkh>
 <MW4PR18MB52449EBFE3862D7C3065D22AA60B2@MW4PR18MB5244.namprd18.prod.outlook.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <MW4PR18MB52449EBFE3862D7C3065D22AA60B2@MW4PR18MB5244.namprd18.prod.outlook.com>

On Sat, Apr 13, 2024 at 10:58:37AM +0000, Vamsi Krishna Attunuru wrote:
> 
> 
> > -----Original Message-----
> > From: Greg KH <gregkh@linuxfoundation.org>
> > Sent: Saturday, April 13, 2024 11:18 AM
> > To: Vamsi Krishna Attunuru <vattunuru@marvell.com>
> > Cc: arnd@arndb.de; linux-kernel@vger.kernel.org; Jerin Jacob
> > <jerinj@marvell.com>
> > Subject: Re: [EXTERNAL] Re: [PATCH v5 1/1] misc: mrvl-cn10k-dpi: add
> > Octeon CN10K DPI administrative driver
> > 
> > On Fri, Apr 12, 2024 at 04:19:58PM +0000, Vamsi Krishna Attunuru wrote:
> > >
> > >
> > > > -----Original Message-----
> > > > From: Greg KH <gregkh@linuxfoundation.org>
> > > > Sent: Friday, April 12, 2024 9:05 PM
> > > > To: Vamsi Krishna Attunuru <vattunuru@marvell.com>
> > > > Cc: arnd@arndb.de; linux-kernel@vger.kernel.org
> > > > Subject: Re: [EXTERNAL] Re: [PATCH v5 1/1] misc: mrvl-cn10k-dpi: add
> > > > Octeon CN10K DPI administrative driver
> > > >
> > > > On Fri, Apr 12, 2024 at 01:56:36PM +0000, Vamsi Krishna Attunuru wrote:
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Greg KH <gregkh@linuxfoundation.org>
> > > > > > Sent: Friday, April 12, 2024 5:57 PM
> > > > > > To: Vamsi Krishna Attunuru <vattunuru@marvell.com>
> > > > > > Cc: arnd@arndb.de; linux-kernel@vger.kernel.org
> > > > > > Subject: [EXTERNAL] Re: [PATCH v5 1/1] misc: mrvl-cn10k-dpi: add
> > > > > > Octeon CN10K DPI administrative driver
> > > > > >
> > > > > > Prioritize security for external emails: Confirm sender and
> > > > > > content safety before clicking links or opening attachments
> > > > > >
> > > > > > ----------------------------------------------------------------
> > > > > > ----
> > > > > > -- On Fri, Apr 12, 2024 at 05:10:05AM -0700, Vamsi Attunuru wrote:
> > > > > > > Adds a misc driver for Marvell CN10K DPI(DMA Engine) device's
> > > > > > > physical function which initializes DPI DMA hardware's global
> > > > > > > configuration and enables hardware mailbox channels between
> > > > > > > physical function (PF) and it's virtual functions (VF). VF
> > > > > > > device drivers (User space drivers) use this hw mailbox to
> > > > > > > communicate any required device configuration on it's respective
> > VF device.
> > > > > > > Accordingly, this DPI PF driver provisions the VF device resources.
> > > > > > >
> > > > > > > At the hardware level, the DPI physical function (PF) acts as
> > > > > > > a management interface to setup the VF device resources, VF
> > > > > > > devices are only provisioned to handle or control the actual
> > > > > > > DMA Engine's data transfer
> > > > > > capabilities.
> > > > > >
> > > > > > No pointer to the userspace code that uses this?  Why not?  How
> > > > > > are we supposed to be able to review this?
> > > > >
> > > > > Userspace code will use two functionalities (mailbox & ioctl) from
> > > > > this driver. DPDK DMA driver uses the mailbox and the dpdk
> > > > > application uses the ioctl to setup the device attributes. We are
> > > > > waiting for this kernel driver get merged  to update the
> > > > > corresponding support in DPDK
> > > > driver and applications. I will provide the pointers to both the use
> > > > cases in userspace code.
> > > > > Meanwhile below is the current dpdk dma driver that uses sysfs
> > > > > based scheme to convey mbox requests to the kernel DPI driver
> > > > > which gets
> > > > replaced with hardware mailbox scheme once mrvl-cn10k-dpi kernel
> > > > driver is merged.
> > > > > https://urldefense.proofpoint.com/v2/url?u=https-
> > > > 3A__github.com_DPDK_d
> > > > > pdk_blob_main_drivers_common_cnxk_roc-
> > > > 5Fdpi.c&d=DwIBAg&c=nKjWec2b6R0mO
> > > > >
> > > >
> > yPaz7xtfQ&r=WllrYaumVkxaWjgKto6E_rtDQshhIhik2jkvzFyRhW8&m=o3EhoL
> > > > s7dsod
> > > > > -YHS438Wl2Pf_MKMBYegGSKteoX3qFTB0HV897ykpCVbTp-
> > > > nmj4e&s=A6TJDFUtPm3ksJh
> > > > > qop89CL8GgKj4sjkJIVi1-RdnUr8&e=
> > > >
> > > > So this is a DPDK thing?  Ugh, do the networking people know about this?
> > > > If not, why aren't they reviewing this?
> > >
> > > Actually, It's not networking related. Like the Linux kernel, DPDK
> > > also supports multiple subsystems like network, scheduler, DMA,
> > > mempool etc. Regarding the usecases, the DPDK Marvell DMA/DPI VF
> > > driver interacts(over hardware mailbox) with the mrvl-cn10k-dpi misc
> > kernel driver(administrative driver) for setting up the VF device resources.
> > 
> > So this is something that the PCI core should be concerned about then?
> 
> No, it's a normal PCIe sriov capability implemented in all sriov capable PCIe devices.
> Our PF device aka this driver in kernel space service mailbox requests from userspace
> applications via VF devices. For instance, DPI VF device from user space writes into
> mailbox registers and the DPI hardware triggers an interrupt to DPI PF device.
> Upon PF interrupt, this driver services the mailbox requests.

Isn't that a "normal" PCI thing?  How is this different from other
devices that have VF?

> > > DPDK is one example that uses this driver, there can be other
> > > userspace generic frameworks/applications where the virtual functions
> > > are binded to userspace drivers and interact with physical/administrative
> > function driver running in the kernel.
> > 
> > Are there other devices/drivers that do this today in Linux?  Why make a
> > device-specific api for this common functionality?
> 
> The apis defined in this driver are specific to Marvell DPI hardware.

The api, yes, but that's the point, shouldn't this be generic for all
hardware that supports this?  Implementation should be device specific,
in the driver.

> For instance,
> the variables molr(max outstanding load requests), fifo_max, ebus_port are
> DPI hardware specific. Generally, drivers use driver-specific api to configure
> any device-specific configuration which does not fit in common functionality right.
> 
> Mailbox operations like dpi_queue_open & close are requests sent from VF device
> to PF device for setting up the VF queue resources.

Why is an ioctl to a random character device the correct api to
userspace for this type of thing?  Shouldn't this go through the PCI
layer api instead?

thanks,

greg k-h