Received-SPF: pass (google.com: domain of linux-kernel+bounces-144081-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Date: Sun, 14 Apr 2024 11:46:25 +0200
From: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
To: Vamsi Krishna Attunuru <vattunuru@marvell.com>
Cc: Arnd Bergmann <arnd@arndb.de>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Jerin Jacob <jerinj@marvell.com>
Subject: Re: [EXTERNAL] Re: [PATCH v5 1/1] misc: mrvl-cn10k-dpi: add Octeon
 CN10K DPI administrative driver
Message-ID: <2024041458-directory-image-6a0c@gregkh>
References: <2024041250-nursing-tidy-db7e@gregkh>
 <MW4PR18MB5244FA7231C64F8A7928B83EA6042@MW4PR18MB5244.namprd18.prod.outlook.com>
 <2024041258-reminder-widely-00c0@gregkh>
 <MW4PR18MB52447D1E7F0CD0D1CA6BAACCA6042@MW4PR18MB5244.namprd18.prod.outlook.com>
 <2024041351-endowment-underrate-4b8d@gregkh>
 <MW4PR18MB52449EBFE3862D7C3065D22AA60B2@MW4PR18MB5244.namprd18.prod.outlook.com>
 <2024041310-bundle-patio-8a22@gregkh>
 <MW4PR18MB5244807D8DE91CA01EEB91B5A60B2@MW4PR18MB5244.namprd18.prod.outlook.com>
 <b5d400e6-9e4a-4f5d-b48f-a75c65ec1ae8@app.fastmail.com>
 <MW4PR18MB52449BD843670B347C0383EDA60A2@MW4PR18MB5244.namprd18.prod.outlook.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <MW4PR18MB52449BD843670B347C0383EDA60A2@MW4PR18MB5244.namprd18.prod.outlook.com>

On Sun, Apr 14, 2024 at 09:33:37AM +0000, Vamsi Krishna Attunuru wrote:
> 
> 
> > -----Original Message-----
> > From: Arnd Bergmann <arnd@arndb.de>
> > Sent: Sunday, April 14, 2024 12:41 AM
> > To: Vamsi Krishna Attunuru <vattunuru@marvell.com>; Greg Kroah-Hartman
> > <gregkh@linuxfoundation.org>
> > Cc: linux-kernel@vger.kernel.org; Jerin Jacob <jerinj@marvell.com>
> > Subject: Re: [EXTERNAL] Re: [PATCH v5 1/1] misc: mrvl-cn10k-dpi: add
> > Octeon CN10K DPI administrative driver
> > 
> > On Sat, Apr 13, 2024, at 18:17, Vamsi Krishna Attunuru wrote:
> > > From: Greg KH <gregkh@linuxfoundation.org>
> > >> On Sat, Apr 13, 2024 at 10:58:37AM +0000, Vamsi Krishna Attunuru wrote:
> > >> > From: Greg KH <gregkh@linuxfoundation.org>
> > >> >
> > >> > No, it's a normal PCIe sriov capability implemented in all sriov
> > >> > capable PCIe
> > >> devices.
> > >> > Our PF device aka this driver in kernel space service mailbox
> > >> > requests from userspace applications via VF devices. For instance,
> > >> > DPI VF device from user space writes into mailbox registers and the
> > >> > DPI hardware
> > >> triggers an interrupt to DPI PF device.
> > >> > Upon PF interrupt, this driver services the mailbox requests.
> > >>
> > >> Isn't that a "normal" PCI thing?  How is this different from other
> > >> devices that have VF?
> > >
> > > Looks like there is a lot of confusion for this device. Let me explain
> > > There are two aspects for this DPI PF device.
> > > a) It's a PCIe device so it is "using" some of the PCI services
> > > provided PCIe HW or PCI subsystem
> > > b) It is "providing" non PCIe service(DPI HW administrative function)
> > > by using (a) Let me enumerate PF device operations with above aspects.
> > > 1) Means to create VF(s) from PF. It's category (a) service and driver
> > > uses API (pci_sriov_configure_simple()) from PCI subsystem to
> > > implement it.
> > > 2) Means to get the interrupt(mailbox or any device specific
> > > interrupt). It's category (a) service and driver uses API
> > > (pci_alloc_irq_vectors()) from PCI subsystem to implement it.
> > > 3) Means to get the mailbox content from VF by using (2). It's
> > > category
> > > (b) service. This service is not part of PCI specification.
> > > DPI PF device has the mailbox registers(DPI_MBOX_PF_VF_DATA
> > registers)
> > > in its PCIe BAR space which are device specific.
> > > 4) Upon receiving DPI HW administrative function mailbox request,
> > > service it. Its category (b) service. This service is not part of PCI
> > > specification.
> > > For instance, dpi_queue_open & close are requests sent from DPI VF
> > > device to DPI PF device for setting up the DPI VF queue resources.
> > > Once its setup by DPI PF, then DPI VF device can use these queues.
> > > These queues are not part of PCIe specification. These queues are used
> > > for making DMA by DPI VF device/driver.
> > 
> > It's not directly my area either, but as far as I can tell from reading the
> > competing sr-iov based device drivers, these seem to handle all of the above
> > in the network driver that owns the PF rather than a separate driver, e.g. for
> > the first point:
> > 
> > $ git grep -w sriov_configure.= drivers/net/
> > drivers/net/ethernet/amazon/ena/ena_netdev.c:   .sriov_configure =
> > pci_sriov_configure_simple,
> > drivers/net/ethernet/amd/pds_core/main.c:       .sriov_configure =
> > pdsc_sriov_configure,
> > drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c:       .sriov_configure =
> > bnx2x_sriov_configure,
> > drivers/net/ethernet/broadcom/bnxt/bnxt.c:      .sriov_configure =
> > bnxt_sriov_configure,
> > drivers/net/ethernet/cavium/liquidio/lio_main.c:        .sriov_configure =
> > liquidio_enable_sriov,
> > drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c:        .sriov_configure =
> > cxgb4_iov_configure,
> > drivers/net/ethernet/emulex/benet/be_main.c:    .sriov_configure =
> > be_pci_sriov_configure,
> > drivers/net/ethernet/freescale/enetc/enetc_pf.c:        .sriov_configure =
> > enetc_sriov_configure,
> > drivers/net/ethernet/fungible/funeth/funeth_main.c:     .sriov_configure =
> > funeth_sriov_configure,
> > drivers/net/ethernet/hisilicon/hns3/hns3_enet.c:        .sriov_configure =
> > hns3_pci_sriov_configure,
> > drivers/net/ethernet/huawei/hinic/hinic_main.c: .sriov_configure =
> > hinic_pci_sriov_configure,
> > drivers/net/ethernet/intel/fm10k/fm10k_pci.c:   .sriov_configure        =
> > fm10k_iov_configure,
> > drivers/net/ethernet/intel/i40e/i40e_main.c:    .sriov_configure =
> > i40e_pci_sriov_configure,
> > drivers/net/ethernet/intel/ice/ice_main.c:      .sriov_configure =
> > ice_sriov_configure,
> > drivers/net/ethernet/intel/idpf/idpf_main.c:    .sriov_configure        =
> > idpf_sriov_configure,
> > drivers/net/ethernet/intel/igb/igb_main.c:      .sriov_configure =
> > igb_pci_sriov_configure,
> > drivers/net/ethernet/intel/ixgbe/ixgbe_main.c:  .sriov_configure =
> > ixgbe_pci_sriov_configure,
> > drivers/net/ethernet/marvell/octeon_ep/octep_main.c:    .sriov_configure =
> > octep_sriov_configure,
> > drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c:   .sriov_configure =
> > otx2_sriov_configure
> > drivers/net/ethernet/netronome/nfp/nfp_main.c:  .sriov_configure        =
> > nfp_pcie_sriov_configure,
> > drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c:    .sriov_configure =
> > ionic_sriov_configure,
> > drivers/net/ethernet/qlogic/qede/qede_main.c:   .sriov_configure =
> > qede_sriov_configure,
> > drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c:       .sriov_configure =
> > qlcnic_pci_sriov_configure,
> > drivers/net/ethernet/sfc/ef10.c:        .sriov_configure =
> > efx_ef10_sriov_configure,
> > drivers/net/ethernet/sfc/ef100.c:       .sriov_configure =
> > ef100_pci_sriov_configure,
> > drivers/net/ethernet/sfc/ef100_nic.c:   .sriov_configure =
> > IS_ENABLED(CONFIG_SFC_SRIOV) ?
> > drivers/net/ethernet/sfc/efx.c: .sriov_configure = efx_pci_sriov_configure,
> > drivers/net/ethernet/sfc/siena/efx.c:   .sriov_configure =
> > efx_pci_sriov_configure,
> > drivers/net/ethernet/sfc/siena/siena.c: .sriov_configure =
> > efx_siena_sriov_configure,
> > 
> > In what way is your hardware different from all the others?
> 
> All of above devices are network devices which implements struct net_device_ops.
> i.e Those PCI devices are networking devices which are capable of sending/receiving network packets.
> This device doesn't have networking functionality to implement struct net_device_ops, It's a simple PCIe PF
> device enables it's VFs and services any mailbox requests.

What driver handles the "mailbox requests"?  What are these requests
for?

thanks,

greg k-h