Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <1535453838-12154-1-git-send-email-sunil.kovvuri@gmail.com>
 <1535453838-12154-12-git-send-email-sunil.kovvuri@gmail.com>
 <CAK8P3a0q0P-FuG6cTgMNkeZbtXE4pMk1iJq36_x20WRddtwaDg@mail.gmail.com>
 <CA+sq2Cc1CSKmuEjKt8psR7Zyv9PzLkN10hzy8mrai136SF4-qA@mail.gmail.com>
 <CAK8P3a2CMcYg6vMmRAc4N80BpFwczGiPnaLNCDZYXCsRydtPQw@mail.gmail.com>
 <CA+sq2Ce3ew0X+RMm7fQ+iUZF0HKptzBr1dBk3NuRL8o82wBTjg@mail.gmail.com> <CAK8P3a1oeOFVYc2xH+KO1gV0m9utHziGpNtQLAEv-RBxKkB7Og@mail.gmail.com>
In-Reply-To: <CAK8P3a1oeOFVYc2xH+KO1gV0m9utHziGpNtQLAEv-RBxKkB7Og@mail.gmail.com>
From:   Sunil Kovvuri <sunil.kovvuri@gmail.com>
Date:   Thu, 30 Aug 2018 23:25:33 +0530
Message-ID: <CA+sq2CfAvasZoNmnNQadhdsEz-eyXQPBtXbDmvt1nM0DQsm+8g@mail.gmail.com>
Subject: Re: [PATCH 11/15] soc: octeontx2: Add Marvell OcteonTX2 CGX driver
To:     Arnd Bergmann <arnd@arndb.de>
Cc:     LKML <linux-kernel@vger.kernel.org>, olof@lixom.net,
        LAKML <linux-arm-kernel@lists.infradead.org>,
        linux-soc@vger.kernel.org, Sunil Goutham <sgoutham@marvell.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, Aug 30, 2018 at 7:37 PM Arnd Bergmann <arnd@arndb.de> wrote:
>
> On Tue, Aug 28, 2018 at 3:10 PM Sunil Kovvuri <sunil.kovvuri@gmail.com> wrote:
> >
> > > > > If this is a regular PCI ethernet driver, why do you put it into driver/soc
> > > > > rather than drivers/net/ethernet/ ?
> > > >
> > > > No, this is not a ethernet driver, as mentioned in the cover letter
> > > > this driver and AF driver doesn't
> > > > handle any IO. There will be a separate ethernet driver (will submit
> > > > that as well in future) which will
> > > > communicate with these drivers for configuring hardware.
> > > >
> > > > The driver in question here is for a serdes controller which handles
> > > > physical ethernet interfaces.
> > > > Admin function driver gathers info w.r.t current state of physical
> > > > ethernet interfaces from this driver
> > > > and notifies actual ethernet driver about changes, if any.
> > >
> > > Ok. Can you describe the structure that the PCI devices appear
> > > in? It might help to be make the connection between the differnet
> > > patches to understand how things fit together. In the final
> > > picture, how many different pci_driver instances do you have,
> > > and what part are they for?
> >
> > List of PCI devices are CGX, RVU PF0-PFn SRIOV physical functions
> > and RVU VF0-VFn SRIOV virtual functions. No of VFs per PF is configurable
> > and done by low level firmware.
> >
> > List of PCI driver instances would be CGX driver, RVU PF0 (i.e admin
> > function) driver,
> > PF1-PFn either netdev driver or crypto driver, VF0-VFn functionality would be
> > same as their PF.
> >
> > The current plan is to have CGX driver, Admin function driver, PF
> > netdev driver, VF netdev driver and PF/VF crypto drivers.
>
> Ok, I think I understand the PF/VF distinction now. One (to me)
> surprising aspect here is that you not just have one physical function
> that you can use to assign resources to multiple virtual functions,
> but also a second level of virtualization that is used to assign
> resources to "physical functions" that are less physical than the
> name suggests.

Yes, PF is just for name sake, on-boot there is no difference between
PFs/VFs as such.
PF0 has privilege access to assign resources to all PFs and their VFs.
This admin function driver loads for PF0.

>
> The part that I have not grasped yet is what the split between
> the CGX and the AF is for, how they relate to one another, and
> what the software abstraction for the two is going to be.

In HW, CGX is a separate PCI device which handles the serdes and
physical ethernet interface.
Ethernet driver in drivers/net/ethernet can only communicate to
admin function driver since they share a mailbox memory.
So we had to bind both CGX and admin function drivers to almost work as one,
inorder to provide relavent info to ethernet drivers. That's why we
have many functions
from CGX driver which AF uses.

eg: Firmware gets to know about a physical interface status change,
which CGX driver gets
to know and it uses AF's mailbox communication to inform ethernet
driver about the event.

>
> > > Is the idea that an ethernet device driver always attaches to a
> > > virtual function that gets created by the main driver, and that
> > > the two drivers share no interfaces on the kernel side, or do
> > > you have multiple drivers linking to each other?
> >
> > Ethernet device driver can attach to both physical function and virtual function
> > whose HW resources are provisioned by admin function driver.
> >
> > Yes the PF/VF ethernet drivers and these drivers won't share any
> > kernel interfaces.
> > Physical ethernet interface is owned by ethernet driver only, this driver just
> > configures which ethernet driver instance uses which physcial interface.
>
> Ok.
>
>          Arnd