Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 23 Nov 2022 08:58:53 +0200
From:   Leon Romanovsky <leon@kernel.org>
To:     Ajit Khaparde <ajit.khaparde@broadcom.com>
Cc:     andrew.gospodarek@broadcom.com, davem@davemloft.net,
        edumazet@google.com, jgg@ziepe.ca, kuba@kernel.org,
        linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
        michael.chan@broadcom.com, netdev@vger.kernel.org,
        pabeni@redhat.com, selvin.xavier@broadcom.com
Subject: Re: [PATCH v4 0/6] Add Auxiliary driver support
Message-ID: <Y33ErZHAsX76y34Z@unreal>
References: <20221109184244.7032-1-ajit.khaparde@broadcom.com>
 <Y2zYPOUKgoArq7mM@unreal>
 <CACZ4nhu_2FoOTmXPuq+amRYAipusq1XcobavytN0cFK=TSE5mQ@mail.gmail.com>
 <Y3Tj/BrskSJPuTFw@unreal>
 <CACZ4nhsv4zyzANrGh90WGKORz0Su=i7+Jmsk6nWoOq4or7Y0=Q@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CACZ4nhsv4zyzANrGh90WGKORz0Su=i7+Jmsk6nWoOq4or7Y0=Q@mail.gmail.com>
Precedence: bulk

On Tue, Nov 22, 2022 at 07:02:45AM -0800, Ajit Khaparde wrote:
> On Wed, Nov 16, 2022 at 5:22 AM Leon Romanovsky <leon@kernel.org> wrote:
> >
> ::snip::
> > > > All PCI management logic and interfaces are needed to be inside eth part
> > > > of your driver and only that part should implement SR-IOV config. Once
> > > > user enabled SR-IOV, the PCI driver should create auxiliary devices for
> > > > each VF. These device will have RDMA capabilities and it will trigger RDMA
> > > > driver to bind to them.
> > > I agree and once the PF creates the auxiliary devices for the VF, the RoCE
> > > Vf indeed get probed and created. But the twist in bnxt_en/bnxt_re
> > > design is that
> > > the RoCE driver is responsible for making adjustments to the RoCE resources.
> >
> > You can still do these adjustments by checking type of function that
> > called to RDMA .probe. PCI core exposes some functions to help distinguish between
> > PF and VFs.
> >
> > >
> > > So once the VF's are created and the bnxt_en driver enables SRIOV adjusts the
> > > NIC resources for the VF,  and such, it tries to call into the bnxt_re
> > > driver for the
> > > same purpose.
> >
> > If I read code correctly, all these resources are for one PCI function.
> >
> > Something like this:
> >
> > bnxt_re_probe()
> > {
> >   ...
> >         if (is_virtfn(p))
> >                  bnxt_re_sriov_config(p);
> >   ...
> > }
> I understand what you are suggesting.
> But what I want is a way to do this in the context of the PF
> preferably before the VFs are probed. 

I don't understand the last sentence. You call to this sriov_config in
bnxt_re driver without any protection from VFs being probed,

> So we are trying to call the
> bnxt_re_sriov_config in the context of handling the PF's
> sriov_configure implementation.  Having the ulp_ops is allowing us to
> avoid resource wastage and assumptions in the bnxt_re driver.

To which resource wastage are you referring?

There are no differences if same limits will be in bnxt_en driver when
RDMA bnxt device is created or in bnxt_re which will be called once RDMA
device is created.

Thanks

> 
> ::snip::