Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Fri, 1 Oct 2021 09:01:53 -0300
From:   Jason Gunthorpe <jgg@ziepe.ca>
To:     Haakon Bugge <haakon.bugge@oracle.com>
Cc:     Leon Romanovsky <leon@kernel.org>,
        Doug Ledford <dledford@redhat.com>,
        Bjorn Helgaas <bhelgaas@google.com>,
        OFED mailing list <linux-rdma@vger.kernel.org>,
        "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: Enabling RO on a VF
Message-ID: <20211001120153.GL3544071@ziepe.ca>
References: <48FF6F8E-95E2-4A29-A059-12EF614B381C@oracle.com>
 <20211001115455.GJ3544071@ziepe.ca>
 <4EAE3BC9-26B6-41E3-B040-2ADAB77D96CE@oracle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <4EAE3BC9-26B6-41E3-B040-2ADAB77D96CE@oracle.com>
Precedence: bulk

On Fri, Oct 01, 2021 at 11:59:15AM +0000, Haakon Bugge wrote:
> 
> 
> > On 1 Oct 2021, at 13:54, Jason Gunthorpe <jgg@ziepe.ca> wrote:
> > 
> > On Fri, Oct 01, 2021 at 11:05:15AM +0000, Haakon Bugge wrote:
> >> Hey,
> >> 
> >> 
> >> Commit 1477d44ce47d ("RDMA/mlx5: Enable Relaxed Ordering by default
> >> for kernel ULPs") uses pcie_relaxed_ordering_enabled() to check if
> >> RO can be enabled. This function checks if the Enable Relaxed
> >> Ordering bit in the Device Control register is set. However, on a
> >> VF, this bit is RsvdP (Reserved for future RW
> >> implementations. Register bits are read-only and must return zero
> >> when read. Software must preserve the value read for writes to
> >> bits.).
> >> 
> >> Hence, AFAICT, RO will not be enabled when using a VF.
> >> 
> >> How can that be fixed?
> > 
> > When qemu takes a VF and turns it into a PF in a VM it must emulate
> > the RO bit and return one
> 
> I have a pass-through VF:
> 
> # lspci -s ff:00.0 -vvv
> ff:00.0 Ethernet controller: Mellanox Technologies MT28800 Family [ConnectX-5 Ex Virtual Function]
> []
> 		DevCtl:	Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
> 			RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset-

Like I said, it is a problem in the qemu area..

Jason