Received-SPF: pass (google.com: domain of linux-kernel+bounces-86491-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Thu, 29 Feb 2024 10:56:43 +0100
From: Niklas Cassel <cassel@kernel.org>
To: Shradha Todi <shradha.t@samsung.com>
Cc: lchen.firstlove@zohomail.com, lpieralisi@kernel.org, kw@linux.com,
	mani@kernel.org, kishon@kernel.org, bhelgaas@google.com,
	linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org,
	pankaj.dubey@samsung.com
Subject: Re: [PATCH v4] PCI: endpoint: Add prefetch BAR support
Message-ID: <ZeBU23Ccvv8WqFx_@fedora>
References: <17e7ad65ff5.d9de88b4962.1109678039880040918@zohomail.com>
 <CGME20240228134451epcas5p1b974d61fcab67fb5f52a7b291cf85966@epcas5p1.samsung.com>
 <20240228134448.56372-1-shradha.t@samsung.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240228134448.56372-1-shradha.t@samsung.com>

On Wed, Feb 28, 2024 at 07:14:48PM +0530, Shradha Todi wrote:
> Reviewed-by: Shradha Todi <shradha.t@samsung.com>
> 
> This patch looks useful. Can we revisit this and get it merged?

Hello Shradha,

This patch is two years old, and no longer applies to pci-next.


However:
Usually, fixed hardware requirements are specified in
struct pci_epc_features (more specifically struct pci_epc_bar_desc).

A requested BAR configuration by an EPF is specified in struct epf_bar.


I don't think that Prefetch is a fixed hardware requirement,
so I do not think that we should put it in struct pci_epc_features.

It seems more like something that an endpoint function driver can
chose to request (or not to request), just like MEM_TYPE_64.

From the PCIe base spec:
"Generally only 64-bit BARs are good candidates, since only Legacy
Endpoints are permitted to set the Prefetchable bit in 32-bit BARs,
and most scalable platforms map all 32-bit Memory BARs into
non-prefetchable Memory Space regardless of the Prefetchable bit value."

"For a PCI Express Endpoint, 64-bit addressing must be supported for all BARs
that have the Prefetchable bit Set. 32-bit addressing is permitted for all BARs
that do not have the Prefetchable bit Set."

"Any device that has a range that behaves like normal memory should mark the
range as prefetchable. A linear frame buffer in a graphics device is an example
of a range that should be marked prefetchable."

We are not a legacy endpoint, so we should never set Prefetch for 32-bit BARs.
For 64-bit BARs, we should always set it, if the EPF-core allocated the memory
(regular memory) for that BAR.


Thus, I think the best solution is to do:

diff --git a/drivers/pci/endpoint/functions/pci-epf-test.c b/drivers/pci/endpoint/functions/pci-epf-test.c
index cd4ffb39dcdc..186c8cd87bb3 100644
--- a/drivers/pci/endpoint/functions/pci-epf-test.c
+++ b/drivers/pci/endpoint/functions/pci-epf-test.c
@@ -879,7 +879,8 @@ static void pci_epf_configure_bar(struct pci_epf *epf,
        for (i = 0; i < PCI_STD_NUM_BARS; i++) {
                epf_bar = &epf->bar[i];
                if (epc_features->bar[i].only_64bit)
-                       epf_bar->flags |= PCI_BASE_ADDRESS_MEM_TYPE_64;
+                       epf_bar->flags |= (PCI_BASE_ADDRESS_MEM_TYPE_64 |
+                                          PCI_BASE_ADDRESS_MEM_PREFETCH);
        }
 }
 
diff --git a/drivers/pci/endpoint/pci-epf-core.c b/drivers/pci/endpoint/pci-epf-core.c
index 0a28a0b0911b..acb93055181b 100644
--- a/drivers/pci/endpoint/pci-epf-core.c
+++ b/drivers/pci/endpoint/pci-epf-core.c
@@ -305,7 +305,8 @@ void *pci_epf_alloc_space(struct pci_epf *epf, size_t size, enum pci_barno bar,
        epf_bar[bar].size = size;
        epf_bar[bar].barno = bar;
        epf_bar[bar].flags |= upper_32_bits(size) ?
-                               PCI_BASE_ADDRESS_MEM_TYPE_64 :
+                               (PCI_BASE_ADDRESS_MEM_TYPE_64 |
+                                PCI_BASE_ADDRESS_MEM_PREFETCH) :
                                PCI_BASE_ADDRESS_MEM_TYPE_32;
 
        return space;


Now when I look at it, the whole "if (epc_features->bar[i].only_64bit)"
should move to pci_epf_alloc_space() IMO, so that not all EPF drivers need to
duplicate this code.


Kind regards,
Niklas