Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Alan Mikhak <alan.mikhak@sifive.com>
To:     linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org,
        jingoohan1@gmail.com, gustavo.pimentel@synopsys.com,
        lorenzo.pieralisi@arm.com, amurray@thegoodpenguin.co.uk,
        bhelgaas@google.com, kishon@ti.com, paul.walmsley@sifive.com
Cc:     Alan Mikhak <alan.mikhak@sifive.com>
Subject: [PATCH v2] PCI: dwc: Warn MEM resource size exceeds max for 32-bits
Date:   Tue, 31 Mar 2020 15:48:35 -0700
Message-Id: <1585694915-32005-1-git-send-email-alan.mikhak@sifive.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

From: Alan Mikhak <alan.mikhak@sifive.com>

Output a warning for MEM resource size with non-zero upper 32-bits.

ATU programming functions limit the size of the translated region to 4GB
by using a u32 size parameter. Function dw_pcie_prog_outbound_atu() does
not program the upper 32-bit ATU limit register. These issues may result
in undefined behavior for resource sizes with non-zero upper 32-bits.

For example, a 128GB address space starting at physical CPU address of
0x2000000000 with size of 0x2000000000 needs the following values
programmed into the lower and upper 32-bit limit registers:
 0x3fffffff in the upper 32-bit limit register
 0xffffffff in the lower 32-bit limit register

Currently, only the lower 32-bit limit register is programmed with a
value of 0xffffffff but the upper 32-bit limit register is not being
programmed. As a result, the upper 32-bit limit register remains at its
default value after reset of 0x0.

This would be a problem for a 128GB PCIe space because the internal
calculation of the limit address in ATU programming functions in effect
reduces the size to 4GB. This may also produce undefined behavior since
the ATU limit address may be lower than the ATU base address.

This limitation also means that multiple ATUs would need to be used to
map larger regions.

ATU programming functions may be changed to specify a u64 size parameter
for the translated region. Along with this change, the internal
calculation of the limit address, the address of the last byte in the
translated region, needs to change such that both the lower 32-bit and
upper 32-bit limit registers can be programmed correctly.

Without change, this issue can go unnoticed. A warning may prompt the
user to look for possible issues.

Changing the ATU programming functions impacts all PCIe drivers that
depend on dwc which seem to be currently happy with the u32 size limit.
A solution that allows existing PCIe drivers to phase into the fix on
their own individual schedules, if they need to, may be desirable.

Signed-off-by: Alan Mikhak <alan.mikhak@sifive.com>
Acked-by: Gustavo Pimentel <gustavo.pimentel@synopsys.com>
---
 drivers/pci/controller/dwc/pcie-designware-host.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
index 395feb8ca051..df5ebe02ffca 100644
--- a/drivers/pci/controller/dwc/pcie-designware-host.c
+++ b/drivers/pci/controller/dwc/pcie-designware-host.c
@@ -325,6 +325,7 @@ int dw_pcie_host_init(struct pcie_port *pp)
 	struct pci_bus *child;
 	struct pci_host_bridge *bridge;
 	struct resource *cfg_res;
+	resource_size_t mem_size;
 	u32 hdr_type;
 	int ret;
 
@@ -362,7 +363,10 @@ int dw_pcie_host_init(struct pcie_port *pp)
 		case IORESOURCE_MEM:
 			pp->mem = win->res;
 			pp->mem->name = "MEM";
-			pp->mem_size = resource_size(pp->mem);
+			mem_size = resource_size(pp->mem);
+			if (upper_32_bits(mem_size))
+				dev_warn(dev, "MEM resource size exceeds max for 32 bits\n");
+			pp->mem_size = mem_size;
 			pp->mem_bus_addr = pp->mem->start - win->offset;
 			break;
 		case 0:
-- 
2.7.4