Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 13 Jun 2022 14:07:51 -0600
From:   Rob Herring <robh@kernel.org>
To:     Serge Semin <Sergey.Semin@baikalelectronics.ru>
Cc:     Bjorn Helgaas <bhelgaas@google.com>,
        Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
        Jingoo Han <jingoohan1@gmail.com>,
        Gustavo Pimentel <gustavo.pimentel@synopsys.com>,
        Krzysztof =?utf-8?Q?Wilczy=C5=84ski?= <kw@linux.com>,
        Pankaj Dubey <pankaj.dubey@samsung.com>,
        Shradha Todi <shradha.t@samsung.com>,
        Serge Semin <fancer.lancer@gmail.com>,
        Alexey Malahov <Alexey.Malahov@baikalelectronics.ru>,
        Pavel Parkhomenko <Pavel.Parkhomenko@baikalelectronics.ru>,
        Frank Li <Frank.Li@nxp.com>,
        Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>,
        linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v4 04/18] PCI: dwc: Set INCREASE_REGION_SIZE flag based
 on limit address
Message-ID: <20220613200751.GA4188875-robh@kernel.org>
References: <20220610082535.12802-1-Sergey.Semin@baikalelectronics.ru>
 <20220610082535.12802-5-Sergey.Semin@baikalelectronics.ru>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220610082535.12802-5-Sergey.Semin@baikalelectronics.ru>
Precedence: bulk

On Fri, Jun 10, 2022 at 11:25:20AM +0300, Serge Semin wrote:
> It was wrong to use the region size parameter in order to determine
> whether the INCREASE_REGION_SIZE flag needs to be set for the outbound
> iATU entry because in general there are cases when combining a region base
> address and size together produces the out of bounds upper range limit
> while upper_32_bits(size) still returns zero. So having a region size
> within the permitted values doesn't mean the region limit address will fit
> to the corresponding CSR. Here is the way iATU calculates the in- and
> outbound untranslated regions if the INCREASE_REGION_SIZE flag is cleared
> [1]:
> 
>   Start address:                      End address:
> 63              31              0   63              31              0
> +---------------+---------------+   +---------------+---------------+
> |               |         |  0s |   |               |         |  Fs |
> +---------------+---------------+   +---------------+---------------+
>    upper base   |   lower base       !upper! base   | limit address
>      address          address           address
> 
> So the region start address is determined by the iATU lower and upper base
> address registers, while the region upper boundary is calculated based on
> the 32-bits limit address register and the upper part of the base address.
> In accordance with that logic for instance the range
> 0xf0000000 @ 0x20000000 does have the size smaller than 4GB, but the
> actual limit address turns to be invalid forming the untranslated address
> map as [0xf0000000; 0x1000FFFF], which isn't what the original range was.
> In order to fix that we need to check whether the size after being added
> to the lower part of the base address causes the 4GB range overflow. If it
> does then we need to set the INCREASE_REGION_SIZE flag thus activating the
> extended limit address by means of an additional iATU CSR (upper limit
> address register) [2]:
> 
>   Start address:                      End address:
> 63              31              0   63      x       31              0
> +---------------+---------------+   +---------------+---------------+
> |               |         |  0s |   |       |       |         |  Fs |
> +---------------+---------------+   +---------------+---------------+
>    upper base   |  lower base         upper | upper | limit address
>      address         address          base  | limit |
>                                      address|address|
> 
> Otherwise there is enough room in the 32-bits wide limit address register,
> and the flag can be left unset.
> 
> Note the case when the size-based flag setting approach is correct implies
> requiring to have the size-aligned base addresses only. But that
> constraint isn't relevant to the PCIe ranges accepted by the kernel.
> There is also no point in implementing it either seeing the problem can be
> easily fixed by checking the whole limit address instead of the region
> size.
> 
> [1] DesignWare Cores PCI Express Controller Databook - DWC PCIe Root Port,
>     v5.40a, March 2019, fig.3-36, p.175
> [2] DesignWare Cores PCI Express Controller Databook - DWC PCIe Root Port,
>     v5.40a, March 2019, fig.3-37, p.176
> 
> Fixes: 5b4cf0f65324 ("PCI: dwc: Add upper limit address for outbound iATU")
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
> Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> Tested-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
> 
> ---
> 
> Changelog v2:
> - Fix the end address in the example of the patch log. It should be
>   0x1000FFFF and not 0x0000FFFF (@Manivannan).
> ---
>  drivers/pci/controller/dwc/pcie-designware.c | 16 ++++++++++------
>  1 file changed, 10 insertions(+), 6 deletions(-)

Reviewed-by: Rob Herring <robh@kernel.org>