Received-SPF: pass (google.com: domain of linux-kernel+bounces-15882-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
References: <20240103170002.1793197-1-enachman@marvell.com>
User-agent: mu4e 1.10.8; emacs 29.1
From: Baruch Siach <baruch@tkos.co.il>
To: Elad Nachman <enachman@marvell.com>
Cc: catalin.marinas@arm.com, will@kernel.org, thunder.leizhen@huawei.com,
 bhe@redhat.com, akpm@linux-foundation.org, yajun.deng@linux.dev,
 chris.zjh@huawei.com, linux-kernel@vger.kernel.org,
 linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH] arm64: mm: Fix SOCs with DDR starting above zero
Date: Wed, 03 Jan 2024 20:47:17 +0200
In-reply-to: <20240103170002.1793197-1-enachman@marvell.com>
Message-ID: <87r0iy45ho.fsf@tarshish>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain

Hi Elad,

On Wed, Jan 03 2024, Elad Nachman wrote:
> From: Elad Nachman <enachman@marvell.com>
>
> Some SOCs, like the Marvell AC5/X/IM, have a combination
> of DDR starting at 0x2_0000_0000 coupled with DMA controllers
> limited to 31 and 32 bit of addressing.
> This requires to properly arrange ZONE_DMA and ZONE_DMA32 for
> these SOCs, so swiotlb and coherent DMA allocation would work
> properly.
> Change initialization so device tree dma zone bits are taken as
> function of offset from DRAM start, and when calculating the
> maximal zone physical RAM address for physical DDR starting above
> 32-bit, combine the physical address start plus the zone mask
> passed as parameter.
> This creates the proper zone splitting for these SOCs:
> 0..2GB for ZONE_DMA
> 2GB..4GB for ZONE_DMA32
> 4GB..8GB for ZONE_NORMAL
>
> Signed-off-by: Elad Nachman <enachman@marvell.com>

For an alternative approach see

  https://lore.kernel.org/all/cover.1703683642.git.baruch@tkos.co.il/

baruch

> ---
>  arch/arm64/mm/init.c | 20 +++++++++++++++-----
>  1 file changed, 15 insertions(+), 5 deletions(-)
>
> diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
> index 74c1db8ce271..8288c778916e 100644
> --- a/arch/arm64/mm/init.c
> +++ b/arch/arm64/mm/init.c
> @@ -115,20 +115,21 @@ static void __init arch_reserve_crashkernel(void)
>  
>  /*
>   * Return the maximum physical address for a zone accessible by the given bits
> - * limit. If DRAM starts above 32-bit, expand the zone to the maximum
> - * available memory, otherwise cap it at 32-bit.
> + * limit. If DRAM starts above 32-bit, expand the zone to the available memory
> + * start limited by the zone bits mask, otherwise cap it at 32-bit.
>   */
>  static phys_addr_t __init max_zone_phys(unsigned int zone_bits)
>  {
>  	phys_addr_t zone_mask = DMA_BIT_MASK(zone_bits);
>  	phys_addr_t phys_start = memblock_start_of_DRAM();
> +	phys_addr_t phys_end = memblock_end_of_DRAM();
>  
>  	if (phys_start > U32_MAX)
> -		zone_mask = PHYS_ADDR_MAX;
> +		zone_mask = phys_start | zone_mask;
>  	else if (phys_start > zone_mask)
>  		zone_mask = U32_MAX;
>  
> -	return min(zone_mask, memblock_end_of_DRAM() - 1) + 1;
> +	return min(zone_mask, phys_end - 1) + 1;
>  }
>  
>  static void __init zone_sizes_init(void)
> @@ -140,7 +141,16 @@ static void __init zone_sizes_init(void)
>  
>  #ifdef CONFIG_ZONE_DMA
>  	acpi_zone_dma_bits = fls64(acpi_iort_dma_get_max_cpu_address());
> -	dt_zone_dma_bits = fls64(of_dma_get_max_cpu_address(NULL));
> +	/*
> +	 * When calculating the dma zone bits from the device tree, subtract
> +	 * the DRAM start address, in case it does not start from address
> +	 * zero. This way. we pass only the zone size related bits to
> +	 * max_zone_phys(), which will add them to the base of the DRAM.
> +	 * This prevents miscalculations on arm64 SOCs which combines
> +	 * DDR starting above 4GB with memory controllers limited to
> +	 * 32-bits or less:
> +	 */
> +	dt_zone_dma_bits = fls64(of_dma_get_max_cpu_address(NULL) - memblock_start_of_DRAM());
>  	zone_dma_bits = min3(32U, dt_zone_dma_bits, acpi_zone_dma_bits);
>  	arm64_dma_phys_limit = max_zone_phys(zone_dma_bits);
>  	max_zone_pfns[ZONE_DMA] = PFN_DOWN(arm64_dma_phys_limit);


-- 
                                                     ~. .~   Tk Open Systems
=}------------------------------------------------ooO--U--Ooo------------{=
   - baruch@tkos.co.il - tel: +972.52.368.4656, http://www.tkos.co.il -