Received-SPF: pass (google.com: domain of linux-kernel+bounces-99272-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <8c8aea4d-e911-44cd-bbec-ead4e44d338a@samsung.com>
Date: Mon, 11 Mar 2024 17:51:04 +0100
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3] ARM64: Dynamically allocate cpumasks and increase
 supported CPUs to 512
Content-Language: en-US
To: Catalin Marinas <catalin.marinas@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>, "Christoph Lameter (Ampere)"
	<cl@linux.com>, "linux-pm@vger.kernel.org" <linux-pm@vger.kernel.org>,
	"Rafael J. Wysocki" <rafael@kernel.org>, Viresh Kumar <vireshk@kernel.org>,
	Will Deacon <will@kernel.org>, Jonathan.Cameron@huawei.com,
	Matteo.Carlini@arm.com, Valentin.Schneider@arm.com,
	akpm@linux-foundation.org, anshuman.khandual@arm.com, Eric Mackay
	<eric.mackay@oracle.com>, dave.kleikamp@oracle.com,
	linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
	linux-mm@kvack.org, linux@armlinux.org.uk, robin.murphy@arm.com,
	vanshikonda@os.amperecomputing.com, yang@os.amperecomputing.com, Nishanth
	Menon <nm@ti.com>, Stephen Boyd <sboyd@kernel.org>
From: Marek Szyprowski <m.szyprowski@samsung.com>
In-Reply-To: <Ze8hmCbN7_GDRMVS@arm.com>
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="utf-8"
CMS-TYPE: 201P
References: <37099a57-b655-3b3a-56d0-5f7fbd49d7db@gentwo.org>
	<CGME20240308140130eucas1p1259c805a0b6491ce2f69c6fca0264b1f@eucas1p1.samsung.com>
	<c1f2902d-cefc-4122-9b86-d1d32911f590@samsung.com>
	<ZesmAO1jJfEjTwxd@arm.com>
	<f160a532-1d31-41a7-b8ae-de8575c395e9@samsung.com>
	<e3952dc0-ec28-e7c7-e858-c47f146c93de@linux.com>
	<Ze71H4AdY786nSvn@FVFF77S0Q05N>
	<8abb1a69-6cbd-4a36-ab1d-d269cdafa391@samsung.com>
	<Ze8hmCbN7_GDRMVS@arm.com>

Hi Catalin,

On 11.03.2024 16:22, Catalin Marinas wrote:
> On Mon, Mar 11, 2024 at 03:56:37PM +0100, Marek Szyprowski wrote:
>> On 11.03.2024 13:12, Mark Rutland wrote:
>>> On Fri, Mar 08, 2024 at 09:08:59AM -0800, Christoph Lameter (Ampere) wrote:
>>>> On Fri, 8 Mar 2024, Marek Szyprowski wrote:
>>>>>>> It looks that cpufreq-dt and/or opp drivers needs some adjustments
>>>>>>> related with this change.
>>>>>> That's strange. Is this with defconfig? I wonder whether NR_CPUS being
>>>>>> larger caused the issue with this specific code. Otherwise
>>>>>> CPUMASK_OFFSTACK may not work that well on arm64.
>>>> cpumask handling must use the accessor functions provided in
>>>> include/linux/cpumask.h for declaring and accessing cpumasks. It is likely
>>>> related to the driver opencoding one of the accessors.
>>> I took a look at both the OPP code and the cpufreq-dt code and it looks like
>>> those are doign the right thing w.r.t. cpumask manipulation (i.e. they only use
>>> the cpumask accessors, and use the cpumask_var_*() functions to dynamically
>>> allocate/free cpumasks). Maybe I've missed something, but superficially those
>>> look right.
>>>
>>> Marek, can you try reverting this commit and trying defconfig + NR_CPUS=512?
>> Yes, with $subject reverted and CONFIG_NR_CPUS=512 everything works
>> fine, so it must be something else broken.
> Thanks for confirming. Would you mind testing the problematic commit
> with CONFIG_DEBUG_PER_CPU_MAPS enabled? If it doesn't show anything
> obvious that can be fixed quickly, I'll revert the commit and queue it
> again after -rc1 for 6.10 (I haven't sent 6.9 the pull request yet).

I've enabled this option, but unfortunately it didn't reveal anything 
more besides the warning and error I've posted in my initial report. I 
will try to analyze this issue further, but I won't manage to do this today.

Best regards
-- 
Marek Szyprowski, PhD
Samsung R&D Institute Poland