Received-SPF: pass (google.com: domain of linux-kernel+bounces-172928-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <51d0dff8-2888-463c-95ab-71b491f12a8f@leemhuis.info>
Date: Wed, 8 May 2024 10:38:26 +0200
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Reply-To: Linux regressions mailing list <regressions@lists.linux.dev>
Subject: Re: Early boot regression from f0551af0213 ("x86/topology: Ignore
 non-present APIC IDs in a present package")
To: Thomas Gleixner <tglx@linutronix.de>, Lyude Paul <lyude@redhat.com>
Cc: x86@kernel.org, linux-kernel@vger.kernel.org,
 Mario Limonciello <mario.limonciello@amd.com>, Borislav Petkov
 <bp@alien8.de>, Linux kernel regressions list <regressions@lists.linux.dev>
References: <3d77cb89857ee43a9c31249f4eab7196013bc4b4.camel@redhat.com>
 <20240418082703.GCZiDZVyra7qOQbyqn@fat_crate.local>
 <fd040809d95b3e12b2fdc78a2409e187716bc66f.camel@redhat.com>
 <87plumxz4x.ffs@tglx>
 <abbb7d7ca781f6c664e4c5b1dffc19394ac79691.camel@redhat.com>
 <87le59vw1y.ffs@tglx>
 <3a0afe545747e5314a9cb6bbaa9ce90b259ddfac.camel@redhat.com>
 <87edautcmz.ffs@tglx>
 <3b1d16e357c1f9badeef405366492f05af26c085.camel@redhat.com>
 <878r11t8zu.ffs@tglx> <016902d9-3858-4c65-b3ec-f7a5103af63c@amd.com>
From: "Linux regression tracking (Thorsten Leemhuis)"
 <regressions@leemhuis.info>
Content-Language: en-US, de-DE
In-Reply-To: <016902d9-3858-4c65-b3ec-f7a5103af63c@amd.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

H! Lyude, Thomas, what's the status here? From here it looks like we
were close to a fix, but then it turned out to be a bad fix -- and
afterwards nothing much seems to have happened. Did it fall through the
cracks, or was this already fixed and I just missed that?

Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat)

On 02.05.24 12:33, Mario Limonciello wrote:
> On 4/25/2024 16:42, Thomas Gleixner wrote:
>> On Thu, Apr 25 2024 at 11:56, Lyude Paul wrote:
>>> On Thu, 2024-04-25 at 04:11 +0200, Thomas Gleixner wrote:
>>>>
>>>> Can you please boot a kernel with the commit in question reverted and
>>>> add 'possible_cpus=8' to the kernel command line?
>>>>
>>>> In theory this should fail too.
>>>
>>> Yep - tried booting a kernel with f0551af0213 reverted and
>>> possible_cpus=8, it definitely looks like that crashes things as well
>>> in the same way.
>>
>> Good. That means it's a problem which existed before but went unnoticed.
>>
>>> Also - it scrolled off the screen before I had a chance to write it
>>> down, but I'm -fairly- sure I saw some sort of complaint about "16 [or
>>> some double digit number] processors exceeds max number of 8". Which
>>> is quite interesting, as this is definitely just a quad core ryzen
>>> processor with hyperthreading - so there should only be 8 threads.
>>
>> Right, that's what we saw with the debug patch. The ACPI/MADT table
>> is clearly bonkers. The effect of it is that it pretends that the system
>> has 16 possible CPUs:
>>
>>      [    0.089381] CPU topo: Allowing 8 present CPUs plus 8 hotplug CPUs
>>
>> Which in turn changes the sizing of the per CPU data and affects some
>> other details which depend on the number of possible CPUs.
> 
> At least this aspect of this I suspect is caused by commit
> fed8d8773b8ea68ad99d9eee8c8343bef9da2c2c.
> 
> If you try reverting that I expect the "hotplug CPUs" disappear.
> 
>>
>> But that should not matter at all because the system scaling should be
>> sufficient with 8 CPUs, but it does not for some completely non-obvious
>> reasons.
>>
>> Can you please try to increase possible_cpus=N on the command line one
>> by one and check when it actually starts to "work" again.
>>
>> One other thing to try is to boot with 'possible_cpus=8' and
>> 'intremap=off' and see whether that makes a difference.
>>
>> I really have no idea where to look and not having the early boot
>> messages in case of the fail is not helpful as I can't add meaningful
>> debug to it.
>>
>> I just checked: the motherboard has a serial port, so it would be
>> extremly helpful to hook up a serial cable to this thing and enable
>> serial console on the kernel command line. That way we might eventually
>> see information which is emitted before it fails to validate the timer
>> interrupt.
>>

--
Everything you wanna know about Linux kernel regression tracking:
https://linux-regtracking.leemhuis.info/about/#tldr
If I did something stupid, please tell me, as explained on that page.

#regzbot poke