Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <23fb8ad7-beb0-ae1c-fa5a-a682a57f79b0@grsecurity.net>
Date:   Wed, 14 Jun 2023 16:30:03 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.10.0
Subject: Re: Observing RCU stalls in kernel 5.4/5.10/5.15/6.1 stable trees
Content-Language: en-US, de-DE
To:     Luiz Capitulino <luizcap@amazon.com>,
        Sven-Haegar Koch <haegar@sdinet.de>
Cc:     Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
        "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        "Bhatnagar, Rishabh" <risbhat@amazon.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "sashal@kernel.org" <sashal@kernel.org>, abuehaze@amazon.com,
        "stable@vger.kernel.org" <stable@vger.kernel.org>
References: <12c6f9a3-d087-b824-0d05-0d18c9bc1bf3@amazon.com>
 <c4724b40-89f6-5aa7-720d-c4a4af57cf45@amazon.com>
 <2023061428-compacter-economic-b648@gregkh>
 <20230614092045.tNY8USjq@linutronix.de>
 <4c4178a1-1050-ced4-e6fb-f95c3bdefc98@amazon.com>
 <2a3fa097-8ba0-5b0e-f506-779fee5b8fef@sdinet.de>
 <f5d2cc62-4aae-2579-1468-2e6e389f28dc@amazon.com>
From:   Mathias Krause <minipli@grsecurity.net>
In-Reply-To: <f5d2cc62-4aae-2579-1468-2e6e389f28dc@amazon.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 14.06.23 15:57, Luiz Capitulino wrote:
> On 2023-06-14 09:45, Sven-Haegar Koch wrote:
>> May be this, talking about the same commit as cause as this thread:
>>
>> Subject: Re: [PATCH] timekeeping: Align tick_sched_timer() with the HZ
>> tick. -- regression report
>> https://lore.kernel.org/lkml/5a56290d-806e-b9a5-f37c-f21958b5a8c0@grsecurity.net/
> 
> Thank you, Sven.
> 
> Sebastian, except for the detailed analysis which we haven't done yet, the
> issue described by Mathias matches 100% what we're observing. Also, we do
> observe this on bare-metal instances which could mean that the initial
> reports are against VMs because those are rebooted more often (our quick
> reproducer boots hundreds of instances in AWS and only 1 or 2 reproduces
> this).

Yeah, we're doing VM based testing more often than bare metal -- less so
on a AWS scale. That's why we observed it first in VMs. But that wasn't
meant to exclude bare metal, not at all. It's just, that we haven't
tried hard enough yet and testing VMs is so much more pleasant when it
comes to debugging boot issues ;)

Thanks,
Mathias

> IMHO, I'd suggest we revert this for now from Linus tree and stable trees.
> We can help testing for the fix maybe for the next merge window.
> 
> - Luiz
>