Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <e4f7b16e-5b6f-1b2c-5f88-fc4a129ae28f@redhat.com>
Date:   Sat, 15 Oct 2022 16:25:00 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.3.1
Subject: Re: alderlake crashes (random memory corruption?) with 6.0 i915 /
 ucode related
Content-Language: en-US, nl
From:   Hans de Goede <hdegoede@redhat.com>
To:     intel-gfx <intel-gfx@lists.freedesktop.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        "Thorsten Leemhuis (regressions address)" <regressions@leemhuis.info>
References: <355dde1c-91e3-13b5-c8e8-75c9b9779b4f@redhat.com>
In-Reply-To: <355dde1c-91e3-13b5-c8e8-75c9b9779b4f@redhat.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi,

On 10/13/22 22:33, Hans de Goede wrote:
> Hi All,
> 
> Yesterday I got a new Lenovo ThinkPad X1 yoga gen 7 laptop, since I plan
> to make this my new day to day laptop I have copied over the entire
> rootfs, /home, etc. from my current laptop to avoid having to tweak
> everything to my liking again.
> 
> This meant I had an initramfs generated for the other laptop. Which should
> be fine since both are Intel machines and the old 5.19.y initramfs-es
> worked fine. But 6.0.0 crashed with what seems like random memory
> corruption (list integrity checks failing) until I regenerated the initrd ...
> 
> Comparing the old vs regenerated initrds showed no relevant differences,
> which made me think this is a CPU ucode issue (which is pre-fixed
> to the initrd for early microcode loading).
> 
> After some tests I have the following obeservations with 6.0.0:
> 
> 1. The least stable is the old initrd (so with the wrong
> ucode prefixed) this crashes before ever reaching gdm.
> I believe that this is caused by late microcode loading
> kicking in in this case (I though that was being removed?)
> and doing load microcode loading on the i7-1260P with its
> mix of P + E cores seems to seriously mess things up.
> 
> 2. Slightly more stable, lasting at least a few minutes
> before crashing is using dis_ucode_ldr
> 
> 3. Using nomodeset seems to stabilize things even with
> the old initrd with the wrong microcode prefixed
> 
> 4. 5.19, with an old initrd and with normal modesetting
> enabled works fine, so in a way this is a 6.0.0 regression
> 
> 5. Using 6.0 with the new initrd with the new microcode
> seems mostly stable, although sometimes this seems to 
> hang very early during boot, esp. if a previous boot
> crashed and I have not run this for a long time yet.
> 
> 6. After crashes it seems to be necessary to powercycle
> the machine to get things back in working condition.
> 
> 
> With 6.0 the following WARN triggers:
> drivers/gpu/drm/i915/display/intel_bios.c:477:
> 
>         drm_WARN(&i915->drm, min_size == 0,
>                  "Block %d min_size is zero\n", section_id);
> 
> Since nomodeset helps this might be quite relevant, in 5.19.13
> this does not happen, but I'm not sure if 5.19 has this check
> at all.
> 
> 
> There is a 2022/10/07 BIOS update which includes a CPU microcode
> update available from Lenovo, I have not applied this yet in case
> people want to investigate this further first.

A quick update on this, the microcode being in the initrd or not
seems to be a bit of a red herring. Yesterday the machine crashed
twice at boot with 6.0.0 with an initrd which did correctly have
the alderlake microcode cpio archive prefixed.

Where as with 5.19 it boots correctly everytime. I will try to
make some time to git bisect this sometime next week. I expect
this is an i915 issue though since 6.0.0 with nomodeset on
the cmdline does seem to boot successfully every time.

Regards,

Hans