Received-SPF: pass (google.com: domain of linux-kernel+bounces-186556-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <0111018fa15e16c0-838d5fbe-fa3c-468c-97bc-e3889a50f893-000000@us-west-1.amazonses.com>
Date: Wed, 22 May 2024 17:34:09 +0000
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Arazil Songweaver <arazil@arazil.net>
Subject: Re: BUG: Framework Laptop 16 i2c-hid Based Touchpad Sometimes Fails
 To Initialize Properly On Early Boot
To: Linux regressions mailing list <regressions@lists.linux.dev>
Cc: linux-input@vger.kernel.org, Jiri Kosina <jikos@kernel.org>, 
	Benjamin Tissoires <bentiss@kernel.org>, 
	LKML <linux-kernel@vger.kernel.org>
References: <0111018f7508efb1-1206661a-d4b5-4bd1-b54f-ccb8504b8b4a-000000@us-west-1.amazonses.com>
 <f0dd90aa-78b0-4253-96b0-3241e8b2a11b@leemhuis.info>
 <0111018f9b597531-d4643a7e-c27a-46f9-9e25-90530d648d65-000000@us-west-1.amazonses.com>
 <75053193-92db-40ad-9658-3cdeb0f553be@leemhuis.info>
Content-Language: en-US
In-Reply-To: <75053193-92db-40ad-9658-3cdeb0f553be@leemhuis.info>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Feedback-ID: ::1.us-west-1.+ToUlskFmoAr0tQOMQW4CAabIYpBdjD9BBfO15fkT3E=:AmazonSES

I was able to reproduce this bug on kernel 6.6.0 which should rule out a 
recent regression. Kernel 6.5.0 refused to start with a modules error. 
Going earlier than that is probably pointless given that we are talking 
about hardware first released in January 2024.

This issue is also on the kernel.org Bugzilla at: 218836

-Arazil

On 5/21/24 8:44 AM, Linux regression tracking (Thorsten Leemhuis) wrote:
> On 21.05.24 15:31, Arazil Songweaver wrote:
>> I was able to reproduce bug again with: Linux version
>> 6.9.0-1-git-10323-g8f6a15f095a6 (linux-git@archlinux) (gcc (GCC) 14.1.1
>> 20240507, GNU ld (GNU Binutils) 2.42.0) #1 SMP PREEMPT_DYNAMIC Tue, 21
>> May 2024 11:58:24 +0000 (not an official Arch Linux package)
>>
>> The relevant I2C_HID module needs to be built in to the kernel for the
>> bug to trigger on a consistent basis. Arch Linux and mkinitcpio is
>> currently getting around this issue by building I2C_HID as a module and
>> delaying the load of that module until the part when autoprobe starts
>> loading relevant kernel modules. The bug became (more) visible on Arch
>> Linux after an update to mkinitcpio moved the I2C_HID module up to the
>> beginning of the boot process.
>>
>> This is not a recent regression. In my testing, I was able to reproduce
>> this issue as far back as version 6.8. I did not test 6.7 or earlier
>> revisions yet.
> Okay, then I won't track this as a regression; might still be worth
> trying a few older kernels in a spare minute to see if it was introduced
> in the last 12 or 18 months and can be bisected.
>
> CCed Jiri and Benjamin nevertheless in case they missed this report on
> the lists.
>
> Ciao, Thorsten
>
>> On 5/21/24 5:57 AM, Linux regression tracking (Thorsten Leemhuis) wrote:
>>> On 14.05.24 04:57, Arazil Songweaver wrote:
>>>> We are experiencing an issue where the touch pad on the Framework Laptop
>>>> 16 fails to initialize properly when the "i2c-hid" is loaded early in
>>>> the boot process. This issue is especially prominent when "i2c-hid" is
>>>> built directly into the kernel. When the "i2c-hid" module is built in,
>>>> the issue occurs roughly 50% of the time.
>>>> (https://community.frame.work/t/touchpad-not-working-since-update-archlinux/50304) Moving the module load to later in the boot process appears to resolve this issue by making initialization more likely to succeed. (https://gitlab.archlinux.org/archlinux/mkinitcpio/mkinitcpio/-/releases/v39.1)
>>>>
>>>> Kernel version: Linux artemis 6.9.0-1-git-01560-ga7c840ba5fa7 #1 SMP
>>>> PREEMPT_DYNAMIC Tue, 14 May 2024 01:49:25 +0000 x86_64 GNU/Linux
>>>>
>>>> I'm using the standard Arch Linux AUR "linux-git" package with the
>>>> following kernel configuration changes:
>>>>
>>>> CONFIG_I2C_DEBUG_CORE=y
>>>> CONFIG_I2C_DEBUG_ALGO=y
>>>> CONFIG_I2C_DEBUG_BUS=y
>>>> CONFIG_I2C_HID=y
>>>> CONFIG_I2C_HID_ACPI=y
>>>> CONFIG_I2C_HID_OF=y
>>>> CONFIG_I2C_HID_CORE=y
>>>>
>>>> We tried reverting the following patches without any behavior impact
>>>> (good or bad):
>>>>
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-6.8.y&id=fb49deec375aa5beca4a5d71d7a74ec951872f28
>>>>
>>>> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-6.8.y&id=418c5575d56410c6e186ab727bf32ae32447d497
>>>>
>>>> Impacted devices seem to include: "PIXA3854:00" and "i2c_designware
>>>> AMDI0010:03"
>>> Any news on this? If this is still unresolved I'll bring this to the
>>> attention of the right developer, if this is a recent regressions (it
>>> sounds like it, but it's not exactly clear; and from the first link
>>> above it sounds like it's partly due to a change in arch's approach to
>>> the initramfs).
>>>
>>> Ciao, Thorsten