Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <006a01d7cead$b9262d70$2b728850$@lucidpixels.com>
 <a4a88807-8f52-ef9a-c58e-0ff454da5ade@acm.org> <CAO9zADxiobgwDE5dtvo98EL0djdgQyrGJA_w4Oxb+pZ9pvOEjQ@mail.gmail.com>
 <CAO9zADycForyq9cmh=epw9r-Wzz=xt32vL3mePuBAPehCgUTjw@mail.gmail.com>
 <50a16ee2-dfa4-d009-17c5-1984cf0a6161@linux.vnet.ibm.com> <CAO9zADwVnuKU-tfZxm4USjf76yJhTZqWfZw4yspv8sc93RuBbQ@mail.gmail.com>
 <e0c2935d-d961-11a0-1b4c-580b55dc6b59@acm.org> <002401d7d305$082971b0$187c5510$@lucidpixels.com>
In-Reply-To: <002401d7d305$082971b0$187c5510$@lucidpixels.com>
From:   Justin Piszcz <jpiszcz@lucidpixels.com>
Date:   Sun, 7 Nov 2021 08:46:56 -0500
Message-ID: <CAO9zADzWcpwZkfJ5VZGZZJT39KQEUr9yGqqCnP18mk7ZAZxbBw@mail.gmail.com>
Subject: Re: kernel 5.15 does not boot with 3ware card (never had this issue
 <= 5.14) - scsi 0:0:0:0: WARNING: (0x06:0x002C) : Command (0x12) timed out,
 resetting card
To:     Bart Van Assche <bvanassche@acm.org>,
        Douglas Miller <dougmill@linux.vnet.ibm.com>
Cc:     LKML <linux-kernel@vger.kernel.org>, linux-scsi@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Sat, Nov 6, 2021 at 7:54 AM Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>
>
>
> -----Original Message-----
> From: Bart Van Assche <bvanassche@acm.org>
> Sent: Wednesday, November 3, 2021 12:23 PM
> To: Justin Piszcz <jpiszcz@lucidpixels.com>; Douglas Miller <dougmill@linux.vnet.ibm.com>
> Cc: LKML <linux-kernel@vger.kernel.org>; linux-scsi@vger.kernel.org
> Subject: Re: kernel 5.15 does not boot with 3ware card (never had this issue <= 5.14) - scsi 0:0:0:0: WARNING: (0x06:0x002C) : Command (0x12) timed out, resetting card
>
> On 11/3/21 9:18 AM, Justin Piszcz wrote:
> > Thanks!-- Has anyone else reading run into this issue and/or are there
> > any suggestions how I can troubleshoot this further (as all -rc's have
> > the same issue)?
>
> How about bisecting this issue
> (https://www.kernel.org/doc/html/latest/admin-guide/bug-bisect.html)?
>
> [ .. ]
>
> I was having some issues finding a list of changes with git bisect, so I started checking the kernel .config and boot parameters:
>
> I found the option that was causing the system not to boot (tested with 5.15.0 and latest linux-git as of 6 NOV 2021)
> append="3w-sas.use_msi=1"
>
> 3w-sas.use_msi defaults to 0 (so now it is using IR-IO-APIC instead of MSI but now the machine boots using 5.15)
> https://lwn.net/Articles/358679/
>
> Something between 5.14 and 5.15 changed regarding x86_64's handling of Message Signaled Interrupts.
> ... which causes the kernel to no longer boot when 3w-sas.use_msi=1 is specified starting with 5.15.

This only partially fixes the issues, trying to reboot also results in
a hard lockup on cpu 1 (this is semi-reproducible)
https://installkernel.tripod.com/5.15-reboot-lockup.jpg

Back to 5.14.x for now...


Justin.