Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Message-ID: <3bca3296-d998-98a5-bf8f-53b0720869d3@linux.vnet.ibm.com>
Date:   Mon, 8 Nov 2021 08:16:29 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.2.0
Subject: Re: kernel 5.15 does not boot with 3ware card (never had this issue
 <= 5.14) - scsi 0:0:0:0: WARNING: (0x06:0x002C) : Command (0x12) timed out,
 resetting card
Content-Language: en-US
To:     Justin Piszcz <jpiszcz@lucidpixels.com>,
        Bart Van Assche <bvanassche@acm.org>
Cc:     LKML <linux-kernel@vger.kernel.org>, linux-scsi@vger.kernel.org
References: <006a01d7cead$b9262d70$2b728850$@lucidpixels.com>
 <a4a88807-8f52-ef9a-c58e-0ff454da5ade@acm.org>
 <CAO9zADxiobgwDE5dtvo98EL0djdgQyrGJA_w4Oxb+pZ9pvOEjQ@mail.gmail.com>
 <CAO9zADycForyq9cmh=epw9r-Wzz=xt32vL3mePuBAPehCgUTjw@mail.gmail.com>
 <50a16ee2-dfa4-d009-17c5-1984cf0a6161@linux.vnet.ibm.com>
 <CAO9zADwVnuKU-tfZxm4USjf76yJhTZqWfZw4yspv8sc93RuBbQ@mail.gmail.com>
 <e0c2935d-d961-11a0-1b4c-580b55dc6b59@acm.org>
 <002401d7d305$082971b0$187c5510$@lucidpixels.com>
 <CAO9zADzWcpwZkfJ5VZGZZJT39KQEUr9yGqqCnP18mk7ZAZxbBw@mail.gmail.com>
From:   Douglas Miller <dougmill@linux.vnet.ibm.com>
In-Reply-To: <CAO9zADzWcpwZkfJ5VZGZZJT39KQEUr9yGqqCnP18mk7ZAZxbBw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Precedence: bulk

The commit I referenced earlier does point back to the commit that 
caused the problem (that I saw). There was a series of commits related 
to IRQ domains, this one seems to have actually caused the problem I saw:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=a5f3d2c17b07


On 11/7/21 07:46, Justin Piszcz wrote:
> On Sat, Nov 6, 2021 at 7:54 AM Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>>
>>
>> -----Original Message-----
>> From: Bart Van Assche <bvanassche@acm.org>
>> Sent: Wednesday, November 3, 2021 12:23 PM
>> To: Justin Piszcz <jpiszcz@lucidpixels.com>; Douglas Miller <dougmill@linux.vnet.ibm.com>
>> Cc: LKML <linux-kernel@vger.kernel.org>; linux-scsi@vger.kernel.org
>> Subject: Re: kernel 5.15 does not boot with 3ware card (never had this issue <= 5.14) - scsi 0:0:0:0: WARNING: (0x06:0x002C) : Command (0x12) timed out, resetting card
>>
>> On 11/3/21 9:18 AM, Justin Piszcz wrote:
>>> Thanks!-- Has anyone else reading run into this issue and/or are there
>>> any suggestions how I can troubleshoot this further (as all -rc's have
>>> the same issue)?
>> How about bisecting this issue
>> (https://www.kernel.org/doc/html/latest/admin-guide/bug-bisect.html)?
>>
>> [ .. ]
>>
>> I was having some issues finding a list of changes with git bisect, so I started checking the kernel .config and boot parameters:
>>
>> I found the option that was causing the system not to boot (tested with 5.15.0 and latest linux-git as of 6 NOV 2021)
>> append="3w-sas.use_msi=1"
>>
>> 3w-sas.use_msi defaults to 0 (so now it is using IR-IO-APIC instead of MSI but now the machine boots using 5.15)
>> https://lwn.net/Articles/358679/
>>
>> Something between 5.14 and 5.15 changed regarding x86_64's handling of Message Signaled Interrupts.
>> ... which causes the kernel to no longer boot when 3w-sas.use_msi=1 is specified starting with 5.15.
> This only partially fixes the issues, trying to reboot also results in
> a hard lockup on cpu 1 (this is semi-reproducible)
> https://installkernel.tripod.com/5.15-reboot-lockup.jpg
>
> Back to 5.14.x for now...
>
>
>
> Justin.