Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
Message-ID: <47e8fd80-3f87-4b87-a875-035e69961392@oracle.com>
Date:   Tue, 21 Nov 2023 11:30:51 +0000
User-Agent: Mozilla Thunderbird
Subject: Re: scsi regression that after months is still not addressed and now
 bothering 6.1.y users, too
Content-Language: en-US
To:     Thorsten Leemhuis <regressions@leemhuis.info>,
        Greg KH <gregkh@linuxfoundation.org>,
        Sagar Biradar <sagar.biradar@microchip.com>,
        James Bottomley <James.Bottomley@HansenPartnership.com>,
        "Martin K. Petersen" <martin.petersen@oracle.com>,
        Adaptec OEM Raid Solutions <aacraid@microsemi.com>
Cc:     "stable@vger.kernel.org" <stable@vger.kernel.org>,
        Sasha Levin <sashal@kernel.org>,
        Linux kernel regressions list <regressions@lists.linux.dev>,
        Hannes Reinecke <hare@suse.de>,
        scsi <linux-scsi@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Gilbert Wu <gilbert.wu@microchip.com>
References: <c6ff53dc-a001-48ee-8559-b69be8e4db81@leemhuis.info>
From:   John Garry <john.g.garry@oracle.com>
Organization: Oracle Corporation
In-Reply-To: <c6ff53dc-a001-48ee-8559-b69be8e4db81@leemhuis.info>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?N0t3ald4ZTNMQjNCSjYzd2xINFoxTm11MGJWdi9OWVRZU3d0ZGJYZVdTcGVn?=
 =?utf-8?B?Q3gzYzZJdm9qcURzT0IwUHIxQmpVZEx5MzJERU5oMGhaWTV0L2xCUmIxK2lR?=
 =?utf-8?B?dUFlV3JsMXo2R2d3Q3Mya2lGYkdpbDhCMjhMTGQwczBwN1M1Zk1qZWtXUUpP?=
 =?utf-8?B?MzFMU2FGUkFINzRvN3EzZHhCZzlwaTgwYTdwRDlVZ0Vxb0ZRS1lDczdrcWNM?=
 =?utf-8?B?TmxmbVVRSG9iSzFQQUcvWFRhNkdvbVJVZ3FjL0hLNHRkakpaVUoxeFpleGJI?=
 =?utf-8?B?d1JGaHhKUmJ6aS8rNzE3Q0Nhc1JvT2hKVlp4QTNLL1lCNEhoM0NUUHlIQjNT?=
 =?utf-8?B?cThSOVpKcmtuWnhqQU5oOGd6K1YwQ3c3cjAyLzBsSXBpTkZDTEpPb2NLMWVE?=
 =?utf-8?B?OFdRMVIyU1UzVWR6VUhiV284c1hKZE5EYkVYWTVDaDJHNGpLT1Y0THJQeldI?=
 =?utf-8?B?RkVkZVEyeGpxQzhDcmpaOFRmSHhLNjFORzRnS3hsbGhMOHZ0czZCU1ZpTjN0?=
 =?utf-8?B?Qi9NM29rT2NhNGtlVU1mQ3NVd3JxZGJGa0wrekZXNytMMXlab2U3VmpBMmJW?=
 =?utf-8?B?VUxBZFN3MFZSRXhSOXZpZGVvUUx5NzFOa2VLQ3FXS0dIN1dqYW5BeWlBNStz?=
 =?utf-8?B?THd1ZHhyRDhwMWFGWGJaeWlwUnd1WDJ3TzBmYXd1dk5xdyt0OGdFWnllZi9x?=
 =?utf-8?B?Q1h1NUVyT1dBdmF1MnliUGpNMlpJTThyU1R2bjQ5VnJtRGZwRGxscm1uSFBm?=
 =?utf-8?B?MnR2N09BK29iS1ZiMCtpOWN2aHpISFo2aDlPSlJWR1NOSzhLbWJQL3hwY3JC?=
 =?utf-8?B?U3kwbUFTU2g1a2I0a0hxeWcyQm1SS1Rqa2M0eGpQTmtYaUlxamtFcXBnNGVK?=
 =?utf-8?B?ejRlWDdSRjBvU0tpYnNybVJoaysyTzNEMkdjanUxSGFISjNIdDNHNFV4V1B0?=
 =?utf-8?B?SVNTZjdyYjlReWpRcDRNYnlrWmtkdkc4Tkpaei9QSmQwbzh2RTg5ZTJvdlR6?=
 =?utf-8?B?eGtSWkdtdFY5VG9vV2xhdllRYUpXTXhwS081TFBCemVuVUdVdDUySTY0OFlj?=
 =?utf-8?B?SWxsaDdTVC8zakppNVQyMFNDcFlQUktzcGEvUGRNYVpBeVEvZ01sRkIvV0g4?=
 =?utf-8?B?M0ZndXhZUC8xT2ZpVVcyNWFWSkZZY05iRGF3WDFmZ1Rsa3RsZ3pvY0V0ZUtp?=
 =?utf-8?B?OVYzQ2UrRUNnZittckh3Ui9pNWR1TzlWRDYwMWpySXp1NHFqQ0h5OEZsZjZo?=
 =?utf-8?B?TE5OYVd1aDNCVFQzVU15SVhpcm9jRnZCb3dUTDE3WmhLK2Fxc1g1Rzdmekwv?=
 =?utf-8?B?ZmZycTdqeEtZQmgwNE1FWGFEV0pQeC8wT25jSk1EbzAya0FJVzFuRWozeXNN?=
 =?utf-8?B?eUlEU2lOR0RXdDMxZzlCdWV6MmhPSVNhaktTNlNZZGtOakNUL3JLdE5vMUxh?=
 =?utf-8?B?clZkKzFuWlV3alk4VDBwYjI2ZTJYV2FVYXlBdjY2bHNTdTNadXVKM01iMEwr?=
 =?utf-8?B?RHlTREhpREduYUQ5bW42Tm1Qa0w1SWd3Q1R0dzVPNHNSVnNycmtBNnJnMVVJ?=
 =?utf-8?B?T3puU3FKZzRiNW9Md3ZYdkk1THZYUy9BcDk1YzU0K3J5eUc0KzV0d1hLK0FM?=
 =?utf-8?B?c2xSdHd1OVQ3K2toaGxYU09Kdm5NOFh5S1RUSFF3MWZGV056MXl4UkV4cWFk?=
 =?utf-8?B?eHkyejZUaTRYQ0ViQkxiOHQ3ZG0weFh4MGxndURIVEkrV3QxSHZITjdtRFcx?=
 =?utf-8?B?U3VLb1BXdllLc3Y0Tm5LQ1lyR21yWVJzMXM5R3JzSnRwemd0SVdaY3ZXOGk2?=
 =?utf-8?B?NFVoakVmM0FTaUVIWGNwLzVjdDFGUWJxaEZEY3pESGZ6VXZFcVVmVnFYdVMw?=
 =?utf-8?B?VDlwbTkyekU5MnUxcGllQkpETjZkMHUyWmVSV0h4ZzF3dXJsWmJqYjBZcE9i?=
 =?utf-8?B?eWRBbFEzR1VQV0dlMEs1SXVWMmJNOVZuS1VCMXVpUzFKZnI4Q05hM0RYdVhv?=
 =?utf-8?B?YXhVQ2crU1RiOG5kejU1STlGemV6d0kxcjRjY3puaGF0aEVBMXpTSmFXcnBZ?=
 =?utf-8?B?Z2hYbDVrVzJrOERsSE5EVm9nQkFnUHY5S0hRNHJNVlRuWjVLRzk3WUlVYkJx?=
 =?utf-8?Q?qG4wTA7wnqxsmOiPiuV3sFc3F?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?YmNna1h3STZHZEhtWXhiSEUwTlZEaG52OEoxZnRkMEIzeWJWU1Y5bEpPWkY5?=
 =?utf-8?B?T1FtZDJJeStxNVhrWnFPaGFCNkdLSUFXeGRLb25PekdLYldnUWwvdDlSdTRh?=
 =?utf-8?B?SmRodWlWbGY5MG5YUDNoQ3pQczJEYjVHRlVFOFZaVVBhSnNZQ0pxcjgwc3Ns?=
 =?utf-8?B?UHhyWG1kYXg3QXVQMForTmFPci9SdHo2WmZqeGRaLzZCd3IxRWhjUXpDVzBh?=
 =?utf-8?B?bEl1RnZ2RkFJWExBU2lUNWgzWlVpdEkyc0p1QW1jNlhTU0JKWjJSRXRUdUxD?=
 =?utf-8?B?TmF3WjdKR1NvcUJqM1RGTTUwTUJ1Q3U2SmVlQlZMUS9qSnpLQmRVZGlOTzlL?=
 =?utf-8?B?NGNERjZxNVdsS0MyaHA4MVlIRmpIeUV0RGxDdzhMNWVTOGQ4LzBzbnQ2YVA1?=
 =?utf-8?B?Nmh5c0g0T2ZDa3h1NFdDczZFK1N1cEd4L2dERWd5Z3BFU2lXdUdFMzc4ZU5z?=
 =?utf-8?B?NCtZd3BBTllIVkZDSHBCMHlKODA1Lzk1OEM0QUpHcDVWUzkrcjZXOERCd3lS?=
 =?utf-8?B?T29PZy9VR1RRQy9ScUVqOE8zdDVNZWRVcU9GWFBiRTNqRzI5S3l1d2FObnRh?=
 =?utf-8?B?UnIzNFVmbW8zdUZtZ0JUUC9WUGVTTGtSUEFWRkpYRDhlV01zZkxnbEovVyt1?=
 =?utf-8?B?ZTdrejNCb3RROCtUNTNUbEdZWGZKZFdHUGp2bWN3Vk1icEVsSEJmT2h5U0ti?=
 =?utf-8?B?VmZHSnNVc3ZpY1RmbGdZVEdHWDVNVDFXZERFZVlJdVkyOG8vVDFqNDJTT1RV?=
 =?utf-8?B?QjdKWGNaWjRkN1JuaUczMXRtZERUaE56Q29ScnJmV3hrZHdtcXJ0SWxkbVJo?=
 =?utf-8?B?cDRXakdMV2RTVDFDZ1lRQ0hKcUM3ZFZZQjd3ZEZZT3NUL1RWZUtFeGtDdTJx?=
 =?utf-8?B?M0FhUmZKNitnVlJHMk9lS0VRNklSRlQyc3lpMkhsbnY0U21pWFFCVjE1dWVz?=
 =?utf-8?B?Und5a1dONmZnOHBnZk5jWER4Zk1CTDFMNmE4cUp1bDdFMWEveGdtRlppR2cz?=
 =?utf-8?B?Z0o5UjFaV094Ylpmblh1Y01BcGpmTUlMcURFYkZFL3hPSll4ZlZ6UENXcGpz?=
 =?utf-8?B?bUFEZ24yT2FXSVFCeHhFM1c1bkNvaDdGMWtOT0hmWTBQZmthcDZCYjZTWkZQ?=
 =?utf-8?B?T2pFeXJsWU5HcUxQUlpaMlhlUGFibG1LS2g5QzdrOS9KRDNQa0xQNE4wdG9T?=
 =?utf-8?B?bFhTSUhlWWN3NFdFQjRuS2xUZjN0N3J1aDZVd3BYM25XaHZ6QWNCVlIwMFV3?=
 =?utf-8?B?UXRrclhPUWQ1eTVZMmgrU2hvSHVQTU9FSG93RmN2Vm9LVzlBQUpOck5peHZw?=
 =?utf-8?B?QjJyY1FjV3pobzE3UlVIbVFqbm15V28vcHNjVGRUcHAzdzQySWhSa2dDRnhh?=
 =?utf-8?B?Um5nbUJaNUkrVDdnOFZpMndEM0I3NXUvUWxGZmMzWkx6ZGJLcmNKcUxLRUpi?=
 =?utf-8?B?TGlvNTJTbnQrVzkyVjVWZGRseUt6MzdVSnJXSVZ3PT0=?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 62587270-bca2-434e-fde4-08dbea855395
X-MS-Exchange-CrossTenant-AuthSource: DM6PR10MB4313.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Nov 2023 11:30:55.9225
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: a8PQWIG3LngGK8QgXxPojhTrSaXd2sA2Rmv+dSXj8Emz4A03QDHdeWt4XVA0A0f5JAUpKTVFMblxNqrTSDkjQA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB7033
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.987,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2023-11-21_04,2023-11-21_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 bulkscore=0
 malwarescore=0 mlxlogscore=999 adultscore=0 phishscore=0 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2311060000
 definitions=main-2311210089
X-Proofpoint-ORIG-GUID: lK0W2RvZ41XI8Xe4herB55PVeey7uJFB
X-Proofpoint-GUID: lK0W2RvZ41XI8Xe4herB55PVeey7uJFB
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Tue, 21 Nov 2023 03:31:53 -0800 (PST)

On 21/11/2023 09:50, Thorsten Leemhuis wrote:
> Quite a few machines with Adaptec controllers seems to hang for a few
> tens of seconds to a few minutes before things start to work normally
> again for a while:
> https://urldefense.com/v3/__https://bugzilla.kernel.org/show_bug.cgi?id=217599__;!!ACWV5N9M2RV99hQ!L26RD0hu99l3f709EFnXU_V7OaB1jG4Hi7BjKvxRuhDWKFmjrgfksLuXA6eBrBCRtOT8JcRRUvzRsHbyEm41r7tL_pbDfw$  
> 
> That problem is apparently caused by 9dc704dcc09eae ("scsi: aacraid:
> Reply queue mapping to CPUs based on IRQ affinity") [v6.4-rc7]. That
> commit despite a warning of mine to Sasha recently made it into 6.1.53
> -- and that way apparently recently reached more users recently, as
> quite a few joined that ticket.

Is there a full kernel log for this hanging system?

I can only see snippets in the ticket.

And what does /sys/class/scsi_host/host*/nr_hw_queues show?

Thanks,
John