Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35;
Date:   Mon, 23 Oct 2023 13:01:10 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Chengfeng Ye <dg573847474@gmail.com>,
        dennis.dalessandro@cornelisnetworks.com
Cc:     leon@kernel.org, dean.luick@cornelisnetworks.com,
        linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] IB/hfi1: Fix potential deadlock on &irq_src_lock and
 &dd->uctxt_lock
Message-ID: <20231023160110.GA886087@nvidia.com>
References: <20230926101116.2797-1-dg573847474@gmail.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230926101116.2797-1-dg573847474@gmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Pvf7vEy9/bDmreYzBTkqmLIcccsckzEM2LySv/EsHUAl0l8Rr82At9/oFlmw?=
 =?us-ascii?Q?+CMApEAlAL3VJKKA/LoPG2YLKSC9MykbAhKLuxc2ydkFp/6+GVVr+k36IyeZ?=
 =?us-ascii?Q?ydV2geZd7kpbv3YGceh4uRgE75eiZU+qPDP7Yg/uLW0X+rsNzPXaupsUsT9b?=
 =?us-ascii?Q?RajURLyyoXJ/ZbMFK641or8hDO+HY8Hrv3gSztSUVaSF6qpQFdQavoAAp6Q5?=
 =?us-ascii?Q?BJsb4qdyjZRi5usOccF35mjMtUfFBSMOaqSz+zLNHV1ghNg6B1ZP7LtOqfuT?=
 =?us-ascii?Q?hwY14V3EpHq4HIq6nzvTnuKXODPuZMLgqCcWh0TAmD9yE+v9geR8RyJw+8fb?=
 =?us-ascii?Q?dS+uw8WoHnoLukSJm4qo9IEUV9j9Qr2NnbN0/+zPuzgux+a/+GRDU8qZU3fF?=
 =?us-ascii?Q?ZLpamhxp9H1aFwfAZVhTpKywRdII0mm9xZ1dnAzPn0MZzopxmTHp0IdSBVeD?=
 =?us-ascii?Q?xKhb00VU2gEWX+LEV/eWq+DvArgtItlyM6OboB1P/w6flB0VxOEqh+/VpPQq?=
 =?us-ascii?Q?eTRk0KQlVZBlhc3e6k4pPiYEcIaBOa8iB6FxGfkR1s7lSkXpISApPcZ5Gxr5?=
 =?us-ascii?Q?EixYOVO8Oh7lkp/Y7RUBq22OfGrRNh244v/W+y305NYOgxFk/c9aWQ+ZrU0Z?=
 =?us-ascii?Q?XEQgrcFcL1qPTAw54TPxaijHgxw02oNGAkgYy1DbuaWmL8s1LPOyaursxH4G?=
 =?us-ascii?Q?xtrvy31nwOCMLNgVnSV3mWXkIVaYAiNoIK0ssD5ElYQzYZ7S8BCKZ6ceGAzT?=
 =?us-ascii?Q?/f3/OcAuxwrRbl+89uXPRI5r8AGj/YQsqHnNe/iwD+X5+nPqdyKQIDH0egJM?=
 =?us-ascii?Q?lbDgVv4BAR6I1EuNo5625jA8lznMTjPsXqt5NzXKdDqqpAAqvXwvkhtxwmAC?=
 =?us-ascii?Q?xj1a4Jz/J0NH0h6UURS1M2+aCXe5E2vWtj10GgjIr7GBqHuygwoIFN5kg46L?=
 =?us-ascii?Q?0Elmj7Iz0IbBZOeZhgD6lUH3+9qqHwuXCVV9nvhfYCZRxhDeVin9UuUieDZ5?=
 =?us-ascii?Q?zI6UEwXiLDlHA1xjYNYcvpB7cBVy8E6BUDVIf4j+fKxQ53zxbKdCEMkunccK?=
 =?us-ascii?Q?P4QCluvb3VQMJRYSbB1n4j4A/DbcO4UiH+iL+FZ9HpWIIuRhEtKZbExw9HeB?=
 =?us-ascii?Q?hDn1JueUPzNbXOM8jDy0OEs34j9wWybFxaeE4b/S+W5VPGlPHmplLSdRIwJL?=
 =?us-ascii?Q?nHIXwTM4LvfDVwxN6ObTUtiUK7jr89g2S9Da2k+RlfeenyNsdmG75WuGe2BX?=
 =?us-ascii?Q?t+OHLxj8ZVge/taP6ZCJNWQ9QeY4SMqum9IUolYasWy+pd9gsP9bTjSN1cpD?=
 =?us-ascii?Q?t5WP/IXDn2oBKVTE2RX8p9dYyCbSp7QFz+omRlwEf3noqVPlFvwb361WWjae?=
 =?us-ascii?Q?RU3zYVzGDvqxuwkdGk5PsbZDQ8OY9HJeEoTejau2rWtkD+LM2pI9YUbkTypG?=
 =?us-ascii?Q?39/ldBnvZMLs5QR71ans5g3JKtjz+yXjWuwKEV9fSJqlFkSsOmCQTdvl2sZy?=
 =?us-ascii?Q?LpDvY78cg2eUlxhwy0HjRCL0lxBNzUfdokfLqrsxOzk86OYxGXOUltrvzt21?=
 =?us-ascii?Q?pC56bKrFBnJnSdIt2ttgqqyE1rD24VIETxxdf/FO?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 01960560-4d2a-4e44-96da-08dbd3e1475b
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2023 16:01:12.5379
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: cyBpPw1/l+juI8IM97yXoE4sIFv74eANrzCiRrkMOCq5JbifkbIxA+EnvGZV0kWp
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4153
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 23 Oct 2023 09:01:41 -0700 (PDT)

On Tue, Sep 26, 2023 at 10:11:16AM +0000, Chengfeng Ye wrote:
> handle_receive_interrupt_napi_sp() running inside interrupt handler
> could introduce inverse lock ordering between &dd->irq_src_lock
> and &dd->uctxt_lock, if read_mod_write() is preempted by the isr.
> 
>           [CPU0]                                        |          [CPU1]
> hfi1_ipoib_dev_open()                                   |
> --> hfi1_netdev_enable_queues()                         |
> --> enable_queues(rx)                                   |
> --> hfi1_rcvctrl()                                      |
> --> set_intr_bits()                                     |
> --> read_mod_write()                                    |
> --> spin_lock(&dd->irq_src_lock)                        |
>                                                         | hfi1_poll()
>                                                         | --> poll_next()
>                                                         | --> spin_lock_irq(&dd->uctxt_lock)
>                                                         |
>                                                         | --> hfi1_rcvctrl()
>                                                         | --> set_intr_bits()
>                                                         | --> read_mod_write()
>                                                         | --> spin_lock(&dd->irq_src_lock)
> <interrupt>                                             |
>    --> handle_receive_interrupt_napi_sp()               |
>    --> set_all_fastpath()                               |
>    --> hfi1_rcd_get_by_index()                          |
>    --> spin_lock_irqsave(&dd->uctxt_lock)               |
> 
> This flaw was found by an experimental static analysis tool I am
> developing for irq-related deadlock.
> 
> To prevent the potential deadlock, the patch use spin_lock_irqsave()
> on &dd->irq_src_lock inside read_mod_write() to prevent the possible
> deadlock scenario.
> 
> Signed-off-by: Chengfeng Ye <dg573847474@gmail.com>
> ---
>  drivers/infiniband/hw/hfi1/chip.c | 5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)

Dennis? This needs your ack/nack

Thanks,
Jason