Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Himanshu Madhani <himanshu.madhani@oracle.com>
To:     Daniel Wagner <dwagner@suse.de>
CC:     "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
        open list <linux-kernel@vger.kernel.org>,
        Hannes Reinecke <hare@suse.de>,
        Sagi Grimberg <sagi@grimberg.me>,
        Keith Busch <kbusch@kernel.org>,
        Ming Lei <ming.lei@redhat.com>,
        Wen Xiong <wenxiong@us.ibm.com>,
        James Smart <jsmart2021@gmail.com>,
        Chao Leng <lengchao@huawei.com>
Subject: Re: [PATCH v7 2/3] nvme-fc: avoid race between time out and tear down
Thread-Topic: [PATCH v7 2/3] nvme-fc: avoid race between time out and tear
 down
Thread-Index: AQHXqUm6bQoXhM+Hf0qUpfm3K4xMBaujjWEA
Date:   Tue, 14 Sep 2021 13:54:04 +0000
Message-ID: <5A104918-0FDD-47A7-A667-7071AB2AEBA9@oracle.com>
References: <20210914092008.40370-1-dwagner@suse.de>
 <20210914092008.40370-3-dwagner@suse.de>
In-Reply-To: <20210914092008.40370-3-dwagner@suse.de>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?SXfbfXj86TrWMDXJyR1lhuqDYZ6Oi+kaolAiWJyrQNRWcu17DL85oQKSn/pu?=
 =?us-ascii?Q?Nhf8Ec8R897nCHiv2Bh/186qoeiYGvaBn53J26tvDyHNtgJZ0NWIGaW3J6nT?=
 =?us-ascii?Q?zZp2il6M2Dv7GLOA5wiVw41N/SArxwaVlZlQeZszvq4g3fTnxNGVp1ckAJM9?=
 =?us-ascii?Q?mo8qoJEQ5nG/2cZwq6Qv7JJmAUyjG09Mj9Wom28Gwlzeaz2TDB61EuvBprtM?=
 =?us-ascii?Q?HT/ZspgbyDlIHO6N2kowWezZ79kxdOsd7c98aN5A3QYmMfmUiNmKsHunqAv5?=
 =?us-ascii?Q?AiRnK8bTvrw0o98lxZXZkz+xyf+zLiPPUdKys9ynfpCWqs8bPPx5Ew/E/HU0?=
 =?us-ascii?Q?v50HhWmzLdeCb76TbFJr1SneYSju+qSFZEi8mJOUdLDatsdBYCYFGCz1JurR?=
 =?us-ascii?Q?3iT7Bt7tXyfG9J5QAN+CkU3keTorErd8b6dNHlufgXMEUnzyBiOlFSo5btnu?=
 =?us-ascii?Q?Y7WqIPYLkWARnA/xrCP7bbfkTefUPs99piMKmE2D/Zs6zyvsvqTqk1ZNAIfd?=
 =?us-ascii?Q?lcCIRrSzHY/QdW3pmWOG2F9rs5QPWPBRt0bhzO7I+re4cyw//MJyjMP50d8n?=
 =?us-ascii?Q?N6OyvkTn1TWCGbkTWp+4/IfBEW8OZLfXHLSqcPxt/DzfqMYQ2i+gEKz6RwUZ?=
 =?us-ascii?Q?bOeu1Yq6aPAlNJigPLGE5h37aTmH7JxYtsRm0Qrhq/YN7QjElnVN/IMnRgbo?=
 =?us-ascii?Q?N5kzjkA8lVJQkzopo+5fVcEHxpK4jAjJgXvgrtX2IlM98ZJjYlI9naouTaEp?=
 =?us-ascii?Q?Al6yTyUPmHxeZRsR4YLKx6LxjcVwLn5pIT5llCa50iOmmmQ4uAGJECyZwXAB?=
 =?us-ascii?Q?WJrZmTtVX7StwYWdZ3ZtWsIRA6fy4zoM5X67RkWxQS9WnG5Re+RolQxRuRUK?=
 =?us-ascii?Q?dZA06dlQfOJqRgrIhv7NQnQE6J19ieH2Q9S7OEf7v6neUCY/qW5jhsr28H0O?=
 =?us-ascii?Q?+bDGjoslizecBAW36z1DF5bnaJjEx5S1GejUWek+uDNB4Bg9+M7S9w2D03B6?=
 =?us-ascii?Q?HNPaHU2PdpiNVNSGcwdRsoP7oBAlja6rojbUAafMHSLWKU4G7pcNO8wRj1iL?=
 =?us-ascii?Q?9j2es6fjGaf9F3cwulG/QeF6GqbUYe2M1E+BCHniV8zR/WiYUc/IRZZPG+1p?=
 =?us-ascii?Q?oQpm1UxAHs+zfvhkRrzvH/cVbl+XwxS2XkA43+4JEDVYt1i7Yalqbhrwg8BI?=
 =?us-ascii?Q?tEO5ybvMnzfa2Cv3dTo5eI5OH2JY/Uqgk5Yy2+sScKBrOSA33y9bleRS3rYb?=
 =?us-ascii?Q?6mOBTd+3Yl/uQweM7wO6k7ckYJ+nvLL01WDbEXdPi3U5YZjKnL90PRfBdCq0?=
 =?us-ascii?Q?8WEzS4zLvGVaUt/0fCMU0AAzIZ7mt839RF4g2P94veOMEw=3D=3D?=
x-ms-exchange-transport-forked: True
Content-Type: text/plain; charset="us-ascii"
Content-ID: <8177ED2B50AA2148AA47663AE1FE5E2B@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB2934.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ca0c4b21-4907-4fda-7817-08d977871d86
X-MS-Exchange-CrossTenant-originalarrivaltime: 14 Sep 2021 13:54:04.8928
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 3m8FP0a1k0h/Tbo/RQRasJD+U9BntSuH0HMrU+SKgNQ3Qc0Laf8d+6BfSywRyCOfoOoyMpBPFwGB0ITq3i/psrIvCKODVQccZS9C58J5lGA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB3127
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10106 signatures=668682
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxscore=0
 mlxlogscore=999 adultscore=0 bulkscore=0 spamscore=0 phishscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109030001 definitions=main-2109140086
X-Proofpoint-GUID: SC7T5Wp8erbUgmdqO8GVBMW6ni_HJKeG
X-Proofpoint-ORIG-GUID: SC7T5Wp8erbUgmdqO8GVBMW6ni_HJKeG
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On Sep 14, 2021, at 4:20 AM, Daniel Wagner <dwagner@suse.de> wrote:
>=20
> From: James Smart <jsmart2021@gmail.com>
>=20
> To avoid race between time out and tear down, in tear down process,
> first we quiesce the queue, and then delete the timer and cancel
> the time out work for the queue.
>=20
> This patch merges the admin and io sync ops into the queue teardown logic
> as shown in the RDMA patch 3017013dcc "nvme-rdma: avoid race between time
> out and tear down". There is no teardown_lock in nvme-fc.
>=20
> Signed-off-by: James Smart <jsmart2021@gmail.com>
> CC: Chao Leng <lengchao@huawei.com>
> Tested-by: Daniel Wagner <dwagner@suse.de>
> Reviewed-by: Daniel Wagner <dwagner@suse.de>
> Reviewed-by: Hannes Reinecke <hare@suse.de>
> ---
> drivers/nvme/host/fc.c | 2 ++
> 1 file changed, 2 insertions(+)
>=20
> diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c
> index b5d9a5507de5..6ebe68396712 100644
> --- a/drivers/nvme/host/fc.c
> +++ b/drivers/nvme/host/fc.c
> @@ -2487,6 +2487,7 @@ __nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl=
 *ctrl, bool start_queues)
> 	 */
> 	if (ctrl->ctrl.queue_count > 1) {
> 		nvme_stop_queues(&ctrl->ctrl);
> +		nvme_sync_io_queues(&ctrl->ctrl);
> 		blk_mq_tagset_busy_iter(&ctrl->tag_set,
> 				nvme_fc_terminate_exchange, &ctrl->ctrl);
> 		blk_mq_tagset_wait_completed_request(&ctrl->tag_set);
> @@ -2510,6 +2511,7 @@ __nvme_fc_abort_outstanding_ios(struct nvme_fc_ctrl=
 *ctrl, bool start_queues)
> 	 * clean up the admin queue. Same thing as above.
> 	 */
> 	blk_mq_quiesce_queue(ctrl->ctrl.admin_q);
> +	blk_sync_queue(ctrl->ctrl.admin_q);
> 	blk_mq_tagset_busy_iter(&ctrl->admin_tag_set,
> 				nvme_fc_terminate_exchange, &ctrl->ctrl);
> 	blk_mq_tagset_wait_completed_request(&ctrl->admin_tag_set);
> --=20
> 2.29.2
>=20

Reviewed-by: Himanshu Madhani <himanshu.madhani@oracle.com>

--
Himanshu Madhani	 Oracle Linux Engineering