Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Niklas Cassel <Niklas.Cassel@wdc.com>
To:     Jens Axboe <axboe@kernel.dk>
CC:     Bart Van Assche <bvanassche@acm.org>,
        Damien Le Moal <Damien.LeMoal@wdc.com>,
        Paolo Valente <paolo.valente@linaro.org>,
        Niklas Cassel <Niklas.Cassel@wdc.com>,
        "linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: [RFC PATCH 1/2] blk-mq: don't call callbacks for requests that
 bypassed the scheduler
Thread-Topic: [RFC PATCH 1/2] blk-mq: don't call callbacks for requests that
 bypassed the scheduler
Thread-Index: AQHXm0DcNbCoMv+LuUuRBM4IFE0DyA==
Date:   Fri, 27 Aug 2021 12:41:31 +0000
Message-ID: <20210827124100.98112-2-Niklas.Cassel@wdc.com>
References: <20210827124100.98112-1-Niklas.Cassel@wdc.com>
In-Reply-To: <20210827124100.98112-1-Niklas.Cassel@wdc.com>
Accept-Language: en-US
Content-Language: en-US
wdcipoutbound: EOP-TRUE
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?fhGk8qkU29WW7+b952CMmnUKK6jS+TipynBoJcvLlTdE0JNRwExJlvePmj?=
 =?iso-8859-1?Q?9R8CrYDQTxfULUUNjXwIfyRsKlDBAGlQBcvbw8ct1rFgHemUbZgyRmA9MV?=
 =?iso-8859-1?Q?oqu+bQR07fwJZb3zkw5dDsSwhDFQvyQT6QQco8udP6G6UyNCJ5GS/Mlk9M?=
 =?iso-8859-1?Q?hvsVGViG7va8c6gU+ZmE7wfAJnMYrF6h+PHXZe1sN3MG7aYE1DjdY681ja?=
 =?iso-8859-1?Q?cNd94OTb4hpHWDCSwxD14sXNWGpIGKePuqZa9MIFAZZunEKljcj2kCtEge?=
 =?iso-8859-1?Q?BOBYb320X9Ve2nIrZlY98ZURa/bjXfnSB6btfNgk2DavzgLiLeo67kMo0z?=
 =?iso-8859-1?Q?PPrXh9OTUYJpUbHMNOGtTk1UOx020Qgu95HR/d93pITk3XhXeAaQrB7ep5?=
 =?iso-8859-1?Q?heVonl/4e8WOa4NuXlauj0ysYC+hftZNFs8Dm2vROfOqFza1+kAeYZjYbn?=
 =?iso-8859-1?Q?ze1RxZKoWms3Ytzc21l1UM3bln06Ajd2FZQawyTPbESPZ4BYJDO9QtEGo1?=
 =?iso-8859-1?Q?txvafLK4ro8LyzkJBkOfDLlCfU3R9V0W20Dq5J951EqwNJf8ns35Owk85q?=
 =?iso-8859-1?Q?5QFF81bU1wfnOrdoo51XKIrX2u0ANqSRjIMNNUY0qKGK4XrgofIWFJoQOT?=
 =?iso-8859-1?Q?cpWYXD4v2RqMEaVEm/T1imRVv4fJNhY7eryNvMl3jgoDhltrjG+qIKsBtE?=
 =?iso-8859-1?Q?2qfO8bZl27lMg+jeZ6t7ymai7bl/FrVJUx+Mqsev7pSaMkcy1YrP8z+o5B?=
 =?iso-8859-1?Q?fmjrjP/QcuARadA8gkCfEw5y/cCeQyl8Ghmq69gWSbleIVsSzeRGIcqjY9?=
 =?iso-8859-1?Q?+XuDvKreD80iV+EGR7Va0QXmVnoY706BtIATWnnWCAly50/HYZ8EssLOeZ?=
 =?iso-8859-1?Q?5bNT/ik5rERnQcsUXunhefw2P3CB1aalvC55gchhM/jiAb8Rmoxi1vhqKX?=
 =?iso-8859-1?Q?/SzXF4fLeJjjWmdEUR44vhv94dq+uWJfXQt1o3KcsYM4VAn5Xh0McYw7NQ?=
 =?iso-8859-1?Q?maicjVRgWRHMlO6Zvq1Asa6U6otIhY/Mcto63fRvHXLTpfs5L8FyHtBXOc?=
 =?iso-8859-1?Q?WWKOXAiQyosGkY6JniPD5bzlowD+1n4kYktbX2O2B6Ie5eLe//pC373Qia?=
 =?iso-8859-1?Q?3JAT6CZFKfGUjOB25tPZJxZKkZztXoR7jHk3TGTK7Yj4NAxrTxAdAUGQEb?=
 =?iso-8859-1?Q?HJ6US2D7GvLWYZKcsztH14PXtTRxfolHMZb/0C14FiSrjOVoXAslqxjrNH?=
 =?iso-8859-1?Q?O1jqntfytFqFrBM/MW4KwEEdFqbOMryKGVUp/6c7WR7lYXQUj5VDuShCT0?=
 =?iso-8859-1?Q?WjQwm4dIto7JC/R6JebaOHM3GzWvJRhRsjxLy3UtzNc42MxuzVCVd59wNQ?=
 =?iso-8859-1?Q?04jZLoYxPUX8Kwp0Xa2vCthlmk4nay/g=3D=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: wdc.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PH0PR04MB7158.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e32570af-78f0-497a-9954-08d96957ff8c
X-MS-Exchange-CrossTenant-originalarrivaltime: 27 Aug 2021 12:41:31.8448
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: VzcuuZy2Gj7HRtL0nes8DrjK8/8DT2Ts1lo3EegmdKElAL5Enq4XXdZaSowaiSxNWfxdku1fRXNtiBjwtHJp7w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR04MB7702
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

From: Niklas Cassel <niklas.cassel@wdc.com>

Currently, __blk_mq_alloc_request() calls ops.prepare_request and sets
RQF_ELVPRIV.

Therefore, (if the request is not a flush) the RQF_ELVPRIV flag will be
set for the request in blk_mq_submit_bio(), regardless if the request
was submitted to a scheduler, or bypassed the scheduler.

Later, blk_mq_free_request() checks if the RQF_ELVPRIV flag is set,
if it is, the ops.finish_request callback will be called.

The problem with this is that the finish_request scheduler callback
will be called for requests that bypassed the scheduler.

Fix this by calling the scheduler ops.prepare_request callback, and
set the RQF_ELVPRIV flag only immediately before calling the insert
callback.

This way, we can reuse the flag, and we don't need to add any
additional checks in blk_mq_sched_requeue_request() and
blk_mq_free_request(), since they already only do the callback
if RQF_ELVPRIV is set, and the existing .prepare_request callbacks
should still work without needing modifications.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
---
 block/blk-mq-sched.c   | 20 ++++++++++++++++++++
 block/blk-mq.c         | 13 -------------
 include/linux/blkdev.h |  3 ++-
 3 files changed, 22 insertions(+), 14 deletions(-)

diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c
index 0f006cabfd91..eacacb7088c1 100644
--- a/block/blk-mq-sched.c
+++ b/block/blk-mq-sched.c
@@ -466,6 +466,14 @@ void blk_mq_sched_insert_request(struct request *rq, b=
ool at_head,
 	if (e) {
 		LIST_HEAD(list);
=20
+		rq->elv.icq =3D NULL;
+		if (e && e->type->ops.prepare_request) {
+			if (e->type->icq_cache)
+				blk_mq_sched_assign_ioc(rq);
+
+			e->type->ops.prepare_request(rq);
+			rq->rq_flags |=3D RQF_ELVPRIV;
+		}
 		list_add(&rq->queuelist, &list);
 		e->type->ops.insert_requests(hctx, &list, at_head);
 	} else {
@@ -495,6 +503,18 @@ void blk_mq_sched_insert_requests(struct blk_mq_hw_ctx=
 *hctx,
=20
 	e =3D hctx->queue->elevator;
 	if (e) {
+		struct request *rq;
+
+		list_for_each_entry(rq, list, queuelist) {
+			rq->elv.icq =3D NULL;
+			if (e && e->type->ops.prepare_request) {
+				if (e->type->icq_cache)
+					blk_mq_sched_assign_ioc(rq);
+
+				e->type->ops.prepare_request(rq);
+				rq->rq_flags |=3D RQF_ELVPRIV;
+			}
+		}
 		e->type->ops.insert_requests(hctx, list, false);
 	} else {
 		/*
diff --git a/block/blk-mq.c b/block/blk-mq.c
index 9d4fdc2be88a..3527dd9fd10e 100644
--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -328,19 +328,6 @@ static struct request *blk_mq_rq_ctx_init(struct blk_m=
q_alloc_data *data,
 	data->ctx->rq_dispatched[op_is_sync(data->cmd_flags)]++;
 	refcount_set(&rq->ref, 1);
=20
-	if (!op_is_flush(data->cmd_flags)) {
-		struct elevator_queue *e =3D data->q->elevator;
-
-		rq->elv.icq =3D NULL;
-		if (e && e->type->ops.prepare_request) {
-			if (e->type->icq_cache)
-				blk_mq_sched_assign_ioc(rq);
-
-			e->type->ops.prepare_request(rq);
-			rq->rq_flags |=3D RQF_ELVPRIV;
-		}
-	}
-
 	data->hctx->queued++;
 	return rq;
 }
diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 2e12320cb121..a5047c7e9448 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -81,7 +81,8 @@ typedef __u32 __bitwise req_flags_t;
 #define RQF_FAILED		((__force req_flags_t)(1 << 10))
 /* don't warn about errors */
 #define RQF_QUIET		((__force req_flags_t)(1 << 11))
-/* elevator private data attached */
+/* The request has been inserted to an elevator, and thus has private
+   data attached */
 #define RQF_ELVPRIV		((__force req_flags_t)(1 << 12))
 /* account into disk and partition IO statistics */
 #define RQF_IO_STAT		((__force req_flags_t)(1 << 13))
--=20
2.31.1