Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Niklas Cassel <Niklas.Cassel@wdc.com>
To:     Ming Lei <ming.lei@redhat.com>
CC:     Jens Axboe <axboe@kernel.dk>, Bart Van Assche <bvanassche@acm.org>,
        Damien Le Moal <Damien.LeMoal@wdc.com>,
        Paolo Valente <paolo.valente@linaro.org>,
        "linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [RFC PATCH 1/2] blk-mq: don't call callbacks for requests that
 bypassed the scheduler
Thread-Topic: [RFC PATCH 1/2] blk-mq: don't call callbacks for requests that
 bypassed the scheduler
Thread-Index: AQHXm0DcNbCoMv+LuUuRBM4IFE0DyKuHWDyAgAR5gwCAAAZ3AIAMwawA
Date:   Tue, 7 Sep 2021 12:59:32 +0000
Message-ID: <YTdiMoPNa/10VWSC@x1-carbon>
References: <20210827124100.98112-1-Niklas.Cassel@wdc.com>
 <20210827124100.98112-2-Niklas.Cassel@wdc.com> <YSjoZ/cul4w2l8tG@T590>
 <YSypVLe8vxmA+pUn@x1-carbon> <YSyuwCKi2sS/RaXS@T590>
In-Reply-To: <YSyuwCKi2sS/RaXS@T590>
Accept-Language: en-US
Content-Language: en-US
wdcipoutbound: EOP-TRUE
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?pHBUsIwAcUI+AsOaGFcE+zgqbZlx9x65G47sfPyrH5sW8Rcpc8v4CUnYf1Wm?=
 =?us-ascii?Q?X9oXv9b5FYzUKRp2PQRGb++iQDkLSX/z9AwveVsoW+f29F9UYeq7jjinfYTG?=
 =?us-ascii?Q?zT65nR3CubUUoQxGN8Jx5EolsnS336xipqiw7YGoFSTMaO/h1ySRy+yGYGYu?=
 =?us-ascii?Q?ae82eBnh/Oyg41Vx0l22Duz02wwRxyP3XYdA/GkD5sMZWDCnyjucCzk1CTRK?=
 =?us-ascii?Q?9cxM97vc3pZeHN1sCqchhPg4a3Die/UB/Jg+CcQafHsO55aLkRw0sWxTvs1U?=
 =?us-ascii?Q?4eXq6901E3kkNyB1LfM96NAGbQkuxIYW/Ytk0ruAOU7DTF3i6nrzDjw6R//Y?=
 =?us-ascii?Q?9KBP4HjM5hD0K+JVhKCwVHWIJKNdE5ETLVGrfbqyjW44ECj4BdLSLKH75n/1?=
 =?us-ascii?Q?Ra2BSKQqPndkD1s8QFaGbB0HBDiK8LlovNZapk/Mf7i14ycOcoT2Xf6zClJT?=
 =?us-ascii?Q?7T99/5t3/W+fTOy7+y1s/s9CMRhPld4gNpE4nwDUWZYBod118l3bjI6w10i4?=
 =?us-ascii?Q?yegdzJKd09FKEj+wPoaMdJK3iRy7wdY1HmaMZgF6lmGEvxDva2juyQd6TQp6?=
 =?us-ascii?Q?VHCaQba5AuwX6u+gP7kTVjbAfttYoAelHyGWw4q+s9eobYXK+Idjugyq+3vB?=
 =?us-ascii?Q?qQk602CsS1KXHPQGA96210Q1Iu4kprBgn4rtLwcO9VBjOCSjYa/BpaUGZNDe?=
 =?us-ascii?Q?ZtD3ZQoKWj78CzX4jhCE8luVi+0LSsZoHjqYwfOeH137MXNLChO3fVthi/X6?=
 =?us-ascii?Q?Q4dBfGdkIiuAzSoa/Dp23focFA4ZwmINOtbL7NCgnQEB2fNBBb9+6BmReUbb?=
 =?us-ascii?Q?wItYnq8fQHeQQo++YLx7BYHz+RoryuWP3o6E1JyjctW6LyovJXo8orqbM5VZ?=
 =?us-ascii?Q?O2v7LUJeClLjd9PgTZ+I6SdPY0I4rc3R88uEhdsPAS5z43QYM6NpwSJN8U16?=
 =?us-ascii?Q?gdalS79d5JZycV+S52CPWVgUocgb1LdbTGOQhrxQZeHSgj4wKAAqRrqdoknk?=
 =?us-ascii?Q?0m5+KjbRkY+d4NzDyR03Nek1p3NIArh/xnIwDMC+L1PObB1dePvqi3dsnnbU?=
 =?us-ascii?Q?OPR4IxiHkaL38knCH9xzSh86eUZ83V0AEDs3QWJ+hoT8GletiEtIKtI6IWBi?=
 =?us-ascii?Q?xqsQzIJChqyprTu+vReAK9Z6yDEHEOsRHFEITPdQCgbOhy4rFjqoCHL2GpW6?=
 =?us-ascii?Q?uMvXl57+eYP5r3Z3Eog8qBjQmIpE8xmi2Vs/o8bCAccZnjDMDuJz+jKHB0W+?=
 =?us-ascii?Q?yKvIgG2lv6rHWlRfCrEXwf+ieRAFJ9DFQVaZHtkCX/PHAhaTwsKSCDtJTx+Q?=
 =?us-ascii?Q?kTT0mqUU9nX+cxEOoSXxqs64//2ce9LATBkV/XoEOrVeog=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <83CF8B41F3CAFE4088AFCE48D291926C@namprd04.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: wdc.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PH0PR04MB7158.namprd04.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 29eb25a2-b516-4911-a3fd-08d971ff561a
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Sep 2021 12:59:32.3099
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: ZJafDJtlZEaCVUSIUQuu+df4Dl7BsWgq+l0DdM3cvAx5g68nfFk6FTrRdpqL1eBdg7AAGRM7iMoLNe8buXL4Qw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR04MB7367
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, Aug 30, 2021 at 06:11:12PM +0800, Ming Lei wrote:
> On Mon, Aug 30, 2021 at 09:48:06AM +0000, Niklas Cassel wrote:
> > On Fri, Aug 27, 2021 at 09:28:07PM +0800, Ming Lei wrote:
> > > On Fri, Aug 27, 2021 at 12:41:31PM +0000, Niklas Cassel wrote:
> > > > From: Niklas Cassel <niklas.cassel@wdc.com>
> > > >=20
> > > > Currently, __blk_mq_alloc_request() calls ops.prepare_request and s=
ets
> > > > RQF_ELVPRIV.
> > > >=20
> > > > Therefore, (if the request is not a flush) the RQF_ELVPRIV flag wil=
l be
> > > > set for the request in blk_mq_submit_bio(), regardless if the reque=
st
> > > > was submitted to a scheduler, or bypassed the scheduler.
> > > >=20
> > > > Later, blk_mq_free_request() checks if the RQF_ELVPRIV flag is set,
> > > > if it is, the ops.finish_request callback will be called.
> > > >=20
> > > > The problem with this is that the finish_request scheduler callback
> > > > will be called for requests that bypassed the scheduler.
> > > >=20
> > > > Fix this by calling the scheduler ops.prepare_request callback, and
> > > > set the RQF_ELVPRIV flag only immediately before calling the insert
> > > > callback.
> > >=20
> > > One request could be inserted more than one times, such as requeue,
> > > however __blk_mq_alloc_request() is just run once, so is it fine to
> > > call ->prepare_request more than one time for same request?
> >=20
> > Calling ->prepare_request multiple times is fine.
> > All the different I/O schedulers (BFQ, mq-deadline, kyber)
> > simply use .prepare_request to clear/set elv->priv to a fixed value.
> >=20
> > >=20
> > > Or I am wondering why not call ->prepare_request when the following
> > > check is true?
> > >=20
> > > 	if (e && e->type->ops.prepare_request && !op_is_flush(data->cmd_flag=
s) &&
> > > 		!blk_op_is_passthrough(data->cmd_flags))
> > > 		e->type->ops.prepare_request()
> >=20
> >=20
> > That might work, and might be a nicer solution indeed.
> >=20
> > If a request got plugged, it will be inserted to the scheduler through
> > blk_flush_plug_list() -> blk_mq_flush_plug_list() -> blk_mq_sched_inser=
t_requests()
> > which will insert them unconditionally.
> > In this case. we know that !op_is_flush() (because if it was, blk_mq_su=
bmit_bio()
> > would have inserted directly.)
> >=20
> >=20
> > If we didn't plug, we do blk_mq_sched_insert_request(), which will add =
it if
> > blk_mq_sched_bypass_insert() returns false:
> >=20
> > blk_mq_sched_bypass_insert() is defined as:
> >=20
> >         if ((rq->rq_flags & RQF_FLUSH_SEQ) || blk_rq_is_passthrough(rq)=
)
> >                 return true;
> > Also in this case. we know that !op_is_flush() (blk_mq_submit_bio() wou=
ld have
> > inserted directly.)
> >=20
> >=20
> > So, we could easily add && !blk_op_is_passthrough(data->cmd_flags) to t=
he
> > ->prepare_request condition in blk_mq_rq_ctx_init() like you suggested,
> > but since the bypass condition also seems to look at RQF_FLUSH_SEQ, wou=
ldn't
> > we need to add RQF_FLUSH_SEQ to the condition in blk_mq_rq_ctx_init() a=
s well?
> >=20
> > This flag is set after blk_mq_rq_ctx_init(). Are we sure that RQF_FLUSH=
_SEQ
> > flag will only be set for a request which op_is_flush() returned true?
> >=20
> > (If so, then only adding  && !blk_op_is_passthrough(data->cmd_flags) sh=
ould
> > be fine.)
>=20
> BTW, what I meant is the following change, is it fine?
>=20
> diff --git a/block/blk-mq.c b/block/blk-mq.c
> index 0a33d16a7298..f98f8cc05644 100644
> --- a/block/blk-mq.c
> +++ b/block/blk-mq.c
> @@ -327,20 +327,6 @@ static struct request *blk_mq_rq_ctx_init(struct blk=
_mq_alloc_data *data,
> =20
>  	data->ctx->rq_dispatched[op_is_sync(data->cmd_flags)]++;
>  	refcount_set(&rq->ref, 1);
> -
> -	if (!op_is_flush(data->cmd_flags)) {
> -		struct elevator_queue *e =3D data->q->elevator;
> -
> -		rq->elv.icq =3D NULL;
> -		if (e && e->type->ops.prepare_request) {
> -			if (e->type->icq_cache)
> -				blk_mq_sched_assign_ioc(rq);
> -
> -			e->type->ops.prepare_request(rq);
> -			rq->rq_flags |=3D RQF_ELVPRIV;
> -		}
> -	}
> -
>  	data->hctx->queued++;
>  	return rq;
>  }
> @@ -359,17 +345,25 @@ static struct request *__blk_mq_alloc_request(struc=
t blk_mq_alloc_data *data)
>  	if (data->cmd_flags & REQ_NOWAIT)
>  		data->flags |=3D BLK_MQ_REQ_NOWAIT;
> =20
> -	if (e) {
> +	if (e && !op_is_flush(data->cmd_flags) &&
> +			!blk_op_is_passthrough(data->cmd_flags)) {
>  		/*
>  		 * Flush/passthrough requests are special and go directly to the
>  		 * dispatch list. Don't include reserved tags in the
>  		 * limiting, as it isn't useful.
>  		 */
> -		if (!op_is_flush(data->cmd_flags) &&
> -		    !blk_op_is_passthrough(data->cmd_flags) &&
> -		    e->type->ops.limit_depth &&
> -		    !(data->flags & BLK_MQ_REQ_RESERVED))
> +		if (e->type->ops.limit_depth &&
> +			    !(data->flags & BLK_MQ_REQ_RESERVED))
>  			e->type->ops.limit_depth(data->cmd_flags, data);
> +
> +		rq->elv.icq =3D NULL;
> +		if (e->type->ops.prepare_request) {
> +			if (e->type->icq_cache)
> +				blk_mq_sched_assign_ioc(rq);
> +
> +			e->type->ops.prepare_request(rq);
> +			rq->rq_flags |=3D RQF_ELVPRIV;
> +		}
>  	}
> =20
>  retry:
>=20

Hello Ming,


Sorry for the delayed reply.

Your patch does not compile, because rq is not defined in this function.

Another problem seems to be that in __blk_mq_alloc_request(), at the end
of the function, calls blk_mq_rq_ctx_init(), which will unconditionally
set rq->rq_flags =3D 0;


The simple patch:

--- a/block/blk-mq.c
+++ b/block/blk-mq.c
@@ -328,7 +328,8 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq=
_alloc_data *data,
        data->ctx->rq_dispatched[op_is_sync(data->cmd_flags)]++;
        refcount_set(&rq->ref, 1);
=20
-       if (!op_is_flush(data->cmd_flags)) {
+       if (!op_is_flush(data->cmd_flags) &&
+           !blk_op_is_passthrough(data->cmd_flags)) {
                struct elevator_queue *e =3D data->q->elevator;
=20
                rq->elv.icq =3D NULL;


Does appear to solve the problem.

My only worry was RQF_FLUSH_SEQ flag, but as far as I can tell, it is
only ever set for a request that which op_is_flush() returned true.


Kind regards,
Niklas=