Received: by 10.223.148.5 with SMTP id 5csp6979612wrq; Wed, 17 Jan 2018 22:40:10 -0800 (PST) X-Google-Smtp-Source: ACJfBot3NurLo5j/s4whGj9oku2r+1srKAVj9NGn8Sh5XKGLAeO6RSPOnqL362Y8F/B9wlllYV4c X-Received: by 10.98.18.150 with SMTP id 22mr35808588pfs.180.1516257610352; Wed, 17 Jan 2018 22:40:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1516257610; cv=none; d=google.com; s=arc-20160816; b=cGdWcMzvLo2Mc3BnOoRieVXLoMW/Py9ch601yqpiy2mQ0TssH5i6g7ACgtsOq9bvEB 5DXnC0uikE0junLXymvWZKuG0lY7xcb27IwFL19wbdMdZDvQBfXTgjT8jntRRLvB6Gw0 6P1W4lb06X5fOgD1QFcIrck1ZRCk35zf3OMx5HBjt5RD7AvedJR71X08u8z/q2uSqoy2 UGp6lksHr9BPPjxiX18nUefLVRxzuODPuaV4mXpN0KLf0qEstRJgMM5VyIlAHdtwQ0oi 8PRKadg+GdJEPgMZ7LZpScRLUN1qzHUbSpbZ1Ix9NyI+OmGSe8bJyWrAmbCGi/JZznvw mo1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:date:cc:to:from:subject:message-id :arc-authentication-results; bh=TF4z8RdOFgI/3IBpZ7fIOGqEB7ZKtxBevGFNTKPji04=; b=fTimx0yTHrK6YXlOUEm9J5pA6YfaN32I3CkVGAbhLYZMvFQCePO/aIBUdiSXT5aT5L 6xyrMxxkC5UcG04DLcoymXOBKPc4XTpNxjyIM4mQQjz9+nJVpprgZ6zmzPQzjSg9JXG2 dtCUAKSyNk/rJ7DF+QRGS3DUyAEMMsrW39Aq4SV6Ge3Mo/kvOjOvO9wYRRf40g7dV6vc BOlIivZjDlrGQ0Se3XoNGvPDKPmCpn8H+azDtvClXTWbQM0ethN0w56LKwafrYk0tyxq jKjBPTca1bST/xKV2Rxwc+EZIr/7BgdGjUTtAmlWeOnuHYZ8h5LcDTLFSS28YmNtZ0gp DYmg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a70si5364631pge.448.2018.01.17.22.39.56; Wed, 17 Jan 2018 22:40:10 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754678AbeARGja (ORCPT + 99 others); Thu, 18 Jan 2018 01:39:30 -0500 Received: from mail-qt0-f196.google.com ([209.85.216.196]:43037 "EHLO mail-qt0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754644AbeARGj2 (ORCPT ); Thu, 18 Jan 2018 01:39:28 -0500 Received: by mail-qt0-f196.google.com with SMTP id s3so27763086qtb.10 for ; Wed, 17 Jan 2018 22:39:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:mime-version:content-transfer-encoding; bh=TF4z8RdOFgI/3IBpZ7fIOGqEB7ZKtxBevGFNTKPji04=; b=FdU+fMKIRoc67fH5Thj7cObezjd/ZHLxIR5PIP2YumgVtpIucpAth1Irxa4WPF8OM9 f0CW68DZQP7yAsYb/bpTsbvdNw10RjIZoysHLU9vN3AMi0DPdckhm4rTb1zsWLEc+b+K IiTtcQiMk6ZzKHr6pJHDQ2BuJWYqMJu4qfmCZJmDtxcWGwmBnrAIPKz0vo9233UpmWtO q8CGpWBZJjOXn4eybjX40kxsQ+NM99vskgPnW8ecF4OjTJYCaKKWHvUVqHFAnJ7GxJ16 SS3R1GAgcnaY9h75w4DMbn9o6S5jGWSp+ypIWHktlHIadaSOP3CWm65LDfKksInDGoUg /oOA== X-Gm-Message-State: AKwxytd6KYhBiqCGTlKFTpayrDSHCODG2rPy9Ncws6Xe0TpVP6ynjGKq SOYyB8u/xTAM2mE0Jn4KVjQKPVbZ1qw= X-Received: by 10.55.171.10 with SMTP id u10mr22979300qke.31.1516257567457; Wed, 17 Jan 2018 22:39:27 -0800 (PST) Received: from loberhel74 (71-88-101-235.dhcp.oxfr.ma.charter.com. [71.88.101.235]) by smtp.gmail.com with ESMTPSA id r53sm4366875qtr.57.2018.01.17.22.39.26 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 17 Jan 2018 22:39:27 -0800 (PST) Message-ID: <1516257566.11458.1.camel@redhat.com> Subject: Re: blk-mq: don't dispatch request in blk_mq_request_direct_issue if queue is busy From: Laurence Oberman To: Mike Snitzer , Ming Lei Cc: Jens Axboe , linux-block@vger.kernel.org, dm-devel@redhat.com, Christoph Hellwig , Bart Van Assche , linux-kernel@vger.kernel.org Date: Thu, 18 Jan 2018 01:39:26 -0500 In-Reply-To: <20180118043608.GA8809@redhat.com> References: <20180118040659.20202-1-ming.lei@redhat.com> <20180118043608.GA8809@redhat.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.22.6 (3.22.6-10.el7) Mime-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 2018-01-17 at 23:36 -0500, Mike Snitzer wrote: > On Wed, Jan 17 2018 at 11:06pm -0500, > Ming Lei wrote: > > > If we run into blk_mq_request_direct_issue(), when queue is busy, > > we > > don't want to dispatch this request into hctx->dispatch_list, and > > what we need to do is to return the queue busy info to caller, so > > that caller can deal with it well. > > > > Fixes: 396eaf21ee ("blk-mq: improve DM's blk-mq IO merging via > > blk_insert_cloned_request feedback") > > Reported-by: Laurence Oberman > > Reviewed-by: Mike Snitzer > > Signed-off-by: Ming Lei > > --- > >  block/blk-mq.c | 22 ++++++++++------------ > >  1 file changed, 10 insertions(+), 12 deletions(-) > > > > diff --git a/block/blk-mq.c b/block/blk-mq.c > > index 4d4af8d712da..1af7fa70993b 100644 > > --- a/block/blk-mq.c > > +++ b/block/blk-mq.c > > @@ -1856,15 +1856,6 @@ static blk_status_t > > __blk_mq_issue_directly(struct blk_mq_hw_ctx *hctx, > >   return ret; > >  } > >   > > -static void __blk_mq_fallback_to_insert(struct request *rq, > > - bool run_queue, bool > > bypass_insert) > > -{ > > - if (!bypass_insert) > > - blk_mq_sched_insert_request(rq, false, run_queue, > > false); > > - else > > - blk_mq_request_bypass_insert(rq, run_queue); > > -} > > - > >  static blk_status_t __blk_mq_try_issue_directly(struct > > blk_mq_hw_ctx *hctx, > >   struct request > > *rq, > >   blk_qc_t *cookie, > > @@ -1873,9 +1864,16 @@ static blk_status_t > > __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, > >   struct request_queue *q = rq->q; > >   bool run_queue = true; > >   > > - /* RCU or SRCU read lock is needed before checking > > quiesced flag */ > > + /* > > +  * RCU or SRCU read lock is needed before checking > > quiesced flag. > > +  * > > +  * When queue is stopped or quiesced, ignore > > 'bypass_insert' from > > +  * blk_mq_request_direct_issue(), and return BLK_STS_OK to > > caller, > > +  * and avoid driver to try to dispatch again. > > +  */ > >   if (blk_mq_hctx_stopped(hctx) || blk_queue_quiesced(q)) { > >   run_queue = false; > > + bypass_insert = false; > >   goto insert; > >   } > >   > > @@ -1892,10 +1890,10 @@ static blk_status_t > > __blk_mq_try_issue_directly(struct blk_mq_hw_ctx *hctx, > >   > >   return __blk_mq_issue_directly(hctx, rq, cookie); > >  insert: > > - __blk_mq_fallback_to_insert(rq, run_queue, bypass_insert); > >   if (bypass_insert) > >   return BLK_STS_RESOURCE; > >   > > + blk_mq_sched_insert_request(rq, false, run_queue, false); > >   return BLK_STS_OK; > >  } > > OK so you're just leveraging blk_mq_sched_insert_request()'s > ability to resort to__blk_mq_insert_request() if !q->elevator. I tested this against Mike's latest combined tree and its stable. This fixes the list corruption issue. Many Thanks Ming and Mike. I will apply it to Bart's latest SRP/SRPT tree tomorrow as its very late here but it will clearly fix the issue in Bart's tree too. Tested-by: Laurence Oberman