Received: by 2002:ac0:a5b6:0:0:0:0:0 with SMTP id m51-v6csp4189098imm; Wed, 30 May 2018 00:26:21 -0700 (PDT) X-Google-Smtp-Source: ADUXVKKcG5O7MWqxV9GJrUrMSR5zJEVtvx3zHVyf/6ph6YS24VpRgdNYWP6Hxt6P5Ihv9fG+V2Ir X-Received: by 2002:a65:51c4:: with SMTP id i4-v6mr1313257pgq.190.1527665181516; Wed, 30 May 2018 00:26:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527665181; cv=none; d=google.com; s=arc-20160816; b=zFeArqg+O9Ap7AUXLLr9cB3jQiQlHc+CsAZJJL8tDUzz43nPZw52op7d3oCibF9f8h 8/xaKEJEKE4AzUQGtokd+/yOvD7BlPuIaGbIDhof73KfDtmXZRgoHtBH+8U5zZSN7eS9 FEEM62imOTDSp7F7cB0jmExbXSz4BWTq5jfCX20VSoMxMGwxOHEaDEFN0DLazxDXsLCf IE/8l/U6KxOZ1XTntB0SWI1nc0uGiAur+Vvv9xQ87nPEVDhE6VHHYn0Flognr+NG+3eC xccdXVp8y136UqZnqOiO9rIh3Hrem1Xf9Ct3L2LFgIq2wxe33ZCla6U2eMMog6uEZHgK JDbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=gd+4bFQny7axR6Ypbv14jnvmY6YqrsNI4/ka3EcHw1c=; b=J7sWIek/0qUia8zaWP4QgFKlCUnAljqr0uw1ADvhszwusFZ+xaqFMO0XEyFFimEw0s v82whIspFRrkW4DNSv8PMnQKyCA+rgdfep7nWLXvXYeuhCwtAMl//eXL2ur9a8bgv6HL u+m2Eo/k6mRDkHKNHlihLAGQKyXtEmyHEpBTey7ZzGKFklGfMZIroRxv5/Q5AYMYmP+6 7MGZ325ZXq0w6+1uVZJrko4si8EugMIKff+W+eB+/xTYJK8HIpibUwFvg9tzYhI3fFwD I8sHiU8+94faSTRAre6UBovR1inZ95KD5j8/s432Rrv28p2RciOl07QAomLCYvTNR9ME 4svg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2017-10-26 header.b=JGctezyT; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e124-v6si33438224pfe.80.2018.05.30.00.26.07; Wed, 30 May 2018 00:26:21 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@oracle.com header.s=corp-2017-10-26 header.b=JGctezyT; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=oracle.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S937141AbeE3HZZ (ORCPT + 99 others); Wed, 30 May 2018 03:25:25 -0400 Received: from userp2120.oracle.com ([156.151.31.85]:54990 "EHLO userp2120.oracle.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S935032AbeE3HZT (ORCPT ); Wed, 30 May 2018 03:25:19 -0400 Received: from pps.filterd (userp2120.oracle.com [127.0.0.1]) by userp2120.oracle.com (8.16.0.22/8.16.0.22) with SMTP id w4U7LdK7065199; Wed, 30 May 2018 07:25:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id; s=corp-2017-10-26; bh=gd+4bFQny7axR6Ypbv14jnvmY6YqrsNI4/ka3EcHw1c=; b=JGctezyTOxp+VDeRjta/BOBoFoLQBz36gQKcLnBw7TumnzmiTYgTv79Di5tM+5/gquys 3boLA4ZpF4eZnG+xmPrKCAw/t66fLuEFItqWVUtBk88TzLS3LztcuWKi7pOsXCJFbCHO lT7ZLZz+lHIIDstgZ27K+KuPSaOrVbwBtvbVpKSxpShxHwycL6e6KtPyYdPKshsyg2BJ zMzCSBt3JXUUyW59eof3u7eGBnibSaBszryapacm4ThPJ9PmQq/kMNM/KdW49upLSKyh DThMD2kZOzZhRL1oSrNWfbR1GS16DpIwxZN5RQcek7/L4JJiegWWr+KX25iQlzwHrWgS LQ== Received: from userv0022.oracle.com (userv0022.oracle.com [156.151.31.74]) by userp2120.oracle.com with ESMTP id 2j9ev813d9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 May 2018 07:25:13 +0000 Received: from aserv0121.oracle.com (aserv0121.oracle.com [141.146.126.235]) by userv0022.oracle.com (8.14.4/8.14.4) with ESMTP id w4U7PDmA013830 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 30 May 2018 07:25:13 GMT Received: from abhmp0015.oracle.com (abhmp0015.oracle.com [141.146.116.21]) by aserv0121.oracle.com (8.14.4/8.13.8) with ESMTP id w4U7PCUK004363; Wed, 30 May 2018 07:25:12 GMT Received: from will-ThinkCentre-M910s.cn.oracle.com (/10.182.70.254) by default (Oracle Beehive Gateway v4.0) with ESMTP ; Wed, 30 May 2018 00:25:11 -0700 From: Jianchao Wang To: axboe@kernel.dk, osandov@osandov.com Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 0/2] block: kyber: make kyber more friendly with merging Date: Wed, 30 May 2018 15:26:06 +0800 Message-Id: <1527665168-1965-1-git-send-email-jianchao.w.wang@oracle.com> X-Mailer: git-send-email 2.7.4 X-Proofpoint-Virus-Version: vendor=nai engine=5900 definitions=8908 signatures=668702 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=951 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1805220000 definitions=main-1805300085 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Jens This is the 3rd version patchset to make the kyber io scheduler more firendly with merging. Most of time, kyber io scheduler will not leave any requests in ctx rq_list, this is because even if tokens of one domain is used up, kyber will try to dispatch requests from other domain and flush the rq_list there. So there is not any change to do merge. To improve this, setup kyber_ctx_queue (kcq) which is similar with blk_mq_ctx but has rq_list for every domain, and build mapping between kcq and khd as ctx and hctx. Then we could merge, insert and dispatch on rq_list of different domains separately. At the same time, only flush the rq_list of kcq after get domain token successfully, then the requests could be left in the rq_list of that domain and maybe merged with following io. In my local test on NVMe card, the sequential io performance could be improved a lot, especially on high workload. More details please refer to the second patch. Changes V2 to V3: - discard kcq->index and use request->mq_ctx->index_hw - changes based on Omar's suggestion - some other changes to fix warning of checkpatch.pl with --strict option Changes V1 to V2: - Add patch from Jens which abstract out the blk-mq-sched rq list iteration bio merge helper interface and rebase the patch based on it. - merge bio_sched_domain and rq_sched_domain to avoid the duplicated code. - allocate kcqs per khd Jens Axboe (1) 0001-blk-mq-abstract-out-blk-mq-sched-rq-list-iteration-b.patch Jianchao Wang (1) 0002-block-kyber-make-kyber-more-friendly-with-merging.patch block/blk-mq-sched.c | 34 ++++++++++++------ block/kyber-iosched.c | 190 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------- include/linux/blk-mq.h | 3 +- 3 files changed, 184 insertions(+), 43 deletions(-) Thanks Jianchao