Received: by 2002:a05:6a10:c7d3:0:0:0:0 with SMTP id h19csp623696pxy; Sat, 14 Aug 2021 18:15:31 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyLjDMI1+lJLceIfTwOeUquXYcIGrv/k7lkvmdk9uZ9bD7vCB8iyZDyJIEfvFCzbmFFe995 X-Received: by 2002:a92:c212:: with SMTP id j18mr6488764ilo.77.1628990131087; Sat, 14 Aug 2021 18:15:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1628990131; cv=none; d=google.com; s=arc-20160816; b=yZ65g0n8oFE0dlWX3VLXpfyL5t0QfwX00kHWxi2grE2YloqScUlej7XFxSZa3/qHq+ ESQZ/BIkn/F9AGxGVCsUdEgzg9Ekt4f6aG63bcURK5LmEvsjQi7mXN36ySRBQ/eqjRcs c206huUwyAWHIYAmZfA2DnIhIorISH08sRUCoVGNYHE+NPCiw7npFLgU5EnCKRywOMhh K/oLyBly8eue6YRBlo3EBeKppzt8+60gri1mIjJjpo3f0/sinjdQfFKudgR81PH776EL 9Bmq5X6VyoAHtL18f6aaxaGPu+vSJmp+43OZoUOcdc+f9LBtUNMeptj6NrEEhK57FFz7 FQhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:content-language :in-reply-to:mime-version:user-agent:date:message-id:from:references :cc:to:subject:dkim-signature; bh=x2b3137Hg3O35wl4KvAg3fkN/4EPjZhVIIFocKDpOzw=; b=pJ7mRo9l0/9Cha1zGVxOqhlVNmP5BnagpY718o+WOYTUlAADEAFHVfj6ivgZjwVfPW WizS7vY5SOtzWIDAkl/X2jM0jnjCpIRXUYYlErvAAOuzMim6KQQi7mPWR1rq89wwo2Gq vzyr1XZPYFLwdwtu73HWXTy4IuUXEYpcPfA13Jfc9Am53r5RdBJ+Q02ShHOZHIEid8jT BKheUXLJeAPIu3XzT9NyGHG45twsMlxnKQGpHTzp2MGoZmiv5BXY1NdDIVr7yB5KZJZT fX1frSzu7nP18VO/KpiIXsn55l+59bHh/Oxinv3zQ552jVfsXGG080RuKF4WS1eQMdNL B6Ew== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel-dk.20150623.gappssmtp.com header.s=20150623 header.b=JNP1gkSD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id d20si6569992ioo.61.2021.08.14.18.15.19; Sat, 14 Aug 2021 18:15:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel-dk.20150623.gappssmtp.com header.s=20150623 header.b=JNP1gkSD; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233237AbhHOBPL (ORCPT + 99 others); Sat, 14 Aug 2021 21:15:11 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49812 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229453AbhHOBPK (ORCPT ); Sat, 14 Aug 2021 21:15:10 -0400 Received: from mail-io1-xd36.google.com (mail-io1-xd36.google.com [IPv6:2607:f8b0:4864:20::d36]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 532C4C061764 for ; Sat, 14 Aug 2021 18:14:41 -0700 (PDT) Received: by mail-io1-xd36.google.com with SMTP id e186so18322842iof.12 for ; Sat, 14 Aug 2021 18:14:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=x2b3137Hg3O35wl4KvAg3fkN/4EPjZhVIIFocKDpOzw=; b=JNP1gkSDmAye92b824CW/SUEX+EdwwaqZtDqtnkJAUmkDBHjTHWSIsBYNlIXQxvqR1 XYt2ztX4SxZcsvu1LwvlEZKlL0f84LHRTyNUYRg5hfqTLTt8MDuCIoRdZ/diOGS2zvrV vcX4KXpOItr26LiTEAMxsjoDzStZ1sE+BvhUhEblVe92CKn7WnJFf1VDJc8QaH9DMnkc wMNYXBeQeE020Q2qe+xZXJYzhou1sIJ2mrWoXjqNSXTDoxtKOhDtH4UmPW98ZFjTbvOS L/KLNSVZ72TBI3xSYdZnf3/IcyWwpZeIxzRFXyPSXOHUbEDE5mLyKwm3Gixuy2HUgfVb k14w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=x2b3137Hg3O35wl4KvAg3fkN/4EPjZhVIIFocKDpOzw=; b=sgUXW7NinsDTsrLbxFXWTTDgj0VaDaBrXWX3NFCly4zS3QcFraCPxINUCazOaE2XxK mXwS3BIiBYBUVh+Kte1H7F3fRapDNNJUr3yJQyuJqIws5H8w8JpJp41iqYvC22++JOSr VX+NPTqvxs2AjHGpZNWvXrUo3ZbzXGBvqiz766JGHL9YT++EmckK08pwbE7j2h/B71Ku RGxBDkUS3M8y14ywIBmO2x27OZJ8fMBAVJI5UQOpZmBXqOBjt/N4kZv+P8oqzM7k4pHC XO18D1WWLUqOALaZyMRKZJ9Uge8bCrRXb6bIQGtkHiEYyHBYQe7I77JFzi0WazXfI+ro a4Tg== X-Gm-Message-State: AOAM531DQy13sXF6v8BBfcBzJmATo6ErVwIPHulQ6eVhD39d3NrRzUAA ECYeaw3tGZtLRRH7VtCdIRi7iw== X-Received: by 2002:a02:cc22:: with SMTP id o2mr8769511jap.26.1628990080703; Sat, 14 Aug 2021 18:14:40 -0700 (PDT) Received: from [192.168.1.116] ([66.219.217.159]) by smtp.gmail.com with ESMTPSA id u14sm2730446iol.24.2021.08.14.18.14.39 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 14 Aug 2021 18:14:39 -0700 (PDT) Subject: Re: [PATCH v3] blk-throtl: optimize IOPS throttle for large IO scenarios To: brookxu , tj@kernel.org Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org References: <65869aaad05475797d63b4c3fed4f529febe3c26.1627876014.git.brookxu@tencent.com> From: Jens Axboe Message-ID: <53196a1f-9fdc-269d-801b-ce1ff2963cb0@kernel.dk> Date: Sat, 14 Aug 2021 19:14:37 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <65869aaad05475797d63b4c3fed4f529febe3c26.1627876014.git.brookxu@tencent.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 8/1/21 9:51 PM, brookxu wrote: > From: Chunguang Xu > > After patch 54efd50 (block: make generic_make_request handle > arbitrarily sized bios), the IO through io-throttle may be larger, > and these IOs may be further split into more small IOs. However, > IOPS throttle does not seem to be aware of this change, which > makes the calculation of IOPS of large IOs incomplete, resulting > in disk-side IOPS that does not meet expectations. Maybe we should > fix this problem. > > We can reproduce it by set max_sectors_kb of disk to 128, set > blkio.write_iops_throttle to 100, run a dd instance inside blkio > and use iostat to watch IOPS: > > dd if=/dev/zero of=/dev/sdb bs=1M count=1000 oflag=direct > > As a result, without this change the average IOPS is 1995, with > this change the IOPS is 98. Applied for 5.15, thanks. > v3: Optimize the use of atomic variables. > v2: Use atomic variables to solve synchronization problems. Just a note for the future, changelog stuff goes below the --- line. -- Jens Axboe