Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752598AbcCVVfr (ORCPT ); Tue, 22 Mar 2016 17:35:47 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:13070 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752502AbcCVVfl (ORCPT ); Tue, 22 Mar 2016 17:35:41 -0400 Subject: Re: [PATCH 6/6] writeback: throttle buffered writeback To: Shaohua Li References: <1458669320-6819-1-git-send-email-axboe@fb.com> <1458669320-6819-7-git-send-email-axboe@fb.com> <56F1A8D0.4060403@fb.com> <20160322213000.GA45596@kernel.org> CC: Jeff Moyer , , , From: Jens Axboe Message-ID: <56F1BA9D.7020209@fb.com> Date: Tue, 22 Mar 2016 15:35:25 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101 Thunderbird/38.6.0 MIME-Version: 1.0 In-Reply-To: <20160322213000.GA45596@kernel.org> Content-Type: text/plain; charset="windows-1252"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [192.168.54.13] X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2016-03-22_09:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1822 Lines: 42 On 03/22/2016 03:30 PM, Shaohua Li wrote: > On Tue, Mar 22, 2016 at 02:19:28PM -0600, Jens Axboe wrote: >> On 03/22/2016 02:12 PM, Jeff Moyer wrote: >>> Hi, Jens, >>> >>> Jens Axboe writes: >>> >>>> If the device has write back caching, 'wb_cache_delay' delays by >>>> this amount of usecs when a write completes before allowing more. >>> >>> What's the reason behind that? >> >> For classic write back caching, the cache can absorb a bunch of writes >> shortly, which means that the completion cost only shows a small part of the >> overall cost. This means that if we just throttle on completion, then when >> the device starts committing to media, then we'll end up starving other IO >> anyway. This knob is a way to attempt to tame that. > > Does request size matter? I think it's yes. If request size will be accounted, > there will be issue how to evaluate IO cost of each request, which is hard. The code currently deliberately ignores it, since we do the throttling checks post merging. We can experiment with doing it on a per-request basis. I didn't want to complicate it too much, in my testing, for this sort of application, the size of the request doesn't matter too much. That's mainly because we, by default, bound the size. If it was unbounded, then that would be different. > Looks the throttling is done regardless if there is other IO running, which > could hurt writeback. I wanted to make the first cut very tough on the writes. We always want to throttle, but perhaps not as much as we do now. But you'd be surprised how close this basic low depth gets to ideal performance, on most devices! Background writeback does not have to be at 100% or 99% of the device capability. If we sync or wait on it, then yes, we want it to go really fast. And it should. -- Jens Axboe