Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Mon, 12 Apr 2021 11:23:25 +0800
From:   Ming Lei <ming.lei@redhat.com>
To:     Damien Le Moal <Damien.LeMoal@wdc.com>
Cc:     Bart Van Assche <bvanassche@acm.org>,
        Changheun Lee <nanich.lee@samsung.com>,
        "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        Johannes Thumshirn <Johannes.Thumshirn@wdc.com>,
        "asml.silence@gmail.com" <asml.silence@gmail.com>,
        "axboe@kernel.dk" <axboe@kernel.dk>,
        "hch@infradead.org" <hch@infradead.org>,
        "jisoo2146.oh@samsung.com" <jisoo2146.oh@samsung.com>,
        "junho89.kim@samsung.com" <junho89.kim@samsung.com>,
        "linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "mj0123.lee@samsung.com" <mj0123.lee@samsung.com>,
        "osandov@fb.com" <osandov@fb.com>,
        "patchwork-bot@kernel.org" <patchwork-bot@kernel.org>,
        "seunghwan.hyun@samsung.com" <seunghwan.hyun@samsung.com>,
        "sookwan7.kim@samsung.com" <sookwan7.kim@samsung.com>,
        "tj@kernel.org" <tj@kernel.org>,
        "tom.leiming@gmail.com" <tom.leiming@gmail.com>,
        "woosung2.lee@samsung.com" <woosung2.lee@samsung.com>,
        "yt0928.kim@samsung.com" <yt0928.kim@samsung.com>
Subject: Re: [RESEND,v5,1/2] bio: limit bio max size
Message-ID: <YHO9LQUt1e0J6+l9@T590>
References: <YG1iC944hUkBniDM@kroah.com>
 <CGME20210407094610epcas1p472207e8d3ca0e5e697974c993a2a34f7@epcas1p4.samsung.com>
 <20210407092836.13016-1-nanich.lee@samsung.com>
 <BL0PR04MB65146F70831B61CB6B9AFE7CE7759@BL0PR04MB6514.namprd04.prod.outlook.com>
 <cdd26ed3-2013-7edd-b25e-557a56397614@acm.org>
 <BL0PR04MB65148B0629FF23847341BAE8E7719@BL0PR04MB6514.namprd04.prod.outlook.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BL0PR04MB65148B0629FF23847341BAE8E7719@BL0PR04MB6514.namprd04.prod.outlook.com>
Precedence: bulk

On Sun, Apr 11, 2021 at 10:13:01PM +0000, Damien Le Moal wrote:
> On 2021/04/09 23:47, Bart Van Assche wrote:
> > On 4/7/21 3:27 AM, Damien Le Moal wrote:
> >> On 2021/04/07 18:46, Changheun Lee wrote:
> >>> I'll prepare new patch as you recommand. It will be added setting of
> >>> limit_bio_size automatically when queue max sectors is determined.
> >>
> >> Please do that in the driver for the HW that benefits from it. Do not do this
> >> for all block devices.
> > 
> > Hmm ... is it ever useful to build a bio with a size that exceeds 
> > max_hw_sectors when submitting a bio directly to a block device, or in 
> > other words, if no stacked block driver sits between the submitter and 
> > the block device? Am I perhaps missing something?
> 
> Device performance wise, the benefits are certainly not obvious to me either.
> But for very fast block devices, I think the CPU overhead of building more
> smaller BIOs may be significant compared to splitting a large BIO into multiple
> requests. Though it may be good to revisit this with some benchmark numbers.

This patch tries to address issue[1] in do_direct_IO() in which
Changheun observed that other operations takes time between adding page
to bio.

However, do_direct_IO() just does following except for adding bio and
submitting bio:

- retrieves pages at batch(pin 64 pages each time from VM) and 

- retrieve block mapping(get_more_blocks), which is still done usually
very less times for 32MB; for new mapping, clean_bdev_aliases() may
take a bit time.

If there isn't system memory pressure, pin 64 pages won't be slow, but
get_more_blocks() may take a bit time.

Changheun, can you check if multiple get_more_blocks() is called for submitting
32MB in your test?

In my 32MB sync dio f2fs test on x86_64 VM, one buffer_head mapping can
hold 32MB, but it is one freshly new f2fs.

I'd suggest to understand the issue completely before figuring out one
solution.


[1] https://lore.kernel.org/linux-block/20210202041204.28995-1-nanich.lee@samsung.com/


Thanks,
Ming