Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932169AbcDNMcz (ORCPT ); Thu, 14 Apr 2016 08:32:55 -0400 Received: from mail-lf0-f53.google.com ([209.85.215.53]:34116 "EHLO mail-lf0-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755107AbcDNMcv (ORCPT ); Thu, 14 Apr 2016 08:32:51 -0400 MIME-Version: 1.0 In-Reply-To: <1460029691-7550-1-git-send-email-ashishsangwan2@gmail.com> References: <1460029691-7550-1-git-send-email-ashishsangwan2@gmail.com> Date: Thu, 14 Apr 2016 18:02:44 +0530 Message-ID: Subject: Re: [PATCH] FUSE: Improve aio directIO write performance for size extending writes. From: Ashish Sangwan To: Miklos Szeredi Cc: fuse-devel , linux-fsdevel , LKML , Ashish Sangwan Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3869 Lines: 96 *ping* Last time it bounced off the fuse mailing list. On Thu, Apr 7, 2016 at 5:18 PM, Ashish Sangwan wrote: > While sending the blocking directIO in fuse, the write request is broken > into sub-requests, each of default size 128k and all the requests are sent > in non-blocking background mode if async_dio mode is supported by libfuse. > The process which issue the write wait for the completion of all the > sub-requests. Sending multiple requests parallely gives a chance to perform > parallel writes in the user space fuse implementation if it is > multi-threaded and hence improves the performance. > > When there is a size extending aio dio write, we switch to > blocking mode so that we can properly update the size of the file after > completion of the writes. However, in this situation all the sub-requests > are sent in serialized manner where the next request is sent only after > receiving the reply of the current request. Hence the multi-threaded user > space implementation is not utilized properly. > > This patch changes the size extending aio dio behavior to exactly follow > blocking dio. For multi threaded fuse implementation having 10 threads and > using buffer size of 64MB to perform async directIO, we are getting double > the speed. > > Signed-off-by: Ashish Sangwan > --- > fs/fuse/file.c | 16 ++++++++-------- > fs/fuse/fuse_i.h | 1 + > 2 files changed, 9 insertions(+), 8 deletions(-) > > diff --git a/fs/fuse/file.c b/fs/fuse/file.c > index 9dde38f..b4f8b83 100644 > --- a/fs/fuse/file.c > +++ b/fs/fuse/file.c > @@ -572,11 +572,11 @@ static void fuse_aio_complete(struct fuse_io_priv *io, int err, ssize_t pos) > io->bytes = pos; > > left = --io->reqs; > - if (!left && is_sync) > + if (!left && (is_sync || io->blocking_aio)) > complete(io->done); > spin_unlock(&io->lock); > > - if (!left && !is_sync) { > + if (!left && !is_sync && !io->blocking_aio) { > ssize_t res = fuse_get_res_by_io(io); > > if (res >= 0) { > @@ -2884,17 +2884,17 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter, loff_t offset) > */ > io->async = async_dio; > io->iocb = iocb; > + io->blocking_aio = false; > > /* > - * We cannot asynchronously extend the size of a file. We have no method > - * to wait on real async I/O requests, so we must submit this request > - * synchronously. > + * We cannot asynchronously extend the size of a file. > + * In such case the aio will behave exactly like sync io. > */ > if (!is_sync && (offset + count > i_size) && > iov_iter_rw(iter) == WRITE) > - io->async = false; > + io->blocking_aio = true; > > - if (io->async && is_sync) { > + if (io->async && (is_sync | io->blocking_aio)) { > /* > * Additional reference to keep io around after > * calling fuse_aio_complete() > @@ -2914,7 +2914,7 @@ fuse_direct_IO(struct kiocb *iocb, struct iov_iter *iter, loff_t offset) > fuse_aio_complete(io, ret < 0 ? ret : 0, -1); > > /* we have a non-extending, async request, so return */ > - if (!is_sync) > + if (!is_sync && !io->blocking_aio) > return -EIOCBQUEUED; > > wait_for_completion(&wait); > diff --git a/fs/fuse/fuse_i.h b/fs/fuse/fuse_i.h > index eddbe02..a7cf03f 100644 > --- a/fs/fuse/fuse_i.h > +++ b/fs/fuse/fuse_i.h > @@ -256,6 +256,7 @@ struct fuse_io_priv { > struct kiocb *iocb; > struct file *file; > struct completion *done; > + bool blocking_aio; > }; > > #define FUSE_IO_PRIV_SYNC(f) \ > -- > 1.7.9.5 >