Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Feedback-ID: id8a24192:Fastmail
Message-ID: <712cd802-f3bb-9840-e334-385cd42325f2@fastmail.fm>
Date:   Thu, 3 Nov 2022 23:28:29 +0100
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.3.3
Subject: Re: [RFC PATCH 4/4] ublk_drv: support splice based read/write zero
 copy
To:     Ming Lei <ming.lei@redhat.com>, Jens Axboe <axboe@kernel.dk>,
        io-uring@vger.kernel.org
Cc:     linux-block@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-fsdevel@vger.kernel.org,
        Miklos Szeredi <mszeredi@redhat.com>,
        Stefan Hajnoczi <stefanha@redhat.com>,
        ZiyangZhang <ZiyangZhang@linux.alibaba.com>
References: <20221103085004.1029763-1-ming.lei@redhat.com>
 <20221103085004.1029763-5-ming.lei@redhat.com>
Content-Language: en-US
From:   Bernd Schubert <bernd.schubert@fastmail.fm>
In-Reply-To: <20221103085004.1029763-5-ming.lei@redhat.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk


On 11/3/22 09:50, Ming Lei wrote:
> Pass ublk block IO request pages to kernel backend IO handling code via
> pipe, and request page copy can be avoided. So far, the existed
> pipe/splice mechanism works for handling write request only.
> 
> The initial idea of using splice for zero copy is from Miklos and Stefan.
> 
> Read request's zero copy requires pipe's change to allow one read end to
> produce buffers for another read end to consume. The added SPLICE_F_READ_TO_READ
> flag is for supporting this feature.
> 
> READ is handled by sending IORING_OP_SPLICE with SPLICE_F_DIRECT |
> SPLICE_F_READ_TO_READ. WRITE is handled by sending IORING_OP_SPLICE with
> SPLICE_F_DIRECT. Kernel internal pipe is used for simplifying userspace,
> meantime potential info leak could be avoided.


Sorry to ask, do you have an ublk branch that gives an example how to 
use this?

I still have several things to fix in my branches, but I got basic fuse 
uring with copies working. Adding back splice would be next after 
posting rfc patches. My initial assumption was that I needed to 
duplicate everything splice does into the fuse .uring_cmd handler - 
obviously there is a better way with your patches.

This week I have a few days off, by end of next week or the week after I 
might have patches in an rfc state (one thing I'm going to ask about is 
how do I know what is the next CQE in the kernel handler - ublk does 
this with tags through mq, but I don't understand yet where the tag is 
increased and what the relation between tag and right CQE order is).

This got modeled a bit after ublk, but then diverged a bit.

https://github.com/aakefbs/linux/tree/fuse-uring
https://github.com/aakefbs/libfuse/tree/uring

(Again, the branches are not ready by any means for review yet).


Thanks,
Bernd