Hi,
when I read about the splice() system call, I thought it was obvious
that it could copy data between two pipes. I was surprised that this
assumption is wrong, it's not possible on 2.6.29, I get EINVAL. Can
anybody please explain this limitation?
Background: I want to forward data between two subprocesses, which are
connected to me with a pipe().
I have attached a small test program which prints a table of supported
splice operations. Here's the output on 2.6.29.1:
in\out pipe sock reg chr
pipe no yes yes yes
sock no no no no
reg yes no no no
chr no no no no
Max
Max Kellermann <[email protected]> writes:
I don't think splice is about handling all possible cases,
but just cases where the kernel can do better than user space.
I don't think that's the case here.
> when I read about the splice() system call, I thought it was obvious
> that it could copy data between two pipes.
It would be more efficient if you used fd passing to pass the fd
around to the other process and let it read directly.
-Andi
--
[email protected] -- Speaking for myself only.
On 2009/04/29 17:23, Andi Kleen <[email protected]> wrote:
> I don't think splice is about handling all possible cases,
> but just cases where the kernel can do better than user space.
> I don't think that's the case here.
If splice() is about passing pointers of a pipe buffer, what's more
trivial (and natural) than passing that pointer between two pipes?
> > when I read about the splice() system call, I thought it was obvious
> > that it could copy data between two pipes.
>
> It would be more efficient if you used fd passing to pass the fd
> around to the other process and let it read directly.
That's not so easy in my case. The header output of the one process
has to be parsed before the rest of it (or part of the rest) is going
to be forwarded to the second one. My master process would lose
control over the transfer. splice() looks like the perfect solution.
Max
On Wed, Apr 29, 2009 at 09:42:55PM +0200, Max Kellermann wrote:
> On 2009/04/29 17:23, Andi Kleen <[email protected]> wrote:
> > I don't think splice is about handling all possible cases,
> > but just cases where the kernel can do better than user space.
> > I don't think that's the case here.
>
> If splice() is about passing pointers of a pipe buffer, what's more
> trivial (and natural) than passing that pointer between two pipes?
>
> > > when I read about the splice() system call, I thought it was obvious
> > > that it could copy data between two pipes.
> >
> > It would be more efficient if you used fd passing to pass the fd
> > around to the other process and let it read directly.
>
> That's not so easy in my case. The header output of the one process
> has to be parsed before the rest of it (or part of the rest) is going
> to be forwarded to the second one. My master process would lose
> control over the transfer. splice() looks like the perfect solution.
indeed, that could make sense. From what I have seen in the splicing
code, I think tht implementing pipe to pipe should not be *that* hard,
starting from existing code (eg: net to pipe). Maybe you could try to
implement it since you have the code which makes use of it ? I think
it is the kind of feature which can only improve step by step based
on application needs.
BTW, I like your test program. Simple and easy. I have completed it to
test tcp and udp, you can find it attached.
Regards,
Willy
On Wed, Apr 29 2009, Max Kellermann wrote:
> Hi,
>
> when I read about the splice() system call, I thought it was obvious
> that it could copy data between two pipes. I was surprised that this
> assumption is wrong, it's not possible on 2.6.29, I get EINVAL. Can
> anybody please explain this limitation?
>
> Background: I want to forward data between two subprocesses, which are
> connected to me with a pipe().
>
> I have attached a small test program which prints a table of supported
> splice operations. Here's the output on 2.6.29.1:
>
> in\out pipe sock reg chr
> pipe no yes yes yes
> sock no no no no
> reg yes no no no
> chr no no no no
See sys_tee().
--
Jens Axboe
On 2009/04/30 08:21, Jens Axboe <[email protected]> wrote:
> See sys_tee().
Hm, so I could tee() from pipe1 to pipe2, then delete data from pipe1
by splicing it to /dev/null...
Max
On Thu, 30 Apr 2009, Willy Tarreau wrote:
> BTW, I like your test program. Simple and easy. I have completed it to
> test tcp and udp, you can find it attached.
I think the shown ability to splice to a unix socket may be misleading.
Can someone offer some insight to my previous post (22nd April, "splice()
on unix sockets")? Does splice to a unix socket actually result in zero
copy?
$ ./test_splice
in\out pipe reg chr unix tcp udp
pipe no yes yes yes yes yes
reg yes no no no no no
chr no no no no no no
unix no no no no no no
tcp yes no no no no no
udp no no no no no no
Thanks
--
Mark