2002-10-16 08:46:18

by Zilvinas Valinskas

[permalink] [raw]
Subject: sendfile(2) behaviour has changed ?

This sample code copies a file using sendfile(2) call works just fine on
2.2.x and 2.4.x kernels. On 2.5.x kernels (not sure starting which
version) it stopped working. Program terminates with EINVAL error.

$ ./sendfile
sendfile: Invalid argument

Is this expected behaviour ? that sendfile(2) on 2.5.4x linux kernel requires
socket as an output fd paramter ?

Was it ever legal to copy file(s) on filesystem using sendfile(2) ?
(which was kindda nice feature ... )


Attachments:
(No filename) (470.00 B)
sendfile.c (642.00 B)
Download all attachments

2002-10-16 09:04:52

by Matti Aarnio

[permalink] [raw]
Subject: Re: sendfile(2) behaviour has changed ?

On Wed, Oct 16, 2002 at 10:49:08AM +0200, Zilvinas Valinskas wrote:
> This sample code copies a file using sendfile(2) call works just fine on
> 2.2.x and 2.4.x kernels. On 2.5.x kernels (not sure starting which
> version) it stopped working. Program terminates with EINVAL error.
>
> $ ./sendfile
> sendfile: Invalid argument
>
> Is this expected behaviour ? that sendfile(2) on 2.5.4x linux kernel requires
> socket as an output fd paramter ?

It has only been intended for output to a TCP stream socket.

> Was it ever legal to copy file(s) on filesystem using sendfile(2) ?
> (which was kindda nice feature ... )

No. It was a nice misfeature.

/Matti Aarnio

2002-10-16 10:02:31

by David Miller

[permalink] [raw]
Subject: Re: sendfile(2) behaviour has changed ?

From: Matti Aarnio <[email protected]>
Date: Wed, 16 Oct 2002 12:10:46 +0300

On Wed, Oct 16, 2002 at 10:49:08AM +0200, Zilvinas Valinskas wrote:
> Is this expected behaviour ? that sendfile(2) on 2.5.4x linux kernel requires
> socket as an output fd paramter ?

It has only been intended for output to a TCP stream socket.

To be honest, I'm not so sure about this.

For example, I definitely see us supporting this in the
opposite direction when commodity 10gbit hits the market.

Initially I thought "sys_receivefile()" but it makes no
sense when we have a system call that is perfectly capable
of describing the tcp_socket --> page_cache operation.

And I don't think the vfs copy operation using sendfile
is such a bad thing either. It definitely opens the door
for some interesting optimizations. For example, if the
source page is not mapped by a process it could be possible
to just unhash it, mark it dirty, then hash it into the
destination file. Exactly 2 I/O operations and the cpu
doesn't touch the data at all.

2002-10-16 10:19:13

by bert hubert

[permalink] [raw]
Subject: Re: sendfile(2) behaviour has changed ?

On Wed, Oct 16, 2002 at 02:59:35AM -0700, David S. Miller wrote:
> It has only been intended for output to a TCP stream socket.
>
> To be honest, I'm not so sure about this.

The old manpage did not state this properly. In November 2000 I tried to use
sendfile as recvfile and discovered that it did not work. The ensuing
discussion led to the modification of the manpage to state that it would
only write to a socket and only read from a pagecache backed fd.

Regards,

bert hubert

--
http://www.PowerDNS.com Versatile DNS Software & Services
http://www.tk the dot in .tk
http://lartc.org Linux Advanced Routing & Traffic Control HOWTO

2002-10-17 20:46:18

by James Antill

[permalink] [raw]
Subject: Re: sendfile(2) behaviour has changed ?

"David S. Miller" <[email protected]> writes:

> From: Matti Aarnio <[email protected]>
> Date: Wed, 16 Oct 2002 12:10:46 +0300
>
> On Wed, Oct 16, 2002 at 10:49:08AM +0200, Zilvinas Valinskas wrote:
> > Is this expected behaviour ? that sendfile(2) on 2.5.4x linux kernel requires
> > socket as an output fd paramter ?
>
> It has only been intended for output to a TCP stream socket.
>
> To be honest, I'm not so sure about this.
>
> For example, I definitely see us supporting this in the
> opposite direction when commodity 10gbit hits the market.
>
> Initially I thought "sys_receivefile()" but it makes no
> sense when we have a system call that is perfectly capable
> of describing the tcp_socket --> page_cache operation.

It really needs a new interface for recvfile/copyfile/whatever
anyway, as you can only specify an off_t for the from fd at present.

Also consider that if you have 2 network sockets you really want a
way to see which did the EAGAIN.

Which leads to something like...

ssize_t copyfddata(int out_fd, off_t *offset,
int in_fd, off_t *offset, size_t count, int *in_errno);

...and another for the off64_t API, the errno thing looks crappy but I
think creating EREADAGAIN is even worse (and I just know that won't be
the last if it's done that way) ... unless you can think of another way.

--
# James Antill -- [email protected]
:0:
* ^From: .*james@and\.org
/dev/null

2002-10-17 22:43:29

by David Miller

[permalink] [raw]
Subject: Re: sendfile(2) behaviour has changed ?

From: James Antill <[email protected]>
Date: 17 Oct 2002 16:51:30 -0400

It really needs a new interface for recvfile/copyfile/whatever
anyway, as you can only specify an off_t for the from fd at present.

Ummm, you can use lseek() on the 'to' fd perhaps?

2002-10-17 23:17:35

by James Antill

[permalink] [raw]
Subject: Re: sendfile(2) behaviour has changed ?

"David S. Miller" <[email protected]> writes:

> From: James Antill <[email protected]>
> Date: 17 Oct 2002 16:51:30 -0400
>
> It really needs a new interface for recvfile/copyfile/whatever
> anyway, as you can only specify an off_t for the from fd at present.
>
> Ummm, you can use lseek() on the 'to' fd perhaps?

On the client side it's pretty useful to be able to write into the
same file from the network from multiple connections.

You could say that you are much more likely to have multiple
connections reading from one file than have them writing to the
same file on the server, but then there the errno problem is much more
obvious.

--
# James Antill -- [email protected]
:0:
* ^From: .*james@and\.org
/dev/null