2003-05-01 22:19:08

by Chris Friesen

[permalink] [raw]
Subject: Re: sendfile

P?l Halvorsen wrote:

> As far as i understand mmap/send, you'll have a copy operation in the
> kernel here. mmap shares the kernel and user buffer, but when sending the
> packet data is copied to the socket buffer!!??

Yes, there is a copy there.

> OK, but I understand that my streaming scenario is not the target
> application for sendfile.

What stops you from using sendfile (with TCP) to each destination separately,
with the client only reading from the pipe as needed (presumably with a number
of frames worth of buffer on the client side)?


Chris

--
Chris Friesen | MailStop: 043/33/F10
Nortel Networks | work: (613) 765-0557
3500 Carling Avenue | fax: (613) 765-2986
Nepean, ON K2H 8E9 Canada | email: [email protected]


2003-05-01 23:16:00

by Ketil Froyn

[permalink] [raw]
Subject: Re: sendfile

On Thu, 1 May 2003, Chris Friesen wrote:

> P?l Halvorsen wrote:
>
> > OK, but I understand that my streaming scenario is not the target
> > application for sendfile.
>
> What stops you from using sendfile (with TCP) to each destination
> separately, with the client only reading from the pipe as needed
> (presumably with a number of frames worth of buffer on the client
> side)?

I don't think TCP is suitable for streaming multimedia stuff to clients.
For instance, if a packet does not arrive on the client, it's better to
handle this in the client and skip a frame or show one of worse quality
than to have the video stop while waiting for the server to resend.

Ketil


2003-05-02 02:23:30

by Mark Mielke

[permalink] [raw]
Subject: Re: sendfile

On Thu, May 01, 2003 at 06:31:05PM -0400, Chris Friesen wrote:
> P?l Halvorsen wrote:
> >As far as i understand mmap/send, you'll have a copy operation in the
> >kernel here. mmap shares the kernel and user buffer, but when sending the
> >packet data is copied to the socket buffer!!??
> Yes, there is a copy there.

As far as I understand, sendfile() still requires the data to get from the
disk to a page in memory, similar to how send() referencing an mmap()'d page
may cause a page fault, reading the data from disk to a page in memory. One
copy each. I don't know of a kernel interface that lets data be copied from
disk to ethernet card without involving a temporary copy to be in paged
memory at some point in time... perhaps the iSCSI stuff can do this? I dunno.

Somebody else pointed out that mmap() may not yet be implemented completely
optimally. I will have to look at the code before I continue to make my
'in theory' comments, however the following 'NOTE' in the manpage for sendfile
makes me suspect that sendfile() is very similar to mmap()/write():

-- CUT --
Presently the descriptor from which data is read cannot correspond to a
socket, it must correspond to a file which supports mmap()-like opera-
tions.
-- CUT --

> >OK, but I understand that my streaming scenario is not the target
> >application for sendfile.
> What stops you from using sendfile (with TCP) to each destination
> separately, with the client only reading from the pipe as needed
> (presumably with a number of frames worth of buffer on the client side)?

TCP isn't very well suited for video feeds. First, it is streamed, which
makes it a little annoying to ensure that only whole frames get through.
Second, its acknowledgement scheme prefers reliability over low latency.

I'm hoping for good things from SCTP. From what I've read, it looks as
if it should provide a compromise between TCP and UDP that is quite
optimal.

mark

--
[email protected]/[email protected]/[email protected] __________________________
. . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder
|\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ |
| | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada

One ring to rule them all, one ring to find them, one ring to bring them all
and in the darkness bind them...

http://mark.mielke.cc/

2003-05-02 08:50:00

by Bernd Eckenfels

[permalink] [raw]
Subject: Re: sendfile

In article <[email protected]> you wrote:
> I don't think TCP is suitable for streaming multimedia stuff to clients.
> For instance, if a packet does not arrive on the client, it's better to
> handle this in the client and skip a frame or show one of worse quality
> than to have the video stop while waiting for the server to resend.

Yes, this is a problem, but on the other hand, if you want to stream to a
large number of clients, you need to consider deployment and firewalling
issues.

Nearly all streaming applications out there nowaday offer at least a TCP (or
HTTP) fallback, or use only TCP.

Greetings
Bernd
--
eckes privat - http://www.eckes.org/
Project Freefire - http://www.freefire.org/