2002-03-28 02:52:44

by Mark Atwood

[permalink] [raw]
Subject: How to tell how much to expect from a fd


Does there exist a fcntl or some other way to tell how much data is
"ready to be read" from a fd?

I'm doing this thing where I make the fd non-blocking, select on it,
and then read on it into a buffer that I am pregrowing with realloc.

When the high water mark is up to the top of the buffer, I realloc the
buffer to make it bigger. At present, I'm just adding a constant
value to the buffer size each time I need to do this, but if there was
a way to easily tell how much was "ready to be read" from the fd.

It's not necessary to be exact. If more becomes available between the
time I do this wanted magic and do the read, read's 3rd parameter will
keep me safe, and if it's too low, like if a dup of the fd already
snarfed the data, also no big deal, I'm non-blocking and check the
return value.

So, is this "nice to have" available?


--
Mark Atwood | Well done is better than well said.
[email protected] | http://www.pobox.com/~mra


2002-03-28 13:28:19

by Eric Lammerts

[permalink] [raw]
Subject: Re: How to tell how much to expect from a fd


On 27 Mar 2002, Mark Atwood wrote:

> Does there exist a fcntl or some other way to tell how much data is
> "ready to be read" from a fd?

int val;
ioctl(fd, FIONREAD, &val);

See tcp(7).

Eric

2002-03-31 03:03:32

by David Schwartz

[permalink] [raw]
Subject: Re: How to tell how much to expect from a fd


On 27 Mar 2002 18:52:39 -0800, Mark Atwood wrote:
>
>Does there exist a fcntl or some other way to tell how much data is
>"ready to be read" from a fd?
>
>I'm doing this thing where I make the fd non-blocking, select on it,
>and then read on it into a buffer that I am pregrowing with realloc.
>
>When the high water mark is up to the top of the buffer, I realloc the
>buffer to make it bigger. At present, I'm just adding a constant
>value to the buffer size each time I need to do this, but if there was
>a way to easily tell how much was "ready to be read" from the fd.
>
>It's not necessary to be exact. If more becomes available between the
>time I do this wanted magic and do the read, read's 3rd parameter will
>keep me safe, and if it's too low, like if a dup of the fd already
>snarfed the data, also no big deal, I'm non-blocking and check the
>return value.
>
>So, is this "nice to have" available?

It is possible that you are the one-in-a-million case that really needs
this, but the vast majority of the time people ask for this, they don't
really want it.

Consider two very important points:

First, by the time you get this information, it's obsolete. If more data
becomes available in-between when you make this call and when you attempt to
read, you'll take a double penalty. You'll need an extra 'read' later to get
the rest and your next poll/select will break out immediately (which can be
very expensive if you're dealing with a large number of fds).

Second, this would double the number of system calls you need to read the
data from the socket. There's almost no conceivable scenario in which it's
worth the cost of doing this when you can either keep a buffer that's a bit
too large around or copy it into a right-sized buffer and you can choose
which option after you know how many bytes you got.

DS