Hi,
Is the following behaviour correct on a tcp connection:
* the server issues a shutdown(sock, RW)
* the client side socket passes in CLOSE-WAIT state
* the client issues a write on the socket which succeds.
I expected the last write to fail, since the other side is not
capable any more to receive data. Can someone confirm to me one
of the following:
1. behaviour is correct and why.
2. shutdown is buggy.
3. write is buggy.
Thanks.
Attached are sample codes for the "server" and the "client". Test
was done on latest 2.4 and 2.5 kernels.
Stelian.
--
Stelian Pop <[email protected]>
Alcove - http://www.alcove.com
From: Stelian Pop <[email protected]>
Date: Fri, 5 Apr 2002 11:50:39 +0200
* the client side socket passes in CLOSE-WAIT state
* the client issues a write on the socket which succeds.
...
Attached are sample codes for the "server" and the "client". Test
was done on latest 2.4 and 2.5 kernels.
Your client does not do any write()'s after the shutdown call.
It simply exit(0)'s.
From: Stelian Pop <[email protected]>
Date: Fri, 5 Apr 2002 12:47:33 +0200
On Fri, Apr 05, 2002 at 02:04:43AM -0800, David S. Miller wrote:
> Your client does not do any write()'s after the shutdown call.
> It simply exit(0)'s.
You mean the 'server' ? Even if I add a sleep(600) between the
shutdown() call and the exit() call I get the same behaviour.
Oh I see now. Here is what should happen:
* server shutdown(ALL)
* the write() should succeed on the client
* client socket receives a TCP reset
If this isn't what is happening, send us a trace.
On Fri, Apr 05, 2002 at 02:04:43AM -0800, David S. Miller wrote:
> From: Stelian Pop <[email protected]>
> Date: Fri, 5 Apr 2002 11:50:39 +0200
>
> * the client side socket passes in CLOSE-WAIT state
> * the client issues a write on the socket which succeds.
> ...
> Attached are sample codes for the "server" and the "client". Test
> was done on latest 2.4 and 2.5 kernels.
>
> Your client does not do any write()'s after the shutdown call.
> It simply exit(0)'s.
You mean the 'server' ? Even if I add a sleep(600) between the
shutdown() call and the exit() call I get the same behaviour.
Stelian.
--
Stelian Pop <[email protected]>
Alcove - http://www.alcove.com
On Fri, Apr 05, 2002 at 02:44:35AM -0800, David S. Miller wrote:
> > Your client does not do any write()'s after the shutdown call.
> > It simply exit(0)'s.
>
> You mean the 'server' ? Even if I add a sleep(600) between the
> shutdown() call and the exit() call I get the same behaviour.
>
> Oh I see now. Here is what should happen:
>
> * server shutdown(ALL)
> * the write() should succeed on the client
So this is really supposed to succeed then... Somewhat illogical to
me...
> * client socket receives a TCP reset
How is the client socket supposed to know it received a TCP reset
(I am talking from the application point of view, not the kernel...) ?
Stelian.
--
Stelian Pop <[email protected]>
Alcove - http://www.alcove.com
From: Stelian Pop <[email protected]>
Date: Fri, 5 Apr 2002 12:55:09 +0200
> * client socket receives a TCP reset
How is the client socket supposed to know it received a TCP reset
(I am talking from the application point of view, not the kernel...) ?
You may find out by attempting to read data, or you may use the
extended IP error reporting Linux has.
But all of this is irrelevant. When a server closes and says "send me
no more data", this implies that the server told the client it doesn't
want any more data. If the client sends data, this is a gross fatal
error, so TCP resets in FIN_WAIT{1,2} states.
RFC 793 originally specified to queue the data, RFC 1122 is where
the current behavior is defined.
"David S. Miller" <[email protected]> writes:
>
> But all of this is irrelevant. When a server closes and says "send me
> no more data", this implies that the server told the client it doesn't
> want any more data. If the client sends data, this is a gross fatal
> error, so TCP resets in FIN_WAIT{1,2} states.
Linux has one hole in this. When the reset arrives too late the pending
error is not passed up and propagated through shutdown/close.
-Andi
On Fri, Apr 05, 2002 at 03:02:51AM -0800, David S. Miller wrote:
> From: Stelian Pop <[email protected]>
> Date: Fri, 5 Apr 2002 12:55:09 +0200
>
> > * client socket receives a TCP reset
>
> How is the client socket supposed to know it received a TCP reset
> (I am talking from the application point of view, not the kernel...) ?
>
> You may find out by attempting to read data, or you may use the
> extended IP error reporting Linux has.
In the client, I've added a read(fd, buf, 1) just after the write,
and rerun:
client server
... ...
shutdown()
...
lsof will shown the client socket being in CLOSE_WAIT at this point
...
write(fd, buf, 5) = 5
read(fd, buf, 1) = 0
...
exit(0)
exit(0)
As you can see, read() doesn't return any error, just 0 to
indicate end-of-file (seems correct interpretation of remote
shutdown here), but it doesn't report any error from the
precedent write... Bug ?
> But all of this is irrelevant. When a server closes and says "send me
> no more data", this implies that the server told the client it doesn't
> want any more data.
Perfectly valid but only if the 'applicative close' has reached the
other end. If it's still queued in TCP buffers, the client may not
have received yet that close... I thought that this was the only
reason that shutdown() existed at all: flush the buffers preparing
an imminent close...
> If the client sends data, this is a gross fatal
> error, so TCP resets in FIN_WAIT{1,2} states.
>
> RFC 793 originally specified to queue the data, RFC 1122 is where
> the current behavior is defined.
Thanks for the pointer.
Stelian.
--
Stelian Pop <[email protected]>
Alcove - http://www.alcove.com
From: Stelian Pop <[email protected]>
Date: Fri, 5 Apr 2002 14:00:55 +0200
As you can see, read() doesn't return any error, just 0 to
indicate end-of-file (seems correct interpretation of remote
shutdown here), but it doesn't report any error from the
precedent write... Bug ?
Race, wait a bit, the reset will arrive.
This is a error state in your code, writing when the server expects no
data. Always remember that when you are trying to judge if
whatever socket operation results is valid or invalid.
Since we find out about the reset asynchronously, this is what
happens.
> But all of this is irrelevant. When a server closes and says "send me
> no more data", this implies that the server told the client it doesn't
> want any more data.
Perfectly valid but only if the 'applicative close' has reached the
other end. If it's still queued in TCP buffers, the client may not
have received yet that close... I thought that this was the only
reason that shutdown() existed at all: flush the buffers preparing
an imminent close...
Not a close "in TCP" but a close "in your apps protocol".
As in:
write(socket_fd, "Ok client I have all your data, lets close now")
See? If the server closes without telling the client that, all
bets are off.
Look, your app is buggy, PERIOD. Once you start to write to a closed
socket, sorry the phase of the moon decides what happens to you. Most
of the time you'll be lucky and see an error.
On Fri, Apr 05, 2002 at 04:04:51AM -0800, David S. Miller wrote:
> From: Stelian Pop <[email protected]>
> Date: Fri, 5 Apr 2002 14:00:55 +0200
>
> As you can see, read() doesn't return any error, just 0 to
> indicate end-of-file (seems correct interpretation of remote
> shutdown here), but it doesn't report any error from the
> precedent write... Bug ?
>
> Race, wait a bit, the reset will arrive.
30 seconds later the read still returns 0...
> Look, your app is buggy, PERIOD. Once you start to write to a closed
> socket, sorry the phase of the moon decides what happens to you. Most
> of the time you'll be lucky and see an error.
The socket is not closed on the client side. I expect the kernel to
signal me an error (reset of read/write return code, whatever) if
the connection is closed by the _server_ (with close OR shutdown).
Stelian.
--
Stelian Pop <[email protected]>
Alcove - http://www.alcove.com
On Fri, Apr 05, 2002 at 04:04:51AM -0800, David S. Miller wrote:
> As you can see, read() doesn't return any error, just 0 to
> indicate end-of-file (seems correct interpretation of remote
> shutdown here), but it doesn't report any error from the
> precedent write... Bug ?
>
> Race, wait a bit, the reset will arrive.
Ok, I investigated this a bit more using setsockopt(...,SO_ERROR,...)
After the write in the client (which is done after the server has
shutdown()'ed it), the error bit is set on the client socket
(-EPIPE).
If the client issues a second write, the write fails (correctly)
setting errno to -EPIPE.
If the client issues a read, read doesn't return an error. The
socket error bit is still there however, even after read() returns.
If the client issues a close, close will return 0 too.
Whether the read() should return the error bit or not is
debatable, but IMHO close at least should propagate the error.
Stelian.
--
Stelian Pop <[email protected]>
Alcove - http://www.alcove.com