2009-05-19 21:29:57

by Nick Pelly

[permalink] [raw]
Subject: Expected behavior of shutdown() in multi-threaded socket programming

Hi,

I am interested in the expected behavior of shutdown() on a socket
that is also blocked on connect(), accept(), read(), write(), poll()
or select() in another thread.

For example:

THREAD 1
fd = socket()
listen(fd)
bind(fd)
accept(fd) <--- blocks

THREAD 2
shutdown(fd) <--- what is meant to happen to accept() in thread 1?

If thread 2 is run after thread 1, what should happen to the blocked
accept() call when shutdown() is called in thread 2?


My observations are that
TCP sockets: accept() immediately returns with errno EINVAL
unix domain sockets: accept() immediately returns with return errno EINVAL
RFCOMM sockets: accept() continues to block
L2CAP sockets: accept() continues to block

I tested on 2.6.18, 2.6.28 and 2.6.29 and results were the same.

Included is a sample program sock_shutdown_test.c that can easily
exhibit this behavior. For example
sock_shutdown_test unix accept_shutdown # accept() returns
sock_shutdown_test tcp accept_shutdown # accept() returns
sock_shutdown_test rfcomm accept_shutdown # accept() blocks forever
sock_shutdown_test l2cap accept_shutdown # accept() blocks forever

I also have similar results for other blocking syscalls such as
connect(), read(), write(), poll() etc, but the test program is not as
simple.

So my question is: What is the correct behavior for sockets here?


It is desirable for Android that shutdown() to force other threads
blocked on that socket to return. In fact, if they don't it makes
multi-threaded socket programming very hard since there is no other
simple way to abort a blocked I/O operation. We have to resort to
using poll() in combination with a selfpipe that we can write a byte
to in order to abort the poll(). But this is quite inefficient as it
triples the number of fd's needed for every socket not to mention the
4k buffer space needed in the kernel for the selfpipe. A global
selfpipe per process is an improvement but it is quite messy getting
this correct without race conditions.


Input as to the correct behavior is appreciated,
Nick


Attachments:
sock_shutdown_test.c (6.71 kB)

2009-06-15 21:20:43

by Nick Pelly

[permalink] [raw]
Subject: Re: Expected behavior of shutdown() in multi-threaded socket programming

On Fri, Jun 12, 2009 at 1:46 AM, Iain Hibbert<[email protected]> wrote:
> On Thu, 11 Jun 2009, Nick Pelly wrote:
>
>> Any comments on this one? I would like to correct the behavior of
>> shutdown() on AF_BLUETOOTH sockets, but I have been advised by Marcel
>> Holtmann that we need to agree on the correct behavior first.
>>
>> How should shutdown() behave when other threads are blocked on the same =
socket?
>
> IMHO consistency should apply.
>
> The opengroup specification for shutdown() says
>
> =A0"The shutdown() function shall cause all or part of a full-duplex
> =A0connection on the socket associated with the file descriptor socket to
> =A0be shut down."
>
> and while that does not really cover the case when the socket is blocked
> in accept(), if all the other socket types abort the block then that is
> what the PF_BLUETOOTH sockets should do too.
>
> The opengroup specification for accept() suggests EINVAL would be returne=
d
> if the socket was not accepting connections and arguably that is the case
> after a shutdown(), though ECONNABORTED could be used too (your program
> displays ECONNABORTED on NetBSD for instance)
>
>> I also have similar results for other blocking syscalls such as
>> connect(), read(), write(), poll() etc, but the test program is not as
>> simple.
>
> They should all handle the shutdown().

Sounds good to me.

Thanks for the input.

Nick

2009-06-12 08:46:05

by Iain Hibbert

[permalink] [raw]
Subject: Re: Expected behavior of shutdown() in multi-threaded socket programming

On Thu, 11 Jun 2009, Nick Pelly wrote:

> Any comments on this one? I would like to correct the behavior of
> shutdown() on AF_BLUETOOTH sockets, but I have been advised by Marcel
> Holtmann that we need to agree on the correct behavior first.
>
> How should shutdown() behave when other threads are blocked on the same socket?

IMHO consistency should apply.

The opengroup specification for shutdown() says

"The shutdown() function shall cause all or part of a full-duplex
connection on the socket associated with the file descriptor socket to
be shut down."

and while that does not really cover the case when the socket is blocked
in accept(), if all the other socket types abort the block then that is
what the PF_BLUETOOTH sockets should do too.

The opengroup specification for accept() suggests EINVAL would be returned
if the socket was not accepting connections and arguably that is the case
after a shutdown(), though ECONNABORTED could be used too (your program
displays ECONNABORTED on NetBSD for instance)

> I also have similar results for other blocking syscalls such as
> connect(), read(), write(), poll() etc, but the test program is not as
> simple.

They should all handle the shutdown().

iain

2009-06-12 00:24:21

by Nick Pelly

[permalink] [raw]
Subject: Re: Expected behavior of shutdown() in multi-threaded socket programming

Any comments on this one? I would like to correct the behavior of
shutdown() on AF_BLUETOOTH sockets, but I have been advised by Marcel
Holtmann that we need to agree on the correct behavior first.

How should shutdown() behave when other threads are blocked on the same socket?

More detail in the original mail below.

Thanks,
Nick

On Tue, May 19, 2009 at 2:29 PM, Nick Pelly<[email protected]> wrote:
> Hi,
>
> I am interested in the expected behavior of shutdown() on a socket
> that is also blocked on connect(), accept(), read(), write(), poll()
> or select() in another thread.
>
> For example:
>
> THREAD 1
> fd = socket()
> listen(fd)
> bind(fd)
> accept(fd) ?<--- blocks
>
> THREAD 2
> shutdown(fd) ?<--- what is meant to happen to accept() in thread 1?
>
> If thread 2 is run after thread 1, what should happen to the blocked
> accept() call when shutdown() is called in thread 2?
>
>
> My observations are that
> TCP sockets: accept() immediately returns with errno EINVAL
> unix domain sockets: accept() immediately returns with return errno EINVAL
> RFCOMM sockets: accept() continues to block
> L2CAP sockets: accept() continues to block
>
> I tested on 2.6.18, 2.6.28 and 2.6.29 and results were the same.
>
> Included is a sample program sock_shutdown_test.c that can easily
> exhibit this behavior. For example
> sock_shutdown_test unix accept_shutdown ?# accept() returns
> sock_shutdown_test tcp accept_shutdown ? # accept() returns
> sock_shutdown_test rfcomm accept_shutdown ? # accept() blocks forever
> sock_shutdown_test l2cap accept_shutdown ? # accept() blocks forever
>
> I also have similar results for other blocking syscalls such as
> connect(), read(), write(), poll() etc, but the test program is not as
> simple.
>
> So my question is: What is the correct behavior for sockets here?
>
>
> It is desirable for Android that shutdown() to force other threads
> blocked on that socket to return. In fact, if they don't it makes
> multi-threaded socket programming very hard since there is no other
> simple way to abort a blocked I/O operation. We have to resort to
> using poll() in combination with a selfpipe that we can write a byte
> to in order to abort the poll(). But this is quite inefficient as it
> triples the number of fd's needed for every socket not to mention the
> 4k buffer space needed in the kernel for the selfpipe. A global
> selfpipe per process is an improvement but it is quite messy getting
> this correct without race conditions.
>
>
> Input as to the correct behavior is appreciated,
> Nick
>