Return-Path: MIME-Version: 1.0 From: Nick Pelly Date: Tue, 19 May 2009 14:29:57 -0700 Message-ID: <35c90d960905191429x1c3b6104qca2a672d50f0364c@mail.gmail.com> Subject: Expected behavior of shutdown() in multi-threaded socket programming To: linux-bluetooth@vger.kernel.org, netdev@vger.kernel.org, davem@redhat.com Content-Type: multipart/mixed; boundary=0015174c33f0f87fd4046a4a9d6c Sender: linux-bluetooth-owner@vger.kernel.org List-ID: --0015174c33f0f87fd4046a4a9d6c Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hi, I am interested in the expected behavior of shutdown() on a socket that is also blocked on connect(), accept(), read(), write(), poll() or select() in another thread. For example: THREAD 1 fd = socket() listen(fd) bind(fd) accept(fd) <--- blocks THREAD 2 shutdown(fd) <--- what is meant to happen to accept() in thread 1? If thread 2 is run after thread 1, what should happen to the blocked accept() call when shutdown() is called in thread 2? My observations are that TCP sockets: accept() immediately returns with errno EINVAL unix domain sockets: accept() immediately returns with return errno EINVAL RFCOMM sockets: accept() continues to block L2CAP sockets: accept() continues to block I tested on 2.6.18, 2.6.28 and 2.6.29 and results were the same. Included is a sample program sock_shutdown_test.c that can easily exhibit this behavior. For example sock_shutdown_test unix accept_shutdown # accept() returns sock_shutdown_test tcp accept_shutdown # accept() returns sock_shutdown_test rfcomm accept_shutdown # accept() blocks forever sock_shutdown_test l2cap accept_shutdown # accept() blocks forever I also have similar results for other blocking syscalls such as connect(), read(), write(), poll() etc, but the test program is not as simple. So my question is: What is the correct behavior for sockets here? It is desirable for Android that shutdown() to force other threads blocked on that socket to return. In fact, if they don't it makes multi-threaded socket programming very hard since there is no other simple way to abort a blocked I/O operation. We have to resort to using poll() in combination with a selfpipe that we can write a byte to in order to abort the poll(). But this is quite inefficient as it triples the number of fd's needed for every socket not to mention the 4k buffer space needed in the kernel for the selfpipe. A global selfpipe per process is an improvement but it is quite messy getting this correct without race conditions. Input as to the correct behavior is appreciated, Nick --0015174c33f0f87fd4046a4a9d6c Content-Type: text/x-csrc; charset=US-ASCII; name="sock_shutdown_test.c" Content-Disposition: attachment; filename="sock_shutdown_test.c" Content-Transfer-Encoding: base64 X-Attachment-Id: f_fux3srsy0 LyoqIHRlc3RpbmcgYmVoYXZpb3Igb2Ygc2h1dGRvd24oKSAqLwoKI2luY2x1ZGUgPHN0ZGxpYi5o PgojaW5jbHVkZSA8c3RkaW8uaD4KI2luY2x1ZGUgPGVycm5vLmg+CiNpbmNsdWRlIDxzeXMvdWlv Lmg+CiNpbmNsdWRlIDx1bmlzdGQuaD4KCiNpbmNsdWRlIDxwdGhyZWFkLmg+CiNpbmNsdWRlIDxz dGRpby5oPgojaW5jbHVkZSA8c3RkbGliLmg+CiNpbmNsdWRlIDxlcnJuby5oPgojaW5jbHVkZSA8 dW5pc3RkLmg+CiNpbmNsdWRlIDxzeXMvc29ja2V0Lmg+CiNpbmNsdWRlIDxzeXMvaW9jdGwuaD4K I2luY2x1ZGUgPHN5cy9wb2xsLmg+CiNpbmNsdWRlIDxzeXMvdW4uaD4KI2luY2x1ZGUgPG5ldGlu ZXQvaW4uaD4KCiNpbmNsdWRlIDxibHVldG9vdGgvYmx1ZXRvb3RoLmg+CiNpbmNsdWRlIDxibHVl dG9vdGgvcmZjb21tLmg+CiNpbmNsdWRlIDxibHVldG9vdGgvc2NvLmg+CiNpbmNsdWRlIDxibHVl dG9vdGgvbDJjYXAuaD4KCmVudW0gc29ja190eXBlIHsKICAgIFVOSVggPSAwLAogICAgUkZDT01N LAogICAgU0NPLAogICAgTDJDQVAsCiAgICBUQ1AsCn07CgpzdHJ1Y3QgdGhyZWFkX2FyZ3Mgewog ICAgaW50IGZkOwogICAgaW50IHR5cGU7CiAgICBpbnQgZGVsYXk7Cn07CgpzdHJ1Y3Qgc29ja2Fk ZHJfdW4gIGxvY2FsX2FkZHJfdW4gID0ge0FGX1VOSVgsICIvdG1wL2ZvbyJ9OwpzdHJ1Y3Qgc29j a2FkZHJfcmMgIGxvY2FsX2FkZHJfcmMgID0ge0FGX0JMVUVUT09USCwgKkJEQUREUl9BTlksIDR9 OwpzdHJ1Y3Qgc29ja2FkZHJfc2NvIGxvY2FsX2FkZHJfc2NvID0ge0FGX0JMVUVUT09USCwgKkJE QUREUl9MT0NBTH07CnN0cnVjdCBzb2NrYWRkcl9sMiAgbG9jYWxfYWRkcl9sMiAgPSB7QUZfQkxV RVRPT1RILCBodG9icygweDEwMDEpLCAqQkRBRERSX0FOWSwgMH07CnN0cnVjdCBzb2NrYWRkcl9p biAgbG9jYWxfYWRkcl9pbiAgPSB7QUZfSU5FVCwgOTk5OSwgezB9LCB7MH19OwoKc3RydWN0IHNv Y2thZGRyX3VuICByZW1vdGVfYWRkcl91biAgOwpzdHJ1Y3Qgc29ja2FkZHJfcmMgIHJlbW90ZV9h ZGRyX3JjICA7CnN0cnVjdCBzb2NrYWRkcl9zY28gcmVtb3RlX2FkZHJfc2NvIDsKc3RydWN0IHNv Y2thZGRyX2wyICByZW1vdGVfYWRkcl9sMiAgOwpzdHJ1Y3Qgc29ja2FkZHJfaW4gIHJlbW90ZV9h ZGRyX2luICA7CgpzdGF0aWMgaW50IF9zb2NrZXQoaW50IHR5cGUpIHsKICAgIGludCByZXQ7CiAg ICBpbnQgZmFtaWx5ID0gLTE7CiAgICBpbnQgdHlwID0gLTE7CiAgICBpbnQgcHJvdG9jb2wgPSAt MTsKCiAgICBzd2l0Y2ggKHR5cGUpIHsKICAgIGNhc2UgVU5JWDoKICAgICAgICBmYW1pbHkgPSBQ Rl9VTklYOwogICAgICAgIHR5cCA9IFNPQ0tfU1RSRUFNOwogICAgICAgIHByb3RvY29sID0gMDsK ICAgICAgICBicmVhazsKICAgIGNhc2UgUkZDT01NOgogICAgICAgIGZhbWlseSA9IFBGX0JMVUVU T09USDsKICAgICAgICB0eXAgPSBTT0NLX1NUUkVBTTsKICAgICAgICBwcm90b2NvbCA9IEJUUFJP VE9fUkZDT01NOwogICAgICAgIGJyZWFrOwogICAgY2FzZSBTQ086CiAgICAgICAgZmFtaWx5ID0g UEZfQkxVRVRPT1RIOwogICAgICAgIHR5cCA9IFNPQ0tfU0VRUEFDS0VUOwogICAgICAgIHByb3Rv Y29sID0gQlRQUk9UT19TQ087CiAgICAgICAgYnJlYWs7CiAgICBjYXNlIEwyQ0FQOgogICAgICAg IGZhbWlseSA9IFBGX0JMVUVUT09USDsKICAgICAgICB0eXAgPSBTT0NLX1NFUVBBQ0tFVDsKICAg ICAgICBwcm90b2NvbCA9IEJUUFJPVE9fTDJDQVA7CiAgICAgICAgYnJlYWs7CiAgICBjYXNlIFRD UDoKICAgICAgICBmYW1pbHkgPSBQRl9JTkVUOwogICAgICAgIHR5cCA9IFNPQ0tfU1RSRUFNOwog ICAgICAgIHByb3RvY29sID0gMDsKICAgICAgICBicmVhazsKICAgIH0KCiAgICBwcmludGYoIiVk OiBzb2NrZXQoKVxuIiwgZ2V0dGlkKCkpOwogICAgcmV0ID0gc29ja2V0KGZhbWlseSwgdHlwLCBw cm90b2NvbCk7CiAgICBwcmludGYoIiVkOiBzb2NrZXQoKSA9ICVkXG4iLCBnZXR0aWQoKSwgcmV0 KTsKICAgIGlmIChyZXQgPCAwKSBwcmludGYoIlx0ZXJyICVkICglcylcbiIsIGVycm5vLCBzdHJl cnJvcihlcnJubykpOwoKICAgIHJldHVybiByZXQ7Cn0KCnN0YXRpYyBpbnQgX2Nsb3NlKGludCBm ZCkgewogICAgaW50IHJldDsKCiAgICBwcmludGYoIiVkOiBjbG9zZSglZClcbiIsIGdldHRpZCgp LCBmZCk7CiAgICByZXQgPSBjbG9zZShmZCk7CiAgICBwcmludGYoIiVkOiBjbG9zZSglZCkgPSAl ZFxuIiwgZ2V0dGlkKCksIGZkLCByZXQpOwogICAgaWYgKHJldCA8IDApIHByaW50ZigiXHRlcnIg JWQgKCVzKVxuIiwgZXJybm8sIHN0cmVycm9yKGVycm5vKSk7CgogICAgcmV0dXJuIHJldDsKfQoK c3RhdGljIGludCBfYmluZChpbnQgZmQsIGludCB0eXBlKSB7CiAgICBpbnQgbGVuID0gMDsKICAg IGludCByZXQ7CiAgICBzdHJ1Y3Qgc29ja2FkZHIgKmFkZHIgPSBOVUxMOwoKICAgIHN3aXRjaCAo dHlwZSkgewogICAgY2FzZSBVTklYOgogICAgICAgIHVubGluayhsb2NhbF9hZGRyX3VuLnN1bl9w YXRoKTsKICAgICAgICBhZGRyID0gKHN0cnVjdCBzb2NrYWRkciAqKSAmbG9jYWxfYWRkcl91bjsK ICAgICAgICBsZW4gPSBzaXplb2YobG9jYWxfYWRkcl91bik7CiAgICAgICAgYnJlYWs7CiAgICBj YXNlIFJGQ09NTToKICAgICAgICBhZGRyID0gKHN0cnVjdCBzb2NrYWRkciAqKSAmbG9jYWxfYWRk cl9yYzsKICAgICAgICBsZW4gPSBzaXplb2YobG9jYWxfYWRkcl9yYyk7CiAgICAgICAgYnJlYWs7 CiAgICBjYXNlIFNDTzoKICAgICAgICBhZGRyID0gKHN0cnVjdCBzb2NrYWRkciAqKSAmbG9jYWxf YWRkcl9zY287CiAgICAgICAgbGVuID0gc2l6ZW9mKGxvY2FsX2FkZHJfc2NvKTsKICAgICAgICBi cmVhazsKICAgIGNhc2UgTDJDQVA6CiAgICAgICAgYWRkciA9IChzdHJ1Y3Qgc29ja2FkZHIgKikg JmxvY2FsX2FkZHJfbDI7CiAgICAgICAgbGVuID0gc2l6ZW9mKGxvY2FsX2FkZHJfbDIpOwogICAg ICAgIGJyZWFrOwogICAgY2FzZSBUQ1A6CiAgICAgICAgYWRkciA9IChzdHJ1Y3Qgc29ja2FkZHIg KikgJmxvY2FsX2FkZHJfaW47CiAgICAgICAgbGVuID0gc2l6ZW9mKGxvY2FsX2FkZHJfaW4pOwog ICAgICAgIGJyZWFrOwogICAgfQoKICAgIHByaW50ZigiJWQ6IGJpbmQoJWQpXG4iLCBnZXR0aWQo KSwgZmQpOwogICAgcmV0ID0gYmluZChmZCwgYWRkciwgbGVuKTsKICAgIHByaW50ZigiJWQ6IGJp bmQoJWQpID0gJWRcbiIsIGdldHRpZCgpLCBmZCwgcmV0KTsKICAgIGlmIChyZXQgPCAwKSBwcmlu dGYoIlx0ZXJyICVkICglcylcbiIsIGVycm5vLCBzdHJlcnJvcihlcnJubykpOwoKICAgIHJldHVy biByZXQ7Cn0KCnN0YXRpYyBpbnQgX2xpc3RlbihpbnQgZmQsIGludCB0eXBlKSB7CiAgICBpbnQg cmV0OwoKICAgIHByaW50ZigiJWQ6IGxpc3RlbiglZClcbiIsIGdldHRpZCgpLCBmZCk7CiAgICBy ZXQgPSBsaXN0ZW4oZmQsIDEpOwogICAgcHJpbnRmKCIlZDogbGlzdGVuKCVkKSA9ICVkXG4iLCBn ZXR0aWQoKSwgZmQsIHJldCk7CiAgICBpZiAocmV0IDwgMCkgcHJpbnRmKCJcdGVyciAlZCAoJXMp XG4iLCBlcnJubywgc3RyZXJyb3IoZXJybm8pKTsKCiAgICByZXR1cm4gcmV0Owp9CgpzdGF0aWMg aW50IF9hY2NlcHQoaW50IGZkLCBpbnQgdHlwZSkgewogICAgaW50IHJldDsKICAgIGludCBsZW47 CiAgICBzdHJ1Y3Qgc29ja2FkZHIgKmFkZHIgPSBOVUxMOwoKICAgIHN3aXRjaCAodHlwZSkgewog ICAgY2FzZSBVTklYOgogICAgICAgIGFkZHIgPSAoc3RydWN0IHNvY2thZGRyICopICZyZW1vdGVf YWRkcl91bjsKICAgICAgICBsZW4gPSBzaXplb2YocmVtb3RlX2FkZHJfdW4pOwogICAgICAgIGJy ZWFrOwogICAgY2FzZSBSRkNPTU06CiAgICAgICAgYWRkciA9IChzdHJ1Y3Qgc29ja2FkZHIgKikg JnJlbW90ZV9hZGRyX3JjOwogICAgICAgIGxlbiA9IHNpemVvZihyZW1vdGVfYWRkcl9yYyk7CiAg ICAgICAgYnJlYWs7CiAgICBjYXNlIFNDTzoKICAgICAgICBhZGRyID0gKHN0cnVjdCBzb2NrYWRk ciAqKSAmcmVtb3RlX2FkZHJfc2NvOwogICAgICAgIGxlbiA9IHNpemVvZihyZW1vdGVfYWRkcl9z Y28pOwogICAgICAgIGJyZWFrOwogICAgY2FzZSBMMkNBUDoKICAgICAgICBhZGRyID0gKHN0cnVj dCBzb2NrYWRkciAqKSAmcmVtb3RlX2FkZHJfbDI7CiAgICAgICAgbGVuID0gc2l6ZW9mKHJlbW90 ZV9hZGRyX2wyKTsKICAgICAgICBicmVhazsKICAgIGNhc2UgVENQOgogICAgICAgIGFkZHIgPSAo c3RydWN0IHNvY2thZGRyICopICZyZW1vdGVfYWRkcl9pbjsKICAgICAgICBsZW4gPSBzaXplb2Yo cmVtb3RlX2FkZHJfaW4pOwogICAgICAgIGJyZWFrOwogICAgfQoKICAgIHByaW50ZigiJWQ6IGFj Y2VwdCglZClcbiIsIGdldHRpZCgpLCBmZCk7CiAgICByZXQgPSBhY2NlcHQoZmQsIGFkZHIsICZs ZW4pOwogICAgcHJpbnRmKCIlZDogYWNjZXB0KCVkKSA9ICVkXG4iLCBnZXR0aWQoKSwgZmQsIHJl dCk7CiAgICBpZiAocmV0IDwgMCkgcHJpbnRmKCJcdGVyciAlZCAoJXMpXG4iLCBlcnJubywgc3Ry ZXJyb3IoZXJybm8pKTsKICAgIGVsc2UgewogICAgICAgIHByaW50ZigiXHRsZW4gPSAlZFxuIiwg bGVuKTsKICAgIH0KCiAgICByZXR1cm4gcmV0Owp9CgpzdGF0aWMgaW50IF9zaHV0ZG93bihpbnQg ZmQsIGludCBob3cpIHsKICAgIGludCByZXQ7CgogICAgcHJpbnRmKCIlZDogc2h1dGRvd24oJWQp XG4iLCBnZXR0aWQoKSwgZmQpOwogICAgcmV0ID0gc2h1dGRvd24oZmQsIGhvdyk7CiAgICBwcmlu dGYoIiVkOiBzaHV0ZG93biglZCkgPSAlZFxuIiwgZ2V0dGlkKCksIGZkLCByZXQpOwogICAgaWYg KHJldCA8IDApIHByaW50ZigiXHRlcnIgJWQgKCVzKVxuIiwgZXJybm8sIHN0cmVycm9yKGVycm5v KSk7CgogICAgcmV0dXJuIHJldDsKfQoKc3RhdGljIHZvaWQgdGhyZWFkX2FjY2VwdChzdHJ1Y3Qg dGhyZWFkX2FyZ3MgKmFyZ3MpIHsKICAgIHByaW50ZigiJWQ6IFNUQVJUXG4iLCBnZXR0aWQoKSk7 CiAgICBzbGVlcChhcmdzLT5kZWxheSk7CiAgICBfYWNjZXB0KGFyZ3MtPmZkLCBhcmdzLT50eXBl KTsKICAgIHByaW50ZigiJWQ6IEVORFxuIiwgZ2V0dGlkKCkpOwp9CgpzdGF0aWMgaW50IGRvX2Fj Y2VwdF9zaHV0ZG93bihpbnQgdHlwZSkgewogICAgaW50IGZkOwogICAgcHRocmVhZF90IHRocmVh ZDsKICAgIHN0cnVjdCB0aHJlYWRfYXJncyBhcmdzID0gey0xLCB0eXBlLCAwfTsKCiAgICBmZCA9 IF9zb2NrZXQodHlwZSk7CiAgICBpZiAoZmQgPCAwKSBnb3RvIGVycm9yOwoKICAgIGlmIChfYmlu ZChmZCwgdHlwZSkgPCAwKSBnb3RvIGVycm9yOwoKICAgIGlmIChfbGlzdGVuKGZkLCB0eXBlKSA8 IDApIGdvdG8gZXJyb3I7CgogICAgYXJncy5mZCA9IGZkOwogICAgcHRocmVhZF9jcmVhdGUoJnRo cmVhZCwgTlVMTCwgKHZvaWQgKil0aHJlYWRfYWNjZXB0LCAodm9pZCAqKSZhcmdzKTsKCiAgICBz bGVlcCgyKTsKICAgIF9zaHV0ZG93bihmZCwgU0hVVF9SRFdSKTsKCiAgICBwdGhyZWFkX2pvaW4o dGhyZWFkLCBOVUxMKTsKCiAgICBfY2xvc2UoZmQpOwoKICAgIHJldHVybiAwOwoKZXJyb3I6CiAg ICByZXR1cm4gLTE7Cn0KCnN0cnVjdCB7CiAgICBjaGFyICpuYW1lOwogICAgaW50ICgqcHRyKShp bnQpOwp9IGFjdGlvbl90YWJsZVtdICA9IHsKICAgIHsiYWNjZXB0X3NodXRkb3duIiwgZG9fYWNj ZXB0X3NodXRkb3dufSwKICAgIHtOVUxMLCBOVUxMfSwKfTsKCnN0cnVjdCB7CiAgICBjaGFyICpu YW1lOwogICAgZW51bSBzb2NrX3R5cGUgdHlwZTsKfSB0eXBlX3RhYmxlW10gID0gewogICAgeyJ1 bml4IiwgVU5JWH0sCiAgICB7InJmY29tbSIsIFJGQ09NTX0sCiAgICB7InNjbyIsIFNDT30sCiAg ICB7ImwyY2FwIiwgTDJDQVB9LAogICAgeyJ0Y3AiLCBUQ1B9LAogICAge05VTEwsIC0xfSwKfTsK CnN0YXRpYyB2b2lkIHVzYWdlKCkgewogICAgaW50IGk7CgogICAgcHJpbnRmKCJzb2NrX3NodXRk b3duX3Rlc3QgVFlQRSBBQ1RJT05cbiIpOwogICAgcHJpbnRmKCJcblRZUEU6XG4iKTsKICAgIGZv ciAoaSA9IDA7IHR5cGVfdGFibGVbaV0ubmFtZTsgaSsrKSB7CiAgICAgICAgcHJpbnRmKCJcdCVz XG4iLCB0eXBlX3RhYmxlW2ldLm5hbWUpOwogICAgfQogICAgcHJpbnRmKCJcbkFDVElPTjpcbiIp OwogICAgZm9yIChpID0gMDsgYWN0aW9uX3RhYmxlW2ldLm5hbWU7IGkrKykgewogICAgICAgIHBy aW50ZigiXHQlc1xuIiwgYWN0aW9uX3RhYmxlW2ldLm5hbWUpOwogICAgfQp9CgppbnQgbWFpbihp bnQgYXJnYywgY2hhciAqKmFyZ3YpIHsKICAgIGludCBpOwogICAgaW50IHR5cGUgPSAtMTsKCiAg ICBpZiAoYXJnYyAhPSAzKSB7CiAgICAgICAgdXNhZ2UoKTsKICAgICAgICByZXR1cm4gLTE7CiAg ICB9CiAgICBmb3IgKGkgPSAwOyB0eXBlX3RhYmxlW2ldLm5hbWU7IGkrKykgewogICAgICAgIGlm ICghc3RyY21wKGFyZ3ZbMV0sIHR5cGVfdGFibGVbaV0ubmFtZSkpIHsKICAgICAgICAgICAgdHlw ZSA9IHR5cGVfdGFibGVbaV0udHlwZTsKICAgICAgICAgICAgYnJlYWs7CiAgICAgICAgfQogICAg fQogICAgaWYgKHR5cGUgPT0gLTEpIHsKICAgICAgICB1c2FnZSgpOwogICAgICAgIHJldHVybiAt MTsKICAgIH0KICAgIGZvciAoaSA9IDA7IGFjdGlvbl90YWJsZVtpXS5uYW1lOyBpKyspIHsKICAg ICAgICBpZiAoIXN0cmNtcChhcmd2WzJdLCBhY3Rpb25fdGFibGVbaV0ubmFtZSkpIHsKICAgICAg ICAgICAgcHJpbnRmKCJUWVBFID0gJXMgQUNUSU9OID0gJXNcbiIsIHR5cGVfdGFibGVbdHlwZV0u bmFtZSwKICAgICAgICAgICAgICAgICAgICBhY3Rpb25fdGFibGVbaV0ubmFtZSk7CiAgICAgICAg ICAgIHJldHVybiAoKmFjdGlvbl90YWJsZVtpXS5wdHIpKHR5cGUpOwogICAgICAgIH0KICAgIH0K ICAgIHVzYWdlKCk7CiAgICByZXR1cm4gLTE7Cn0K --0015174c33f0f87fd4046a4a9d6c--