Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <5099dc39-c6d9-115a-855b-6aa98d17eb4b@collabora.com>
Date:   Tue, 24 May 2022 13:18:55 +0500
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.9.0
From:   Muhammad Usama Anjum <usama.anjum@collabora.com>
Subject: [RFC] EADDRINUSE from bind() on application restart after killing
Cc:     usama.anjum@collabora.com,
        Gabriel Krisman Bertazi <krisman@collabora.com>,
        LKML <linux-kernel@vger.kernel.org>,
        open list <linux-kernel@vger.kernel.org>
Content-Language: en-US
To:     Eric Dumazet <edumazet@google.com>,
        "David S. Miller" <davem@davemloft.net>,
        Hideaki YOSHIFUJI <yoshfuji@linux-ipv6.org>,
        David Ahern <dsahern@kernel.org>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        "open list:NETWORKING [TCP]" <netdev@vger.kernel.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hello,

We have a set of processes which talk with each other through a local
TCP socket. If the process(es) are killed (through SIGKILL) and
restarted at once, the bind() fails with EADDRINUSE error. This error
only appears if application is restarted at once without waiting for 60
seconds or more. It seems that there is some timeout of 60 seconds for
which the previous TCP connection remains alive waiting to get closed
completely. In that duration if we try to connect again, we get the error.

We are able to avoid this error by adding SO_REUSEADDR attribute to the
socket in a hack. But this hack cannot be added to the application
process as we don't own it.

I've looked at the TCP connection states after killing processes in
different ways. The TCP connection ends up in 2 different states with
timeouts:

(1) Timeout associated with FIN_WAIT_1 state which is set through
`tcp_fin_timeout` in procfs (60 seconds by default)

(2) Timeout associated with TIME_WAIT state which cannot be changed. It
seems like this timeout has come from RFC 1337.

The timeout in (1) can be changed. Timeout in (2) cannot be changed. It
also doesn't seem feasible to change the timeout of TIME_WAIT state as
the RFC mentions several hazards. But we are talking about a local TCP
connection where maybe those hazards aren't applicable directly? Is it
possible to change timeout for TIME_WAIT state for only local
connections without any hazards?

We have tested a hack where we replace timeout of TIME_WAIT state from a
value in procfs for local connections. This solves our problem and
application starts to work without any modifications to it.

The question is that what can be the best possible solution here? Any
thoughts will be very helpful.

Regards,

-- 
Muhammad Usama Anjum