Received-SPF: pass (google.com: domain of linux-kernel+bounces-71842-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
From: Kuniyuki Iwashima <kuniyu@amazon.com>
To: <vschneid@redhat.com>
CC: <bigeasy@linutronix.de>, <davem@davemloft.net>, <dccp@vger.kernel.org>,
	<dsahern@kernel.org>, <edumazet@google.com>, <juri.lelli@redhat.com>,
	<kuba@kernel.org>, <linux-kernel@vger.kernel.org>,
	<linux-rt-users@vger.kernel.org>, <mleitner@redhat.com>,
	<netdev@vger.kernel.org>, <pabeni@redhat.com>, <tglozar@redhat.com>,
	<tglx@linutronix.de>, <kuniyu@amazon.com>
Subject: Re: [PATCH v3 1/1] tcp/dcpp: Un-pin tw_timer
Date: Mon, 19 Feb 2024 10:55:37 -0800
Message-ID: <20240219185537.13666-1-kuniyu@amazon.com>
In-Reply-To: <20240219095729.2339914-2-vschneid@redhat.com>
References: <20240219095729.2339914-2-vschneid@redhat.com>
Precedence: bulk
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain

From: Valentin Schneider <vschneid@redhat.com>
Date: Mon, 19 Feb 2024 10:57:29 +0100
> The TCP timewait timer is proving to be problematic for setups where scheduler
> CPU isolation is achieved at runtime via cpusets (as opposed to statically via
> isolcpus=domains).
> 
> What happens there is a CPU goes through tcp_time_wait(), arming the time_wait
> timer, then gets isolated. TCP_TIMEWAIT_LEN later, the timer fires, causing
> interference for the now-isolated CPU. This is conceptually similar to the issue
> described in
>   e02b93124855 ("workqueue: Unbind kworkers before sending them to exit()")
> 
> Keep softirqs disabled, but make the timer un-pinned and arm it *after* the
> hashdance.
> 
> This introduces the following (non-fatal) race:
> 
>   CPU0                        CPU1
>     allocates a tw
>     insert it in hash table
> 				finds the TW and removes it
> 				(timer cancel does nothing)
>     arms a TW timer, lasting
> 
> This partially reverts
>   ed2e92394589 ("tcp/dccp: fix timewait races in timer handling")
> and
>   ec94c2696f0b ("tcp/dccp: avoid one atomic operation for timewait hashdance")
> 
> This also reinstores a comment from
>   ec94c2696f0b ("tcp/dccp: avoid one atomic operation for timewait hashdance")
> as inet_twsk_hashdance() had a "Step 1" and "Step 3" comment, but the "Step
> 2" had gone missing.
> 
> Link: https://lore.kernel.org/all/ZPhpfMjSiHVjQkTk@localhost.localdomain/
> Signed-off-by: Valentin Schneider <vschneid@redhat.com>
> ---
>  net/dccp/minisocks.c          | 16 +++++++---------
>  net/ipv4/inet_timewait_sock.c | 20 +++++++++++++++-----
>  net/ipv4/tcp_minisocks.c      | 16 +++++++---------
>  3 files changed, 29 insertions(+), 23 deletions(-)
> 
> diff --git a/net/dccp/minisocks.c b/net/dccp/minisocks.c
> index 64d805b27adde..2f0fad4255e36 100644
> --- a/net/dccp/minisocks.c
> +++ b/net/dccp/minisocks.c
> @@ -53,16 +53,14 @@ void dccp_time_wait(struct sock *sk, int state, int timeo)
>  		if (state == DCCP_TIME_WAIT)
>  			timeo = DCCP_TIMEWAIT_LEN;
>  
> -		/* tw_timer is pinned, so we need to make sure BH are disabled
> -		 * in following section, otherwise timer handler could run before
> -		 * we complete the initialization.
> -		 */
> -		local_bh_disable();
> -		inet_twsk_schedule(tw, timeo);
> -		/* Linkage updates.
> -		 * Note that access to tw after this point is illegal.
> -		 */
> +	       local_bh_disable();

This line seems not correctly indented, same for TCP change.


> +
> +		// Linkage updates
>  		inet_twsk_hashdance(tw, sk, &dccp_hashinfo);
> +		inet_twsk_schedule(tw, timeo);
> +		// Access to tw after this point is illegal.

Also please use /**/ style for these comments, same for TCP too.


> +		inet_twsk_put(tw);
> +
>  		local_bh_enable();
>  	} else {
>  		/* Sorry, if we're out of memory, just CLOSE this