Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
Message-ID: <4c524eba575992cc1adfc41b2b230835946b126c.camel@gmail.com>
Subject: Re: [PATCH v2 net-next] ipv6: avoid atomic fragment on GSO packets
From:   Alexander H Duyck <alexander.duyck@gmail.com>
To:     Yan Zhai <yan@cloudflare.com>, netdev@vger.kernel.org
Cc:     "David S. Miller" <davem@davemloft.net>,
        David Ahern <dsahern@kernel.org>,
        Eric Dumazet <edumazet@google.com>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>, Aya Levin <ayal@nvidia.com>,
        Tariq Toukan <tariqt@nvidia.com>, linux-kernel@vger.kernel.org,
        kernel-team@cloudflare.com, Florian Westphal <fw@strlen.de>,
        Willem de Bruijn <willemdebruijn.kernel@gmail.com>
Date:   Mon, 16 Oct 2023 14:00:28 -0700
In-Reply-To: <ZS1/qtr0dZJ35VII@debian.debian>
References: <ZS1/qtr0dZJ35VII@debian.debian>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.48.4 (3.48.4-1.fc38) 
MIME-Version: 1.0
Precedence: bulk

On Mon, 2023-10-16 at 11:23 -0700, Yan Zhai wrote:
> GSO packets can contain a trailing segment that is smaller than
> gso_size. When examining the dst MTU for such packet, if its gso_size is
> too large, then all segments would be fragmented. However, there is a
> good chance the trailing segment has smaller actual size than both
> gso_size as well as the MTU, which leads to an "atomic fragment". It is
> considered harmful in RFC-8021. An Existing report from APNIC also shows
> that atomic fragments are more likely to be dropped even it is
> equivalent to a no-op [1].
>=20
> Refactor __ip6_finish_output code to separate GSO and non-GSO packet
> processing. It mirrors __ip_finish_output logic now. Add an extra check
> in GSO handling to avoid atomic fragments. Lastly, drop dst_allfrag
> check, which is no longer true since commit 9d289715eb5c ("ipv6: stop
> sending PTB packets for MTU < 1280").
>=20
> Link: https://www.potaroo.net/presentations/2022-03-01-ipv6-frag.pdf [1]
> Fixes: b210de4f8c97 ("net: ipv6: Validate GSO SKB before finish IPv6 proc=
essing")
> Suggested-by: Florian Westphal <fw@strlen.de>
> Reported-by: David Wragg <dwragg@cloudflare.com>
> Signed-off-by: Yan Zhai <yan@cloudflare.com>
> ---
>  net/ipv6/ip6_output.c | 33 +++++++++++++++++++++++----------
>  1 file changed, 23 insertions(+), 10 deletions(-)
>=20
> diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
> index a471c7e91761..1de6f3c11655 100644
> --- a/net/ipv6/ip6_output.c
> +++ b/net/ipv6/ip6_output.c
> @@ -162,7 +162,14 @@ ip6_finish_output_gso_slowpath_drop(struct net *net,=
 struct sock *sk,
>  		int err;
> =20
>  		skb_mark_not_on_list(segs);
> -		err =3D ip6_fragment(net, sk, segs, ip6_finish_output2);
> +		/* Last gso segment might be smaller than actual MTU. Adding
> +		 * a fragment header to it would produce an "atomic fragment",
> +		 * which is considered harmful (RFC-8021)
> +		 */
> +		err =3D segs->len > mtu ?
> +			ip6_fragment(net, sk, segs, ip6_finish_output2) :
> +			ip6_finish_output2(net, sk, segs);
> +
>  		if (err && ret =3D=3D 0)
>  			ret =3D err;
>  	}
> @@ -170,10 +177,19 @@ ip6_finish_output_gso_slowpath_drop(struct net *net=
, struct sock *sk,
>  	return ret;
>  }
> =20
> +static int ip6_finish_output_gso(struct net *net, struct sock *sk,
> +				 struct sk_buff *skb, unsigned int mtu)
> +{
> +	if (!(IP6CB(skb)->flags & IP6SKB_FAKEJUMBO) &&
> +	    !skb_gso_validate_network_len(skb, mtu))
> +		return ip6_finish_output_gso_slowpath_drop(net, sk, skb, mtu);

If we are sending fakejumbo or have a frame that doesn't pass the
muster it is just going immediately to ip6_finish_output. I think the
checks that you removed are needed to keep the socket from getting
stuck sending frames that will probably be discarded.

> +
> +	return ip6_finish_output2(net, sk, skb);
> +}
> +
>  static int __ip6_finish_output(struct net *net, struct sock *sk, struct =
sk_buff *skb)
>  {
>  	unsigned int mtu;
> -

This blank line can probably be left there to separate variable
declarations from code.

>  #if defined(CONFIG_NETFILTER) && defined(CONFIG_XFRM)
>  	/* Policy lookup after SNAT yielded a new policy */
>  	if (skb_dst(skb)->xfrm) {
> @@ -183,17 +199,14 @@ static int __ip6_finish_output(struct net *net, str=
uct sock *sk, struct sk_buff
>  #endif
> =20
>  	mtu =3D ip6_skb_dst_mtu(skb);
> -	if (skb_is_gso(skb) &&
> -	    !(IP6CB(skb)->flags & IP6SKB_FAKEJUMBO) &&
> -	    !skb_gso_validate_network_len(skb, mtu))
> -		return ip6_finish_output_gso_slowpath_drop(net, sk, skb, mtu);
> +	if (skb_is_gso(skb))
> +		return ip6_finish_output_gso(net, sk, skb, mtu);
> =20
> -	if ((skb->len > mtu && !skb_is_gso(skb)) ||
> -	    dst_allfrag(skb_dst(skb)) ||
> +	if (skb->len > mtu ||

This change looks a bit too aggressive to me. Basically if the frame is
gso you now bypass the ip6_fragment entirely and are ignoring the
dst_allfrag and frag_max_size case below. See the fail_toobig code in
ip6_fragment.

>  	    (IP6CB(skb)->frag_max_size && skb->len > IP6CB(skb)->frag_max_size)=
)
>  		return ip6_fragment(net, sk, skb, ip6_finish_output2);
> -	else
> -		return ip6_finish_output2(net, sk, skb);
> +
> +	return ip6_finish_output2(net, sk, skb);
>  }
> =20
>  static int ip6_finish_output(struct net *net, struct sock *sk, struct sk=
_buff *skb)