Message-ID: <6ded592b01ba223bce241d6ff3073246cb5dd18b.camel@gmail.com>
Subject: Re: [PATCH net v4 1/3] ixgbe: allow to increase MTU to 3K with XDP
 enabled
From:   Alexander H Duyck <alexander.duyck@gmail.com>
To:     Maciej Fijalkowski <maciej.fijalkowski@intel.com>
Cc:     Jason Xing <kerneljasonxing@gmail.com>, jesse.brandeburg@intel.com,
        anthony.l.nguyen@intel.com, davem@davemloft.net,
        edumazet@google.com, kuba@kernel.org, pabeni@redhat.com,
        richardcochran@gmail.com, ast@kernel.org, daniel@iogearbox.net,
        hawk@kernel.org, john.fastabend@gmail.com,
        alexandr.lobakin@intel.com, intel-wired-lan@lists.osuosl.org,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        bpf@vger.kernel.org, Jason Xing <kernelxing@tencent.com>
Date:   Wed, 08 Feb 2023 10:57:22 -0800
In-Reply-To: <Y+PNjcrSxKc0vD3s@boxer>
References: <20230208024333.10465-1-kerneljasonxing@gmail.com>
         <2bfcd7d92a6971416f58d9aac6e74840d5ae240a.camel@gmail.com>
         <Y+PNjcrSxKc0vD3s@boxer>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
User-Agent: Evolution 3.44.4 (3.44.4-2.fc36) 
MIME-Version: 1.0
Precedence: bulk

On Wed, 2023-02-08 at 17:27 +0100, Maciej Fijalkowski wrote:
> On Wed, Feb 08, 2023 at 07:37:57AM -0800, Alexander H Duyck wrote:
> > On Wed, 2023-02-08 at 10:43 +0800, Jason Xing wrote:
> > > From: Jason Xing <kernelxing@tencent.com>
> > >=20
> > > Recently I encountered one case where I cannot increase the MTU size
> > > directly from 1500 to a much bigger value with XDP enabled if the
> > > server is equipped with IXGBE card, which happened on thousands of
> > > servers in production environment. After appling the current patch,
> > > we can set the maximum MTU size to 3K.
> > >=20
> > > This patch follows the behavior of changing MTU as i40e/ice does.
> > >=20
> > > Referrences:
> > > [1] commit 23b44513c3e6 ("ice: allow 3k MTU for XDP")
> > > [2] commit 0c8493d90b6b ("i40e: add XDP support for pass and drop act=
ions")
> > >=20
> > > Fixes: fabf1bce103a ("ixgbe: Prevent unsupported configurations with =
XDP")
> > > Signed-off-by: Jason Xing <kernelxing@tencent.com>
> >=20
> > This is based on the broken premise that w/ XDP we are using a 4K page.
> > The ixgbe driver isn't using page pool and is therefore running on
> > different limitations. The ixgbe driver is only using 2K slices of the
> > 4K page. In addition that is reduced to 1.5K to allow for headroom and
> > the shared info in the buffer.
> >=20
> > Currently the only way a 3K buffer would work is if FCoE is enabled and
> > in that case the driver is using order 1 pages and still using the
> > split buffer approach.
>=20
> Hey Alex, interesting, we based this on the following logic from
> ixgbe_set_rx_buffer_len() I guess:
>=20
> #if (PAGE_SIZE < 8192)
> 		if (adapter->flags2 & IXGBE_FLAG2_RSC_ENABLED)
> 			set_bit(__IXGBE_RX_3K_BUFFER, &rx_ring->state);
>=20
> 		if (IXGBE_2K_TOO_SMALL_WITH_PADDING ||
> 		    (max_frame > (ETH_FRAME_LEN + ETH_FCS_LEN)))
> 			set_bit(__IXGBE_RX_3K_BUFFER, &rx_ring->state);
> #endif
>=20
> so we assumed that ixgbe is no different than i40e/ice in these terms, bu=
t
> we ignored whole overhead of LRO/RSC that ixgbe carries.

If XDP is already enabled the LRO/RSC cannot be enabled. I think that
is already disabled if we have XDP enabled.

> I am not actively working with ixgbe but I know that you were the main de=
v
> of it, so without premature dive into the datasheet and codebase, are you
> really sure that 3k mtu for XDP is a no go?

I think I mixed up fm10k and ixgbe, either that or I was thinking of
the legacy setup. They all kind of blur together as I had worked on
pretty much all the Intel drivers up to i40e the last time I was
updating them for all the Rx path stuff. :)

So if I am reading things right the issue is that if XDP is enabled you
cannot set a 3K MTU, but if you set the 3K MTU first then you can
enable XDP after the fact right?

Looking it over again after re-reading the code this looks good to me.

Reviewed-by: Alexander Duyck <alexanderduyck@fb.com>