2020-11-26 12:56:04

by Sven Eckelmann

Subject: [PATCH 1/2] vxlan: Add needed_headroom for lower device

It was observed that sending data via batadv over vxlan (on top of
wireguard) reduced the performance massively compared to raw ethernet or
batadv on raw ethernet. A check of perf data showed that the
vxlan_build_skb was calling all the time pskb_expand_head to allocate
enough headroom for:

min_headroom = LL_RESERVED_SPACE(dst->dev) + dst->header_len
+ VXLAN_HLEN + iphdr_len;

But the vxlan_config_apply only requested needed headroom for:

lowerdev->hard_header_len + VXLAN6_HEADROOM or VXLAN_HEADROOM

So it completely ignored the needed_headroom of the lower device. The first
caller of net_dev_xmit could therefore never make sure that enough headroom
was allocated for the rest of the transmit path.

Cc: Annika Wickert <[email protected]>
Signed-off-by: Sven Eckelmann <[email protected]>
---
drivers/net/vxlan.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/drivers/net/vxlan.c b/drivers/net/vxlan.c
index 236fcc55a5fd..25b5b5a2dfea 100644
--- a/drivers/net/vxlan.c
+++ b/drivers/net/vxlan.c
@@ -3799,6 +3799,7 @@ static void vxlan_config_apply(struct net_device *dev,
dev->gso_max_segs = lowerdev->gso_max_segs;

needed_headroom = lowerdev->hard_header_len;
+ needed_headroom += lowerdev->needed_headroom;

max_mtu = lowerdev->mtu - (use_ipv6 ? VXLAN6_HEADROOM :
VXLAN_HEADROOM);
--
2.29.2

2020-11-26 12:56:37

by Sven Eckelmann

Subject: [PATCH 2/2] vxlan: Copy needed_tailroom from lowerdev

While vxlan doesn't need any extra tailroom, the lowerdev might need it. In
that case, copy it over to reduce the chance for additional (re)allocations
in the transmit path.

Signed-off-by: Sven Eckelmann <[email protected]>
---
drivers/net/vxlan.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/drivers/net/vxlan.c b/drivers/net/vxlan.c
index 25b5b5a2dfea..44bb02122526 100644
--- a/drivers/net/vxlan.c
+++ b/drivers/net/vxlan.c
@@ -3801,6 +3801,8 @@ static void vxlan_config_apply(struct net_device *dev,
needed_headroom = lowerdev->hard_header_len;
needed_headroom += lowerdev->needed_headroom;

+ dev->needed_tailroom = lowerdev->needed_tailroom;
+
max_mtu = lowerdev->mtu - (use_ipv6 ? VXLAN6_HEADROOM :
VXLAN_HEADROOM);
if (max_mtu < ETH_MIN_MTU)
--
2.29.2

2020-11-26 18:22:41

by Annika Wickert

Subject: Re: [PATCH 1/2] vxlan: Add needed_headroom for lower device

Tested-by: Annika Wickert <[email protected]>


On 26.11.20, 13:53, "Sven Eckelmann" <[email protected]> wrote:

It was observed that sending data via batadv over vxlan (on top of
wireguard) reduced the performance massively compared to raw ethernet or
batadv on raw ethernet. A check of perf data showed that the
vxlan_build_skb was calling all the time pskb_expand_head to allocate
enough headroom for:

min_headroom = LL_RESERVED_SPACE(dst->dev) + dst->header_len
+ VXLAN_HLEN + iphdr_len;

But the vxlan_config_apply only requested needed headroom for:

lowerdev->hard_header_len + VXLAN6_HEADROOM or VXLAN_HEADROOM

So it completely ignored the needed_headroom of the lower device. The first
caller of net_dev_xmit could therefore never make sure that enough headroom
was allocated for the rest of the transmit path.

Cc: Annika Wickert <[email protected]>
Signed-off-by: Sven Eckelmann <[email protected]>
---
drivers/net/vxlan.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/drivers/net/vxlan.c b/drivers/net/vxlan.c
index 236fcc55a5fd..25b5b5a2dfea 100644
--- a/drivers/net/vxlan.c
+++ b/drivers/net/vxlan.c
@@ -3799,6 +3799,7 @@ static void vxlan_config_apply(struct net_device *dev,
dev->gso_max_segs = lowerdev->gso_max_segs;

needed_headroom = lowerdev->hard_header_len;
+ needed_headroom += lowerdev->needed_headroom;

max_mtu = lowerdev->mtu - (use_ipv6 ? VXLAN6_HEADROOM :
VXLAN_HEADROOM);
--
2.29.2

2020-12-01 02:44:42

by Jakub Kicinski

Subject: Re: [PATCH 1/2] vxlan: Add needed_headroom for lower device

On Thu, 26 Nov 2020 13:52:46 +0100 Sven Eckelmann wrote:
> It was observed that sending data via batadv over vxlan (on top of
> wireguard) reduced the performance massively compared to raw ethernet or
> batadv on raw ethernet. A check of perf data showed that the
> vxlan_build_skb was calling all the time pskb_expand_head to allocate
> enough headroom for:
>
> min_headroom = LL_RESERVED_SPACE(dst->dev) + dst->header_len
> + VXLAN_HLEN + iphdr_len;
>
> But the vxlan_config_apply only requested needed headroom for:
>
> lowerdev->hard_header_len + VXLAN6_HEADROOM or VXLAN_HEADROOM
>
> So it completely ignored the needed_headroom of the lower device. The first
> caller of net_dev_xmit could therefore never make sure that enough headroom
> was allocated for the rest of the transmit path.
>
> Cc: Annika Wickert <[email protected]>
> Signed-off-by: Sven Eckelmann <[email protected]>

Applied both, thanks!