From: Matteo Croce <[email protected]>
These two patches prefetch some data from RAM so to reduce stall
and speedup the packet processing.
Matteo Croce (2):
mvpp2: prefetch right address
mvpp2: prefetch page
drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 14 +++++++++-----
1 file changed, 9 insertions(+), 5 deletions(-)
--
2.31.1
From: Matteo Croce <[email protected]>
Most of the time during the RX is caused by the compound_head() call
done at the end of the RX loop:
│ build_skb():
[...]
│ static inline struct page *compound_head(struct page *page)
│ {
│ unsigned long head = READ_ONCE(page->compound_head);
65.23 │ ldr x2, [x1, #8]
Prefetch the page struct as soon as possible, to speedup the RX path
noticeabily by a ~3-4% packet rate in a drop test.
│ build_skb():
[...]
│ static inline struct page *compound_head(struct page *page)
│ {
│ unsigned long head = READ_ONCE(page->compound_head);
17.92 │ ldr x2, [x1, #8]
Signed-off-by: Matteo Croce <[email protected]>
---
drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
index 07d8f3e31b52..9bca8c8f9f8d 100644
--- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
+++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
@@ -3900,15 +3900,19 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi,
phys_addr_t phys_addr;
u32 rx_status, timestamp;
int pool, rx_bytes, err, ret;
+ struct page *page;
void *data;
+ phys_addr = mvpp2_rxdesc_cookie_get(port, rx_desc);
+ data = (void *)phys_to_virt(phys_addr);
+ page = virt_to_page(data);
+ prefetch(page);
+
rx_done++;
rx_status = mvpp2_rxdesc_status_get(port, rx_desc);
rx_bytes = mvpp2_rxdesc_size_get(port, rx_desc);
rx_bytes -= MVPP2_MH_SIZE;
dma_addr = mvpp2_rxdesc_dma_addr_get(port, rx_desc);
- phys_addr = mvpp2_rxdesc_cookie_get(port, rx_desc);
- data = (void *)phys_to_virt(phys_addr);
pool = (rx_status & MVPP2_RXD_BM_POOL_ID_MASK) >>
MVPP2_RXD_BM_POOL_ID_OFFS;
@@ -3997,7 +4001,7 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi,
}
if (pp)
- skb_mark_for_recycle(skb, virt_to_page(data), pp);
+ skb_mark_for_recycle(skb, page, pp);
else
dma_unmap_single_attrs(dev->dev.parent, dma_addr,
bm_pool->buf_size, DMA_FROM_DEVICE,
--
2.31.1
Hi Matteo,
śr., 9 cze 2021 o 15:47 Matteo Croce <[email protected]> napisał(a):
>
> From: Matteo Croce <[email protected]>
>
> These two patches prefetch some data from RAM so to reduce stall
> and speedup the packet processing.
>
> Matteo Croce (2):
> mvpp2: prefetch right address
> mvpp2: prefetch page
>
> drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 14 +++++++++-----
> 1 file changed, 9 insertions(+), 5 deletions(-)
>
Thank you for the patches, they seem reasonable, however I'd like to
stress it on the CN913x setup @10G - I should have some slot for that
closer to EOW.
Best regards,
Marcin
From: Matteo Croce <[email protected]>
In the RX buffer, the received data starts after a headroom used to
align the IP header and to allow prepending headers efficiently.
The prefetch() should take this into account, and prefetch from
the very start of the received data.
We can see that ether_addr_equal_64bits(), which is the first function
to access the data, drops from the top of the perf top output.
prefetch(data):
Overhead Shared Object Symbol
11.64% [kernel] [k] eth_type_trans
prefetch(data + MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM):
Overhead Shared Object Symbol
13.42% [kernel] [k] build_skb
10.35% [mvpp2] [k] mvpp2_rx
9.35% [kernel] [k] __netif_receive_skb_core
8.24% [kernel] [k] kmem_cache_free
7.97% [kernel] [k] dev_gro_receive
7.68% [kernel] [k] page_pool_put_page
7.32% [kernel] [k] kmem_cache_alloc
7.09% [mvpp2] [k] mvpp2_bm_pool_put
3.36% [kernel] [k] eth_type_trans
Also, move the eth_type_trans() call a bit down, to give the RAM more
time to prefetch the data.
Signed-off-by: Matteo Croce <[email protected]>
---
drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
index 5663c1b21870..07d8f3e31b52 100644
--- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
+++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
@@ -3938,7 +3938,7 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi,
goto err_drop_frame;
/* Prefetch header */
- prefetch(data);
+ prefetch(data + MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM);
if (bm_pool->frag_size > PAGE_SIZE)
frag_size = 0;
@@ -4008,8 +4008,8 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi,
skb_reserve(skb, MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM);
skb_put(skb, rx_bytes);
- skb->protocol = eth_type_trans(skb, dev);
mvpp2_rx_csum(port, rx_status, skb);
+ skb->protocol = eth_type_trans(skb, dev);
napi_gro_receive(napi, skb);
continue;
--
2.31.1
Hello:
This series was applied to netdev/net-next.git (refs/heads/master):
On Wed, 9 Jun 2021 15:47:12 +0200 you wrote:
> From: Matteo Croce <[email protected]>
>
> These two patches prefetch some data from RAM so to reduce stall
> and speedup the packet processing.
>
> Matteo Croce (2):
> mvpp2: prefetch right address
> mvpp2: prefetch page
>
> [...]
Here is the summary with links:
- [net-next,1/2] mvpp2: prefetch right address
https://git.kernel.org/netdev/net-next/c/d8ea89fe8a49
- [net-next,2/2] mvpp2: prefetch page
https://git.kernel.org/netdev/net-next/c/2f128eb3308a
You are awesome, thank you!
--
Deet-doot-dot, I am a bot.
https://korg.docs.kernel.org/patchwork/pwbot.html