From: Long Li <[email protected]>
It's inefficient to ring the doorbell page every time a WQE is posted to
the received queue.
Move the code for ringing doorbell page to where after we have posted all
WQEs to the receive queue during a callback from napi_poll().
Tests showed no regression in network latency benchmarks.
Cc: [email protected]
Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network Adapter (MANA)")
Signed-off-by: Long Li <[email protected]>
---
drivers/net/ethernet/microsoft/mana/mana_en.c | 10 ++++++++--
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c
index cd4d5ceb9f2d..ef1f0ce8e44d 100644
--- a/drivers/net/ethernet/microsoft/mana/mana_en.c
+++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
@@ -1383,8 +1383,8 @@ static void mana_post_pkt_rxq(struct mana_rxq *rxq)
recv_buf_oob = &rxq->rx_oobs[curr_index];
- err = mana_gd_post_and_ring(rxq->gdma_rq, &recv_buf_oob->wqe_req,
- &recv_buf_oob->wqe_inf);
+ err = mana_gd_post_work_request(rxq->gdma_rq, &recv_buf_oob->wqe_req,
+ &recv_buf_oob->wqe_inf);
if (WARN_ON_ONCE(err))
return;
@@ -1654,6 +1654,12 @@ static void mana_poll_rx_cq(struct mana_cq *cq)
mana_process_rx_cqe(rxq, cq, &comp[i]);
}
+ if (comp_read) {
+ struct gdma_context *gc = rxq->gdma_rq->gdma_dev->gdma_context;
+
+ mana_gd_wq_ring_doorbell(gc, rxq->gdma_rq);
+ }
+
if (rxq->xdp_flush)
xdp_do_flush();
}
--
2.34.1
> -----Original Message-----
> From: [email protected] <[email protected]>
> Sent: Thursday, June 15, 2023 7:28 PM
> To: KY Srinivasan <[email protected]>; Haiyang Zhang
> <[email protected]>; Wei Liu <[email protected]>; Dexuan Cui
> <[email protected]>; David S. Miller <[email protected]>; Eric
> Dumazet <[email protected]>; Jakub Kicinski <[email protected]>; Paolo
> Abeni <[email protected]>; Leon Romanovsky <[email protected]>; Shradha
> Gupta <[email protected]>; Ajay Sharma
> <[email protected]>; Shachar Raindel <[email protected]>;
> Stephen Hemminger <[email protected]>; linux-
> [email protected]; [email protected]; linux-
> [email protected]
> Cc: [email protected]; Long Li <[email protected]>;
> [email protected]
> Subject: [PATCH] net: mana: Batch ringing RX queue doorbell on receiving
> packets
>
> From: Long Li <[email protected]>
>
> It's inefficient to ring the doorbell page every time a WQE is posted to
> the received queue.
>
> Move the code for ringing doorbell page to where after we have posted all
> WQEs to the receive queue during a callback from napi_poll().
>
> Tests showed no regression in network latency benchmarks.
>
> Cc: [email protected]
> Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure Network
> Adapter (MANA)")
> Signed-off-by: Long Li <[email protected]>
> ---
> drivers/net/ethernet/microsoft/mana/mana_en.c | 10 ++++++++--
> 1 file changed, 8 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c
> b/drivers/net/ethernet/microsoft/mana/mana_en.c
> index cd4d5ceb9f2d..ef1f0ce8e44d 100644
> --- a/drivers/net/ethernet/microsoft/mana/mana_en.c
> +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
> @@ -1383,8 +1383,8 @@ static void mana_post_pkt_rxq(struct mana_rxq
> *rxq)
>
> recv_buf_oob = &rxq->rx_oobs[curr_index];
>
> - err = mana_gd_post_and_ring(rxq->gdma_rq, &recv_buf_oob-
> >wqe_req,
> - &recv_buf_oob->wqe_inf);
> + err = mana_gd_post_work_request(rxq->gdma_rq, &recv_buf_oob-
> >wqe_req,
> + &recv_buf_oob->wqe_inf);
> if (WARN_ON_ONCE(err))
> return;
>
> @@ -1654,6 +1654,12 @@ static void mana_poll_rx_cq(struct mana_cq *cq)
> mana_process_rx_cqe(rxq, cq, &comp[i]);
> }
>
> + if (comp_read) {
> + struct gdma_context *gc = rxq->gdma_rq->gdma_dev-
> >gdma_context;
> +
> + mana_gd_wq_ring_doorbell(gc, rxq->gdma_rq);
> + }
> +
Thank you!
Reviewed-by: Haiyang Zhang <[email protected]>
Hi,
I'm sending v2 to address some corner cases discovered during tests.
Thanks,
Long
> -----Original Message-----
> From: Haiyang Zhang <[email protected]>
> Sent: Friday, June 16, 2023 9:49 AM
> To: [email protected]; KY Srinivasan <[email protected]>; Wei Liu
> <[email protected]>; Dexuan Cui <[email protected]>; David S. Miller
> <[email protected]>; Eric Dumazet <[email protected]>; Jakub
> Kicinski <[email protected]>; Paolo Abeni <[email protected]>; Leon
> Romanovsky <[email protected]>; Shradha Gupta
> <[email protected]>; Ajay Sharma
> <[email protected]>; Shachar Raindel <[email protected]>;
> Stephen Hemminger <[email protected]>; linux-
> [email protected]; [email protected]; linux-
> [email protected]
> Cc: [email protected]; Long Li <[email protected]>;
> [email protected]
> Subject: RE: [PATCH] net: mana: Batch ringing RX queue doorbell on receiving
> packets
>
>
>
> > -----Original Message-----
> > From: [email protected] <[email protected]>
> > Sent: Thursday, June 15, 2023 7:28 PM
> > To: KY Srinivasan <[email protected]>; Haiyang Zhang
> > <[email protected]>; Wei Liu <[email protected]>; Dexuan Cui
> > <[email protected]>; David S. Miller <[email protected]>; Eric
> > Dumazet <[email protected]>; Jakub Kicinski <[email protected]>;
> Paolo
> > Abeni <[email protected]>; Leon Romanovsky <[email protected]>;
> Shradha
> > Gupta <[email protected]>; Ajay Sharma
> > <[email protected]>; Shachar Raindel <[email protected]>;
> > Stephen Hemminger <[email protected]>; linux-
> > [email protected]; [email protected]; linux-
> > [email protected]
> > Cc: [email protected]; Long Li <[email protected]>;
> > [email protected]
> > Subject: [PATCH] net: mana: Batch ringing RX queue doorbell on
> > receiving packets
> >
> > From: Long Li <[email protected]>
> >
> > It's inefficient to ring the doorbell page every time a WQE is posted
> > to the received queue.
> >
> > Move the code for ringing doorbell page to where after we have posted
> > all WQEs to the receive queue during a callback from napi_poll().
> >
> > Tests showed no regression in network latency benchmarks.
> >
> > Cc: [email protected]
> > Fixes: ca9c54d2d6a5 ("net: mana: Add a driver for Microsoft Azure
> > Network Adapter (MANA)")
> > Signed-off-by: Long Li <[email protected]>
> > ---
> > drivers/net/ethernet/microsoft/mana/mana_en.c | 10 ++++++++--
> > 1 file changed, 8 insertions(+), 2 deletions(-)
> >
> > diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c
> > b/drivers/net/ethernet/microsoft/mana/mana_en.c
> > index cd4d5ceb9f2d..ef1f0ce8e44d 100644
> > --- a/drivers/net/ethernet/microsoft/mana/mana_en.c
> > +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c
> > @@ -1383,8 +1383,8 @@ static void mana_post_pkt_rxq(struct mana_rxq
> > *rxq)
> >
> > recv_buf_oob = &rxq->rx_oobs[curr_index];
> >
> > - err = mana_gd_post_and_ring(rxq->gdma_rq, &recv_buf_oob-
> > >wqe_req,
> > - &recv_buf_oob->wqe_inf);
> > + err = mana_gd_post_work_request(rxq->gdma_rq, &recv_buf_oob-
> > >wqe_req,
> > + &recv_buf_oob->wqe_inf);
> > if (WARN_ON_ONCE(err))
> > return;
> >
> > @@ -1654,6 +1654,12 @@ static void mana_poll_rx_cq(struct mana_cq
> *cq)
> > mana_process_rx_cqe(rxq, cq, &comp[i]);
> > }
> >
> > + if (comp_read) {
> > + struct gdma_context *gc = rxq->gdma_rq->gdma_dev-
> > >gdma_context;
> > +
> > + mana_gd_wq_ring_doorbell(gc, rxq->gdma_rq);
> > + }
> > +
>
> Thank you!
>
> Reviewed-by: Haiyang Zhang <[email protected]>