Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 24 Mar 2021 23:48:25 +0200
From:   Ilias Apalodimas <ilias.apalodimas@linaro.org>
To:     Lorenzo Bianconi <lorenzo.bianconi@redhat.com>
Cc:     Jesper Dangaard Brouer <brouer@redhat.com>,
        Matteo Croce <mcroce@linux.microsoft.com>,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        Jonathan Lemon <jonathan.lemon@gmail.com>,
        "David S. Miller" <davem@davemloft.net>,
        Jesper Dangaard Brouer <hawk@kernel.org>,
        Lorenzo Bianconi <lorenzo@kernel.org>,
        Saeed Mahameed <saeedm@nvidia.com>,
        David Ahern <dsahern@gmail.com>,
        Saeed Mahameed <saeed@kernel.org>, Andrew Lunn <andrew@lunn.ch>
Subject: Re: [PATCH net-next 6/6] mvneta: recycle buffers
Message-ID: <YFuzqWq/CQ6a9SsU@enceladus>
References: <20210322170301.26017-1-mcroce@linux.microsoft.com>
 <20210322170301.26017-7-mcroce@linux.microsoft.com>
 <20210323160611.28ddc712@carbon>
 <YFsGQx6XMbYRtBOR@lore-desk>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YFsGQx6XMbYRtBOR@lore-desk>
Precedence: bulk

On Wed, Mar 24, 2021 at 10:28:35AM +0100, Lorenzo Bianconi wrote:
> [...]
> > > diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c
> > > index a635cf84608a..8b3250394703 100644
> > > --- a/drivers/net/ethernet/marvell/mvneta.c
> > > +++ b/drivers/net/ethernet/marvell/mvneta.c
> > > @@ -2332,7 +2332,7 @@ mvneta_swbm_build_skb(struct mvneta_port *pp, struct mvneta_rx_queue *rxq,
> > >  	if (!skb)
> > >  		return ERR_PTR(-ENOMEM);
> > >  
> > > -	page_pool_release_page(rxq->page_pool, virt_to_page(xdp->data));
> > > +	skb_mark_for_recycle(skb, virt_to_page(xdp->data), &xdp->rxq->mem);
> > >  
> > >  	skb_reserve(skb, xdp->data - xdp->data_hard_start);
> > >  	skb_put(skb, xdp->data_end - xdp->data);
> > > @@ -2344,7 +2344,7 @@ mvneta_swbm_build_skb(struct mvneta_port *pp, struct mvneta_rx_queue *rxq,
> > >  		skb_add_rx_frag(skb, skb_shinfo(skb)->nr_frags,
> > >  				skb_frag_page(frag), skb_frag_off(frag),
> > >  				skb_frag_size(frag), PAGE_SIZE);
> > > -		page_pool_release_page(rxq->page_pool, skb_frag_page(frag));
> > > +		skb_mark_for_recycle(skb, skb_frag_page(frag), &xdp->rxq->mem);
> > >  	}
> > >  
> > >  	return skb;
> > 
> > This cause skb_mark_for_recycle() to set 'skb->pp_recycle=1' multiple
> > times, for the same SKB.  (copy-pasted function below signature to help
> > reviewers).
> > 
> > This makes me question if we need an API for setting this per page
> > fragment?
> > Or if the API skb_mark_for_recycle() need to walk the page fragments in
> > the SKB and set the info stored in the page for each?
> 
> Considering just performances, I guess it is better open-code here since the
> driver already performs a loop over fragments to build the skb, but I guess
> this approach is quite risky and I would prefer to have a single utility
> routine to take care of linear area + fragments. What do you think?
> 

The mark_for_recycle does two things as you noticed, 
set the pp_recyle bit on the skb head and update the struct page information we
need to trigger the recycling.
We could split those and be more explicit, but isn't the current approach a
bit simpler for the driver writer to get it right?
I don't think setting a single value to 1 will have any noticeable performance
impact, but we can always test it.

> Regards,
> Lorenzo
> 
> > 
> > 
> > -- 
> > Best regards,
> >   Jesper Dangaard Brouer
> >   MSc.CS, Principal Kernel Engineer at Red Hat
> >   LinkedIn: http://www.linkedin.com/in/brouer
> >