Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
References: <20220123115623.94843-1-42.hyeyoo@gmail.com>
 <f835cbb3-a028-1daf-c038-516dd47ce47c@gmail.com>
 <5cca8bdd-bed0-f26a-6c96-d18947d3a50b@gmail.com>
User-agent: mu4e 1.7.5; emacs 28.0.50
From:   Shay Agroskin <shayagr@amazon.com>
To:     Julian Wiedmann <jwiedmann.dev@gmail.com>
CC:     Hyeonggon Yoo <42.hyeyoo@gmail.com>, <netdev@vger.kernel.org>,
        "Arthur Kiyanovski" <akiyano@amazon.com>,
        David Arinzon <darinzon@amazon.com>,
        "Noam Dagan" <ndagan@amazon.com>,
        Saeed Bishara <saeedb@amazon.com>,
        "David S . Miller" <davem@davemloft.net>,
        Jakub Kicinski <kuba@kernel.org>,
        "Daniel Borkmann" <daniel@iogearbox.net>,
        Sameeh Jubran <sameehj@amazon.com>,
        "Wei Yongjun" <weiyongjun1@huawei.com>,
        Lorenzo Bianconi <lorenzo@kernel.org>,
        <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] net: ena: Do not waste napi skb cache
Date:   Mon, 24 Jan 2022 22:50:05 +0200
In-Reply-To: <5cca8bdd-bed0-f26a-6c96-d18947d3a50b@gmail.com>
Message-ID: <pj41zlmtjk7t9a.fsf@u570694869fb251.ant.amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; format=flowed
Precedence: bulk


Julian Wiedmann <jwiedmann.dev@gmail.com> writes:

> On 24.01.22 10:57, Julian Wiedmann wrote:
>> On 23.01.22 13:56, Hyeonggon Yoo wrote:
>>> By profiling, discovered that ena device driver allocates skb 
>>> by
>>> build_skb() and frees by napi_skb_cache_put(). Because the 
>>> driver
>>> does not use napi skb cache in allocation path, napi skb cache 
>>> is
>>> periodically filled and flushed. This is waste of napi skb 
>>> cache.
>>>
>>> As ena_alloc_skb() is called only in napi, Use 
>>> napi_build_skb()
>>> instead of build_skb() to when allocating skb.
>>>
>>> This patch was tested on aws a1.metal instance.
>>>
>>> Signed-off-by: Hyeonggon Yoo <42.hyeyoo@gmail.com>
>>> ---
>>>  drivers/net/ethernet/amazon/ena/ena_netdev.c | 2 +-
>>>  1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/drivers/net/ethernet/amazon/ena/ena_netdev.c 
>>> b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>>> index c72f0c7ff4aa..2c67fb1703c5 100644
>>> --- a/drivers/net/ethernet/amazon/ena/ena_netdev.c
>>> +++ b/drivers/net/ethernet/amazon/ena/ena_netdev.c
>>> @@ -1407,7 +1407,7 @@ static struct sk_buff 
>>> *ena_alloc_skb(struct ena_ring *rx_ring, void *first_frag)
>>>  		skb = netdev_alloc_skb_ip_align(rx_ring->netdev,
>>>  						rx_ring->rx_copybreak);
>> 
>> To keep things consistent, this should then also be 
>> napi_alloc_skb().
>> 
>
> And on closer look, this copybreak path also looks buggy. If 
> rx_copybreak
> gets reduced _while_ receiving a frame, the allocated skb can 
> end up too
> small to take all the data.
>
> @ ena maintainers: can you please fix this?
>

Updating the copybreak value is done through ena_ethtool.c 
(ena_set_tunable()) which updates `adapter->rx_copybreak`.
The adapter->rx_copybreak value is "propagated back" to the ring 
local attributes (rx_ring->rx_copybreak) only after an interface 
toggle which stops the napi routine first.

Unless I'm missing something here I don't think the bug you're 
describing exists.

I agree that the netdev_alloc_skb_ip_align() can become 
napi_alloc_skb(). Hyeonggon Yoo, can you please apply this change 
as well to this patch?

Thanks,
Shay


>>>  	else
>>> -		skb = build_skb(first_frag, ENA_PAGE_SIZE);
>>> +		skb = napi_build_skb(first_frag, ENA_PAGE_SIZE);
>>>  
>>>  	if (unlikely(!skb)) {
>>>  		ena_increase_stat(&rx_ring->rx_stats.skb_alloc_fail, 
>>>  1,
>>