Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 3 Nov 2020 17:16:40 -0800
From:   Rama Nichanamatlu <rama.nichanamatlu@oracle.com>
To:     Matthew Wilcox <willy@infradead.org>
Cc:     Dongli Zhang <dongli.zhang@oracle.com>, linux-mm@kvack.org,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        akpm@linux-foundation.org, davem@davemloft.net, kuba@kernel.org,
        aruna.ramakrishna@oracle.com, bert.barbe@oracle.com,
        venkat.x.venkatsubra@oracle.com, manjunath.b.patil@oracle.com,
        joe.jin@oracle.com, srinivas.eeda@oracle.com
Subject: Re: [PATCH 1/1] mm: avoid re-using pfmemalloc page in
 page_frag_alloc()
Message-ID: <20201104011640.GE2445@rnichana-ThinkPad-T480>
References: <20201103193239.1807-1-dongli.zhang@oracle.com>
 <20201103203500.GG27442@casper.infradead.org>
 <7141038d-af06-70b2-9f50-bf9fdf252e22@oracle.com>
 <20201103211541.GH27442@casper.infradead.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Disposition: inline
In-Reply-To: <20201103211541.GH27442@casper.infradead.org>
User-Agent: Mutt/1.9.4 (2018-02-28)
Precedence: bulk

>Thanks for providing the numbers.  Do you think that dropping (up to)
>7 packets is acceptable?

net.ipv4.tcp_syn_retries = 6

tcp clients wouldn't even get that far leading to connect establish issues.

-rama
On Tue, Nov 03, 2020 at 09:15:41PM +0000, Matthew Wilcox wrote:
>On Tue, Nov 03, 2020 at 12:57:33PM -0800, Dongli Zhang wrote:
>> On 11/3/20 12:35 PM, Matthew Wilcox wrote:
>> > On Tue, Nov 03, 2020 at 11:32:39AM -0800, Dongli Zhang wrote:
>> >> However, once kernel is not under memory pressure any longer (suppose large
>> >> amount of memory pages are just reclaimed), the page_frag_alloc() may still
>> >> re-use the prior pfmemalloc page_frag_cache->va to allocate skb->data. As a
>> >> result, the skb->pfmemalloc is always true unless page_frag_cache->va is
>> >> re-allocated, even the kernel is not under memory pressure any longer.
>> >> +	/*
>> >> +	 * Try to avoid re-using pfmemalloc page because kernel may already
>> >> +	 * run out of the memory pressure situation at any time.
>> >> +	 */
>> >> +	if (unlikely(nc->va && nc->pfmemalloc)) {
>> >> +		page = virt_to_page(nc->va);
>> >> +		__page_frag_cache_drain(page, nc->pagecnt_bias);
>> >> +		nc->va = NULL;
>> >> +	}
>> >
>> > I think this is the wrong way to solve this problem.  Instead, we should
>> > use up this page, but refuse to recycle it.  How about something like this (not even compile tested):
>>
>> Thank you very much for the feedback. Yes, the option is to use the same page
>> until it is used up (offset < 0). Instead of recycling it, the kernel free it
>> and allocate new one.
>>
>> This depends on whether we will tolerate the packet drop until this page is used up.
>>
>> For virtio-net, the payload (skb->data) is of size 128-byte. The padding and
>> alignment will finally make it as 512-byte.
>>
>> Therefore, for virtio-net, we will have at most 4096/512-1=7 packets dropped
>> before the page is used up.
>
>My thinking is that if the kernel is under memory pressure then freeing
>the page and allocating a new one is likely to put even more strain
>on the memory allocator, so we want to do this "soon", rather than at
>each allocation.
>
>Thanks for providing the numbers.  Do you think that dropping (up to)
>7 packets is acceptable?
>
>We could also do something like ...
>
>        if (unlikely(nc->pfmemalloc)) {
>                page = alloc_page(GFP_NOWAIT | __GFP_NOWARN);
>                if (page)
>                        nc->pfmemalloc = 0;
>                put_page(page);
>        }
>
>to test if the memory allocator has free pages at the moment.  Not sure
>whether that's a good idea or not -- hopefully you have a test environment
>set up where you can reproduce this condition on demand and determine
>which of these three approaches is best!