Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [RFC PATCH] Add proc interface to set PF_MEMALLOC flags
To:     Hillf Danton <hdanton@sina.com>
Cc:     Mike Christie <mchristi@redhat.com>,
        "Kirill A. Shutemov" <kirill@shutemov.name>, axboe@kernel.dk,
        James.Bottomley@HansenPartnership.com, martin.petersen@oracle.com,
        linux-kernel@vger.kernel.org, linux-scsi@vger.kernel.org,
        linux-block@vger.kernel.org, Linux-MM <linux-mm@kvack.org>
References: <20190911031348.9648-1-hdanton@sina.com>
 <20190911135237.11248-1-hdanton@sina.com>
From:   Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
Message-ID: <6cff2ae9-4436-8df7-55a7-59e2e80b1054@i-love.sakura.ne.jp>
Date:   Wed, 11 Sep 2019 23:20:38 +0900
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:60.0) Gecko/20100101
 Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <20190911135237.11248-1-hdanton@sina.com>
Content-Type: text/plain; charset=utf-8
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 2019/09/11 22:52, Hillf Danton wrote:
> 
> On Wed, 11 Sep 2019 19:07:34 +0900
>>
>> But I guess that there is a problem.
> 
> Not a new one. (see commit 7dea19f9ee63)
> 
>> Setting PF_MEMALLOC_NOIO causes
>> current_gfp_context() to mask __GFP_IO | __GFP_FS, but the OOM killer cannot
>> be invoked when __GFP_FS is masked. As a result, any userspace thread which
>> has PF_MEMALLOC_NOIO cannot invoke the OOM killer.
> 
> Correct.
> 
>> If the userspace thread
>> which uses PF_MEMALLOC_NOIO is involved in memory reclaiming activities,
>> the memory reclaiming activities won't be able to make forward progress when
>> the userspace thread triggered e.g. a page fault. Can the "userspace components
>> that can run in the IO path" survive without any memory allocation?
> 
> Good question.
> 
> It can be solved without oom killer involved because user should be
> aware of the risk of PF_MEMALLOC_NOIO if they ask for the convenience.
> OTOH we are able to control any abuse of it as you worry, knowing that
> the combination of __GFP_FS and oom killer can not get more than 50 users
> works done, and we have to pay as much attention as we can to the decisions
> they make. In case of PF_MEMALLOC_NOIO, we simply fail the allocation
> rather than killing a random victim.

According to commit c288983dddf71421 ("mm/page_alloc.c: make sure OOM victim can
try allocations with no watermarks once"), memory allocation failure from a page
fault results in invocation of the OOM killer via pagefault_out_of_memory() which
after all kills a random victim.

> 
> 
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -3854,6 +3854,8 @@ __alloc_pages_may_oom(gfp_t gfp_mask, un
>  	 * out_of_memory). Once filesystems are ready to handle allocation
>  	 * failures more gracefully we should just bail out here.
>  	 */
> +	if (current->flags & PF_MEMALLOC_NOIO)
> +		goto out;
>  
>  	/* The OOM killer may not free memory on a specific node */
>  	if (gfp_mask & __GFP_THISNODE)
> 
>