Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Gal Pressman <galpress@amazon.com>
To:     <jgg@ziepe.ca>
CC:     <aarcange@redhat.com>, <akpm@linux-foundation.org>,
        <gokhale2@llnl.gov>, <hch@lst.de>, <jack@suse.cz>,
        <jannh@google.com>, <jhubbard@nvidia.com>, <kirill@shutemov.name>,
        <ktkhai@virtuozzo.com>, <leonro@nvidia.com>,
        <linux-kernel@vger.kernel.org>, <linux-mm@kvack.org>,
        <mcfadden8@llnl.gov>, <oleg@redhat.com>, <peterx@redhat.com>,
        <torvalds@linux-foundation.org>, <wzam@amazon.com>,
        <yang.shi@linux.alibaba.com>
Subject: Re: [PATCH 1/4] mm: Trial do_wp_page() simplification
Date:   Wed, 3 Feb 2021 14:43:58 +0200
Message-ID: <20210203124358.59017-1-galpress@amazon.com>
In-Reply-To: <20210202171327.GN4718@ziepe.ca>
References: <20210202171327.GN4718@ziepe.ca>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
Precedence: bulk

> On Tue, Feb 02, 2021 at 12:05:36PM -0500, Peter Xu wrote:
> 
>> > Gal, you could also MADV_DONTFORK this range if you are explicitly
>> > allocating them via special mmap.
>> 
>> Yeah I wanted to mention this one too but I just forgot when reply: the issue
>> thread previously pasted smells like some people would like to drop
>> MADV_DONTFORK, but if it's able to still be applied I don't know why
>> not..
> 
> I want to drop the MADV_DONTFORK for dynamic data memory allocated by
> the application layers (eg with malloc) without knowledge of how they
> will be used.
> 
> This case is a buffer internal to the communication system that we
> know at allocation time how it will be used; so an explicit,
> deliberate, MADV_DONTFORK is fine

We are referring to libfabric's bounce buffers, correct?
Libfabric could be considered as the "app" here, it's not clear why these
buffers should be DONTFORK'd before ibv_reg_mr() but others don't.

Anyway, it should be simple enough to madvise them after allocation, although I
think it's part of libfabric's generic code (which isn't necessarily used on
top of rdma-core).

I'll take this discussion with the libfabric guys.
Thanks Peter and Jason.