Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 3 Feb 2021 10:00:15 -0400
From:   Jason Gunthorpe <jgg@ziepe.ca>
To:     Gal Pressman <galpress@amazon.com>
Cc:     aarcange@redhat.com, akpm@linux-foundation.org, gokhale2@llnl.gov,
        hch@lst.de, jack@suse.cz, jannh@google.com, jhubbard@nvidia.com,
        kirill@shutemov.name, ktkhai@virtuozzo.com, leonro@nvidia.com,
        linux-kernel@vger.kernel.org, linux-mm@kvack.org,
        mcfadden8@llnl.gov, oleg@redhat.com, peterx@redhat.com,
        torvalds@linux-foundation.org, wzam@amazon.com,
        yang.shi@linux.alibaba.com
Subject: Re: [PATCH 1/4] mm: Trial do_wp_page() simplification
Message-ID: <20210203140015.GP4718@ziepe.ca>
References: <20210202171327.GN4718@ziepe.ca>
 <20210203124358.59017-1-galpress@amazon.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20210203124358.59017-1-galpress@amazon.com>
Precedence: bulk

On Wed, Feb 03, 2021 at 02:43:58PM +0200, Gal Pressman wrote:
> > On Tue, Feb 02, 2021 at 12:05:36PM -0500, Peter Xu wrote:
> > 
> >> > Gal, you could also MADV_DONTFORK this range if you are explicitly
> >> > allocating them via special mmap.
> >> 
> >> Yeah I wanted to mention this one too but I just forgot when reply: the issue
> >> thread previously pasted smells like some people would like to drop
> >> MADV_DONTFORK, but if it's able to still be applied I don't know why
> >> not..
> > 
> > I want to drop the MADV_DONTFORK for dynamic data memory allocated by
> > the application layers (eg with malloc) without knowledge of how they
> > will be used.
> > 
> > This case is a buffer internal to the communication system that we
> > know at allocation time how it will be used; so an explicit,
> > deliberate, MADV_DONTFORK is fine
> 
> We are referring to libfabric's bounce buffers, correct?
> Libfabric could be considered as the "app" here, it's not clear why these
> buffers should be DONTFORK'd before ibv_reg_mr() but others don't.

I assumed they were internal to the EFA code itself.

> Anyway, it should be simple enough to madvise them after allocation, although I
> think it's part of libfabric's generic code (which isn't necessarily used on
> top of rdma-core).

Ah, so that is a reasonable justification for wanting to fix this in
the kernel..

Lets give Peter some time first.

The other direction to validate this approach is to remove the
MAP_HUGETLB flags and rely on THP instead, and/or mark them as
MAP_SHARED.

I'm not sure generic code should be use using MAP_HUGETLB..

This would be enough to confirm that everything else is working as
expected

Thanks,
Jason