Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 11 Oct 2022 21:29:05 +0200
From:   Michal Hocko <mhocko@suse.com>
To:     Frank van der Linden <fvdl@google.com>
Cc:     Zhongkun He <hezhongkun.hzk@bytedance.com>, corbet@lwn.net,
        akpm@linux-foundation.org, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org, linux-api@vger.kernel.org,
        linux-doc@vger.kernel.org, wuyun.abel@bytedance.com
Subject: Re: [RFC] mm: add new syscall pidfd_set_mempolicy()
Message-ID: <Y0XEAUD9Ujcu/j8y@dhcp22.suse.cz>
References: <20221010094842.4123037-1-hezhongkun.hzk@bytedance.com>
 <CAPTztWYTGOb8ZQzfgThqJn+fyi4ZB8=JQQZi5_rUoDhdftKtvg@mail.gmail.com>
 <Y0WE/lEiNvl2ljo1@dhcp22.suse.cz>
 <CAPTztWZZOxtzdEm=wbOiL_VDPJuCaW0XVCvsdRpCHE+ph+5eZQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAPTztWZZOxtzdEm=wbOiL_VDPJuCaW0XVCvsdRpCHE+ph+5eZQ@mail.gmail.com>
Precedence: bulk

On Tue 11-10-22 10:22:23, Frank van der Linden wrote:
> On Tue, Oct 11, 2022 at 8:00 AM Michal Hocko <mhocko@suse.com> wrote:
> >
> > On Mon 10-10-22 09:22:13, Frank van der Linden wrote:
> > > For consistency with process_madvise(), I would suggest calling it
> > > process_set_mempolicy.
> >
> > This operation has per-thread rather than per-process semantic so I do
> > not think your proposed naming is better.
> 
> True. I suppose you could argue that it should have been
> pidfd_madvise() then for consistency, but that ship has sailed.

madvise commands have per mm semantic. It is set_mempolicy which is
kinda special and it allows to have per task_struct semantic even when
the actual allocation is in the same address space. To be honest I am
not really sure why that is this way because threads aim to share memory
so why should they have different memory policies?

I suspect that the original thinking was that some portions that are
private to the process want to have different affinities (e.g. stacks
and dedicated per cpu heap arenas). E.g. worker pools which want to be
per-cpu local with their own allocations but operate on shared data that
requires different policies.

> > > Other than that, this makes sense. To complete
> > > the set, perhaps a process_mbind() should be added as well. What do
> > > you think?
> >
> > Is there any real usecase for this interface? How is the caller supposed
> > to make per-range decisions without a very involved coordination with
> > the target process?
> 
> The use case for a potential pidfd_mbind() is basically a combination
> of what is described for in the process_madvise proposal (
> https://lore.kernel.org/lkml/20200901000633.1920247-1-minchan@kernel.org/
> ), and what this proposal describes: system management software acting
> as an orchestrator that has a better overview of the system as a whole
> (NUMA nodes, memory tiering), and has knowledge of the layout of the
> processes involved.

Well, per address range operation is a completely different beast I
would say. External tool would need to a) understand what that range is
used for (e.g. stack/heap ranges, mmaped shared files like libraries or
private mappings) and b) by in sync with memory layout modifications
done by applications (e.g. that an mmap has been issued to back malloc
request). Quite a lot of understanding about the specific process. I
would say that with that intimate knowledge it is quite better to be
part of the process and do those changes from within of the process
itself.
-- 
Michal Hocko
SUSE Labs