From:   ebiederm@xmission.com (Eric W. Biederman)
To:     Claudio Imbrenda <imbrenda@linux.ibm.com>
Cc:     linux-kernel@vger.kernel.org, linux-mm@kvack.org, thuth@redhat.com,
        frankja@linux.ibm.com, borntraeger@de.ibm.com,
        Ulrich.Weigand@de.ibm.com, david@redhat.com, ultrachin@163.com,
        akpm@linux-foundation.org, vbabka@suse.cz, brookxu.cn@gmail.com,
        xiaoggchen@tencent.com, linuszeng@tencent.com, yihuilu@tencent.com,
        mhocko@suse.com, daniel.m.jordan@oracle.com, axboe@kernel.dk,
        legion@kernel.org, peterz@infradead.org, aarcange@redhat.com,
        christian@brauner.io, tglx@linutronix.de
References: <20211111095008.264412-1-imbrenda@linux.ibm.com>
        <20211111095008.264412-4-imbrenda@linux.ibm.com>
        <874k8ixzx0.fsf@email.froward.int.ebiederm.org>
        <20211112103439.441b4c12@p-imbrenda>
Date:   Fri, 12 Nov 2021 08:57:13 -0600
In-Reply-To: <20211112103439.441b4c12@p-imbrenda> (Claudio Imbrenda's message
        of "Fri, 12 Nov 2021 10:34:39 +0100")
Message-ID: <87v90xv2uu.fsf@email.froward.int.ebiederm.org>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Subject: Re: [RFC v1 2/4] kernel/fork.c: implement new process_mmput_async syscall
Precedence: bulk

Claudio Imbrenda <imbrenda@linux.ibm.com> writes:

> On Thu, 11 Nov 2021 13:20:11 -0600
> ebiederm@xmission.com (Eric W. Biederman) wrote:
>
>> Claudio Imbrenda <imbrenda@linux.ibm.com> writes:
>> 
>> > The goal of this new syscall is to be able to asynchronously free the
>> > mm of a dying process. This is especially useful for processes that use
>> > huge amounts of memory (e.g. databases or KVM guests). The process is
>> > allowed to terminate immediately, while its mm is cleaned/reclaimed
>> > asynchronously.
>> >
>> > A separate process needs use the process_mmput_async syscall to attach
>> > itself to the mm of a running target process. The process will then
>> > sleep until the last user of the target mm has gone.
>> >
>> > When the last user of the mm has gone, instead of synchronously free
>> > the mm, the attached process is awoken. The syscall will then continue
>> > and clean up the target mm.
>> >
>> > This solution has the advantage that the cleanup of the target mm can
>> > happen both be asynchronous and properly accounted for (e.g. cgroups).
>> >
>> > Tested on s390x.
>> >
>> > A separate patch will actually wire up the syscall.  
>> 
>> I am a bit confused.
>> 
>> You want the process report that it has finished immediately,
>> and you want the cleanup work to continue on in the background.
>> 
>> Why do you need a separate process?
>> 
>> Why not just modify the process cleanup code to keep the task_struct
>> running while allowing waitpid to reap the process (aka allowing
>> release_task to run)?  All tasks can be already be reaped after
>> exit_notify in do_exit.
>> 
>> I can see some reasons for wanting an opt-in.  It is nice to know all of
>> a processes resources have been freed when waitpid succeeds.
>> 
>> Still I don't see why this whole thing isn't exit_mm returning
>> the mm_sturct when a flag is set, and then having an exit_mm_late
>> being called and passed the returned mm after exit_notify.
>
> nevermind, exit_notify is done after cgroup_exit, the teardown would
> then not be accounted properly

So you want this new mechanism so you can separate the cleanup from
the exit notification, and so that things are accounted properly.

It would have helped if you had included a link to the previous
conversation.

I think Michal Hoko has a point.  This looks like a job for
"clone(CLONE_VM)" and "prctl(PR_SET_PDEATH_SIG)".  Maybe using a pidfd
instead of the prctl.

AKA just create a child that shares the parents memory, and waits for
the parent to exit and then cleans things up.

That should not need any new kernel mechanisms.


There is the other question: why this is disastrously slow on s390?
Is this a s390 specific issue?  Can the issue be fixed by optimizing
what is happening on s390?

Eric