Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 28 Dec 2022 20:50:36 +0000
From:   Matthew Wilcox <willy@infradead.org>
To:     Hyeonggon Yoo <42.hyeyoo@gmail.com>
Cc:     linux-mm@kvack.org, liam.howlett@oracle.com, surenb@google.com,
        ldufour@linux.ibm.com, michel@lespinasse.org, vbabka@suse.cz,
        linux-kernel@vger.kernel.org
Subject: Re: [QUESTION] about the maple tree and current status of mmap_lock
 scalability
Message-ID: <Y6ysHNPvKayTfeq8@casper.infradead.org>
References: <EC51CFA7-2BC8-4F72-A7D4-3B1A778EDB37@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <EC51CFA7-2BC8-4F72-A7D4-3B1A778EDB37@gmail.com>
Precedence: bulk

On Wed, Dec 28, 2022 at 09:48:51PM +0900, Hyeonggon Yoo wrote:
> Hello mm folks,
> 
> I have a few questions about the current status of mmap_lock scalability.
> 
> =============================================================
> What is currently causing the kernel to use mmap_lock to protect the maple tree?
> =============================================================
> 
> I understand that the long-term goal is to remove the need for mmap_lock in readers
> while traversing the maple tree, using techniques such as RCU or SPF.
> What is the biggest obstacle preventing this from being achieved at this time?

The long term goal is even larger than this.  Ideally, the VMA tree
would be protected by a spinlock rather than a mutex.  That turned out
to be too large a change for the moment (and isn't all that important
compared to enabling RCU readers)

> ==================================================
> How does the maple tree provide RCU-safe manipulation of VMAs?
> ==================================================
> 
> Is it similar to the approach suggested in the RCUVM paper (replacing the original
> root node with a new root node that shares most of its nodes and deferring
> the freeing of stale nodes using RCU)?
> 
> I'm having difficulty understanding the design of the maple tree in this regard.
> 
> [RCUVM paper] https://pdos.csail.mit.edu/papers/rcuvm:asplos12.pdf

While I've read the RCUVM paper, I wouldn't say it was particularly an
inspiration.  The Maple Tree is independent of the VM; it's a general
purpose B-tree.  As with any B-tree, when modifying a node, we don't
touch nodes that we don't need to touch.  As with any RCU data structure,
we defer freeing things while RCU readers might still have a reference
to them.

We don't necessarily go all the way to the root node when modifying a
leaf node.  For example, if we have this structure:

Root: Node A, 4000, Node B
Node A: p1, 50, p2, 100, p3, 150, p4, 200, NULL, 250, p6, 1000, p7
Node B: p8, 4050, p9, 4100, p10, 4150, p11, 4200, NULL, 4250, p13

and we replace p4 with a NULL over the whole range from 150-199,
we construct a new Node A2 that contains:

Node A2: p1, 50, p2, 100, p3, 150, NULL, 250, p6, 1000, p7

and we simply write A2 over the entry in Root.  Then we mark Node A as
dead and RCU-free Node A.  There's no need to replace Root as stores
to a pointer are atomic.  If we need to rebalance between Node A and
Node B, we will need to create a new Root (as well as both A and B),
mark all of them as dead and RCU-free them.