Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 18 Apr 2023 12:05:44 +0900
From:   Sergey Senozhatsky <senozhatsky@chromium.org>
To:     Andrew Morton <akpm@linux-foundation.org>
Cc:     Sergey Senozhatsky <senozhatsky@chromium.org>,
        Minchan Kim <minchan@kernel.org>,
        Yosry Ahmed <yosryahmed@google.com>,
        linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCHv2] zsmalloc: allow only one active pool compaction context
Message-ID: <20230418030544.GS25053@google.com>
References: <20230417135420.1836741-1-senozhatsky@chromium.org>
 <20230417174131.44de959204814209ef73e53e@linux-foundation.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230417174131.44de959204814209ef73e53e@linux-foundation.org>
Precedence: bulk

On (23/04/17 17:41), Andrew Morton wrote:
> On Mon, 17 Apr 2023 22:54:20 +0900 Sergey Senozhatsky <senozhatsky@chromium.org> wrote:
> > Introduce pool compaction mutex and permit only one compaction
> > context at a time. This reduces overall pool->lock contention.
> 
> That isn't what the patch does!  Perhaps an earlier version used a mutex?

Oh, yes.

[..]
> > @@ -2274,6 +2275,9 @@ unsigned long zs_compact(struct zs_pool *pool)
> >  	struct size_class *class;
> >  	unsigned long pages_freed = 0;
> >  
> > +	if (atomic_xchg(&pool->compaction_in_progress, 1))
> > +		return 0;
> > +
> 
> A code comment might be appropriate here.

OK.

> Is the spin_is_contended() test in __zs_compact() still relevant?

I'd say yes, pool->lock is "big pool lock", we use it for everything:
zs_map_object() when we read objects, zs_malloc() when we allocate objects,
zs_free() when we free objects, etc.

> And....  single-threading the operation seems a pretty sad way of
> addressing a contention issue.  zs_compact() is fairly computationally
> expensive - surely a large machine would like to be able to
> concurrently run many instances of zs_compact()?

Compaction is effective only when zspool suffers from internal fragmentation.
Concurrent compaction threads iterate size classes in the same order and are
likely to compete for pool->lock to just find out that previous pool->lock
owner has compacted the class already and there isn't much left to do.

As of concurrent compaction on big machines, the thing is - concurrent
compaction doesn't really happen. We always have just one thread compacting
size classes under pool->lock, the remaining compaction threads just spin on
pool->lock. I believe it used to be slightly different in the past when we
had per size-class lock instead of "global" pool->lock. Commit c0547d0b6a4b
("zsmalloc: consolidate zs_pool's migrate_lock and size_class's locks") has
basically made compaction single-threaded.