Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20220624125423.6126-1-mgorman@techsingularity.net> <20220703162821.f097d6e4b3e2a0114820a8d9@linux-foundation.org>
In-Reply-To: <20220703162821.f097d6e4b3e2a0114820a8d9@linux-foundation.org>
From:   Yu Zhao <yuzhao@google.com>
Date:   Sun, 3 Jul 2022 17:31:09 -0600
Message-ID: <CAOUHufZj87ewG6_OObmDByxHv51DgbkB-O6oMitw72QF1JrkcQ@mail.gmail.com>
Subject: Re: [PATCH v5 00/7] Drain remote per-cpu directly
To:     Andrew Morton <akpm@linux-foundation.org>
Cc:     Mel Gorman <mgorman@techsingularity.net>,
        Nicolas Saenz Julienne <nsaenzju@redhat.com>,
        Marcelo Tosatti <mtosatti@redhat.com>,
        Vlastimil Babka <vbabka@suse.cz>,
        Michal Hocko <mhocko@kernel.org>,
        Hugh Dickins <hughd@google.com>,
        Marek Szyprowski <m.szyprowski@samsung.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Linux-MM <linux-mm@kvack.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Sun, Jul 3, 2022 at 5:28 PM Andrew Morton <akpm@linux-foundation.org> wrote:
>
> On Fri, 24 Jun 2022 13:54:16 +0100 Mel Gorman <mgorman@techsingularity.net> wrote:
>
> > Some setups, notably NOHZ_FULL CPUs, may be running realtime or
> > latency-sensitive applications that cannot tolerate interference due to
> > per-cpu drain work queued by __drain_all_pages().  Introduce a new
> > mechanism to remotely drain the per-cpu lists. It is made possible by
> > remotely locking 'struct per_cpu_pages' new per-cpu spinlocks.  This has
> > two advantages, the time to drain is more predictable and other unrelated
> > tasks are not interrupted.
> >
> > This series has the same intent as Nicolas' series "mm/page_alloc: Remote
> > per-cpu lists drain support" -- avoid interference of a high priority task
> > due to a workqueue item draining per-cpu page lists.  While many workloads
> > can tolerate a brief interruption, it may cause a real-time task running
> > on a NOHZ_FULL CPU to miss a deadline and at minimum, the draining is
> > non-deterministic.
> >
> > Currently an IRQ-safe local_lock protects the page allocator per-cpu
> > lists. The local_lock on its own prevents migration and the IRQ disabling
> > protects from corruption due to an interrupt arriving while a page
> > allocation is in progress.
> >
> > This series adjusts the locking.  A spinlock is added to struct
> > per_cpu_pages to protect the list contents while local_lock_irq is
> > ultimately replaced by just the spinlock in the final patch.  This allows
> > a remote CPU to safely. Follow-on work should allow the spin_lock_irqsave
> > to be converted to spin_lock to avoid IRQs being disabled/enabled in
> > most cases. The follow-on patch will be one kernel release later as it
> > is relatively high risk and it'll make bisections more clear if there
> > are any problems.
>
> I plan to move this and Mel's fix to [7/7] into mm-stable around July 8.

I've thrown it together with the Maple Tree and passed a series of stress tests.