Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20221201192644.1941049-1-axelrasmussen@google.com>
In-Reply-To: <20221201192644.1941049-1-axelrasmussen@google.com>
From:   Yu Zhao <yuzhao@google.com>
Date:   Thu, 1 Dec 2022 12:47:33 -0700
Message-ID: <CAOUHufb8cD_i=RacgeVH5HHwnr++HsXbtGP3Rx9CJi20obHcqg@mail.gmail.com>
Subject: Re: [PATCH] mm: multi-gen LRU: fix LRU size accounting on folio removal
To:     Axel Rasmussen <axelrasmussen@google.com>
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        Suleiman Souhlal <suleiman@google.com>,
        Steven Barrett <steven@liquorix.net>,
        Oleksandr Natalenko <oleksandr@natalenko.name>,
        Suren Baghdasaryan <surenb@google.com>,
        Arnd Bergmann <arnd@arndb.de>, Peter Xu <peterx@redhat.com>,
        Gaosheng Cui <cuigaosheng1@huawei.com>,
        Hugh Dickins <hughd@google.com>,
        Brian Geffon <bgeffon@google.com>,
        "Jan Alexander Steffens (heftig)" <heftig@archlinux.org>,
        linux-kernel@vger.kernel.org, Yosry Ahmed <yosryahmed@google.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, Dec 1, 2022 at 12:26 PM Axel Rasmussen <axelrasmussen@google.com> wrote:
>
> When removing a folio from MGLRU, we want to update the LRU size
> accordingly based on the generation it belonged to previously -
> lru_gen_update_size() does this.
>
> The bug here is, set_mask_bits effectively clears the generation bits.
> Ignoring the complexity set_mask_bits is meant to handle, the code being
> changed here is in effect:
>
>     flags = !reclaiming && lru_gen_is_active(lruvec, gen) ? BIT(PG_active) : 0;
>     flags = *folio->flags = (*folio->flags & ~LRU_GEN_MASK) | flags;
>     gen = ((flags & LRU_GEN_MASK) >> LRU_GEN_PGOFF) - 1;
>
> In other words, the bug is we clear all of the `LGU_GEN_MASK` bits, and
> then we recalculate `gen` - but of course after clearing the bits
> `flags & LRU_GEN_MASK` is always zero, and so now `gen` is always -1.
>
> So we effectively always call:
>
>     lru_gen_update_size(lruvec, folio, -1, -1);
>
> This leads `lru_gen_update_size` to incorrectly conclude that we're
> **adding**, not removing, a folio. We take this path:
>
>     /* addition */
>     if (old_gen < 0) {
>         /* always false, new_gen is -1 too */
>         if (lru_gen_is_active(lruvec, new_gen))
>             /* ... */
>         __update_lru_size(lruvec, lru, zone, delta);
>         return;
>     }
>
> In other words, when removing, we incorrectly *add* the delta to the
> inactive LRU instead of subtracting.
>
> The fix is simple. We already have the generation number the folio
> belonged to: we set `int gen = folio_lru_gen(folio);` at the top of
> `lru_gen_del_folio`. So, just delete the line incorrectly recalculating
> the generation number.
>
> Fixes: ec1c86b25f4b ("mm: multi-gen LRU: groundwork")
> Signed-off-by: Axel Rasmussen <axelrasmussen@google.com>

NAK.

You are referencing our old (9xx) set_mask_bits(), which returns "new"
(a bad behavior). Its latest version returns "old".

Even if it was a bug:

1. lru_gen_update_size(lruvec, folio, -1, -1) would have been caught by
VM_WARN_ON_ONCE(old_gen == -1 && new_gen == -1).

2. The fix is still wrong, because "gen" read from
folio_lru_gen(folio) is non atomic and can change before
set_mask_bits() finishes.