Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <CABWYdi2a=Tc3dRfQ+037PG0GHKvZd5SEXJxBBbNspsrHK1zNpQ@mail.gmail.com>
 <CABWYdi1PeNbgnM4qE001+_BzHJxQcaaY9sLOK=Y7gjqfXZO0=g@mail.gmail.com>
 <YjA439FwajtHsahr@google.com> <YjEOiZCLBMgbw8oc@google.com> <CABWYdi0jd_pG_qqAnnGK6otNNXeNoiAWtmC14Jv+tiSadJPw0w@mail.gmail.com>
In-Reply-To: <CABWYdi0jd_pG_qqAnnGK6otNNXeNoiAWtmC14Jv+tiSadJPw0w@mail.gmail.com>
From:   Ivan Babrou <ivan@cloudflare.com>
Date:   Fri, 18 Mar 2022 09:30:09 -0700
Message-ID: <CABWYdi2gOzAK60gLYKx9gSoSfJRZaAjyAWm+55gLgcSKrDrP9Q@mail.gmail.com>
Subject: Re: zram corruption due to uninitialized do_swap_page fault
To:     Minchan Kim <minchan@kernel.org>
Cc:     Linux MM <linux-mm@kvack.org>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Nitin Gupta <ngupta@vflare.org>,
        Sergey Senozhatsky <senozhatsky@chromium.org>,
        Jens Axboe <axboe@kernel.dk>, linux-block@vger.kernel.org,
        kernel-team <kernel-team@cloudflare.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, Mar 16, 2022 at 11:26 AM Ivan Babrou <ivan@cloudflare.com> wrote:
> I'm making an internal build and will push it to some location to see
> how it behaves, but it might take a few days to get any sort of
> confidence in the results (unless it breaks immediately).
>
> I've also pushed my patch that disables SWP_SYNCHRONOUS_IO to a few
> locations yesterday to see how it fares.

I have some updates before the weekend. There are two experimental groups:

* My patch that removes the SWP_SYNCHRONOUS_IO flag. There are 704
machines in this group across 5 datacenters with cumulative uptime of
916 days.
* Minchan's patch to remove swap_slot_free_notify. There are 376
machines in this group across 3 datacenters with cumulative uptime of
240 days.

Our machines take a couple of hours to start swapping anything after
boot, and I discounted these two hours from the cumulative uptime.

Neither of these two groups experienced unexpected coredumps or
rocksdb corruptions.

I think at this point it's reasonable to proceed with Minchan's patch
(including a backport).