Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
MIME-Version: 1.0
References: <YnWRVd5slCy5H0fC@creeky> <20220507015646.5377-1-hdanton@sina.com>
In-Reply-To: <20220507015646.5377-1-hdanton@sina.com>
From:   Yu Zhao <yuzhao@google.com>
Date:   Sat, 7 May 2022 11:27:15 -0700
Message-ID: <CAOUHufY=xAvDKSaV8vybgObXPBEsPqqS7R3+T_-6ix7bUvQc6w@mail.gmail.com>
Subject: Re: Alpha: rare random memory corruption/segfault in user space bisected
To:     Michael Cree <mcree@orcon.net.nz>
Cc:     Linux-MM <linux-mm@kvack.org>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        Hillf Danton <hdanton@sina.com>,
        Joonsoo Kim <iamjoonsoo.kim@lge.com>
Content-Type: multipart/mixed; boundary="00000000000005798e05de70220d"
Precedence: bulk

--00000000000005798e05de70220d
Content-Type: text/plain; charset="UTF-8"

On Fri, May 6, 2022 at 6:57 PM Hillf Danton <hdanton@sina.com> wrote:
>
> On Sat, 7 May 2022 09:21:25 +1200 Michael Cree wrote:
> > Alpha kernel has been exhibiting rare and random memory
> > corruptions/segaults in user space since the 5.9.y kernel.  First seen
> > on the Debian Ports build daemon when running 5.10.y kernel resulting
> > in the occasional (one or two a day) build failures with gcc ICEs either
> > due to self detected corrupt memory structures or segfaults.  Have been
> > running 5.8.y kernel without such problems for over six months.
> >
> > Tried bisecting last year but went off track with incorrect good/bad
> > determinations due to rare nature of bug.  After trying a 5.16.y kernel
> > early this year and seen the bug is still present retried the bisection
> > and have got to:
> >
> > aae466b0052e1888edd1d7f473d4310d64936196 is the first bad commit
> > commit aae466b0052e1888edd1d7f473d4310d64936196
> > Author: Joonsoo Kim <iamjoonsoo.kim@lge.com>
> > Date:   Tue Aug 11 18:30:50 2020 -0700
> >
> >     mm/swap: implement workingset detection for anonymous LRU

This commit seems innocent to me. While not ruling out anything, i.e.,
this commit, compiler, qemu, userspace itself, etc., my wild guess is
the problem is memory barrier related. Two lock/unlock pairs, which
imply two full barriers, were removed. This is not a small deal on
Alpha, since it imposes no constraints on cache coherency, AFAIK.

Can you please try the attached patch on top of this commit? Thanks!

> > Pretty confident this is the bad commit as the kernel built to the parent
> > commit (3852f6768ede54...) has not failed in four days running. Always have
> > seen the failure within one day of running in past.
>
> See if the fix to the syzbot bisection [1] is not a cure to your issue.
>
> [1] https://lore.kernel.org/lkml/000000000000625fa705dd1802e3@google.com/

--00000000000005798e05de70220d
Content-Type: application/octet-stream; name="test.diff"
Content-Disposition: attachment; filename="test.diff"
Content-Transfer-Encoding: base64
Content-ID: <f_l2w76ukb0>
X-Attachment-Id: f_l2w76ukb0

ZGlmZiAtLWdpdCBhL21tL21lbW9yeS5jIGIvbW0vbWVtb3J5LmMKaW5kZXggZGUzMTFmYzc2Mzll
Li5mMWNmMDc0MTZjZjQgMTAwNjQ0Ci0tLSBhL21tL21lbW9yeS5jCisrKyBiL21tL21lbW9yeS5j
CkBAIC0zMTUwLDYgKzMxNTAsOCBAQCB2bV9mYXVsdF90IGRvX3N3YXBfcGFnZShzdHJ1Y3Qgdm1f
ZmF1bHQgKnZtZikKIAkJCQkJZ290byBvdXRfcGFnZTsKIAkJCQl9CiAKKwkJCQlzbXBfbWIoKTsK
KwogCQkJCXNoYWRvdyA9IGdldF9zaGFkb3dfZnJvbV9zd2FwX2NhY2hlKGVudHJ5KTsKIAkJCQlp
ZiAoc2hhZG93KQogCQkJCQl3b3JraW5nc2V0X3JlZmF1bHQocGFnZSwgc2hhZG93KTsKZGlmZiAt
LWdpdCBhL21tL3N3YXBfc3RhdGUuYyBiL21tL3N3YXBfc3RhdGUuYwppbmRleCBiNzNhYWJkZmQz
NWEuLjMxMGQ0MDQ5Y2RmMyAxMDA2NDQKLS0tIGEvbW0vc3dhcF9zdGF0ZS5jCisrKyBiL21tL3N3
YXBfc3RhdGUuYwpAQCAtNDk5LDYgKzQ5OSw4IEBAIHN0cnVjdCBwYWdlICpfX3JlYWRfc3dhcF9j
YWNoZV9hc3luYyhzd3BfZW50cnlfdCBlbnRyeSwgZ2ZwX3QgZ2ZwX21hc2ssCiAJCWdvdG8gZmFp
bF91bmxvY2s7CiAJfQogCisJc21wX21iKCk7CisKIAlpZiAoc2hhZG93KQogCQl3b3JraW5nc2V0
X3JlZmF1bHQocGFnZSwgc2hhZG93KTsKIAo=
--00000000000005798e05de70220d--