Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp9439886imu; Wed, 5 Dec 2018 04:59:49 -0800 (PST) X-Google-Smtp-Source: AFSGD/Xl4sJiH1gBudSzvS6njuVAf2ZKRUgs2St9RyFpjinMRcF5ho+AaG0GbM2AMOIn3woyhRVc X-Received: by 2002:a17:902:8346:: with SMTP id z6mr24024412pln.340.1544014789733; Wed, 05 Dec 2018 04:59:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544014789; cv=none; d=google.com; s=arc-20160816; b=OZPAPsDgvb7v15NwESQVyKm91eQSZ/swOr49XGbsvhd2Eb5uGudLmWLb1NUKf4QQDu Q1bcCkjNDulSvJ8IEgwwunnYwsngeuitwNz72gZUfpHI9EyEiUddnw6UyGjLq6xY3DtU IrLBF5yGFJC5bEHntjSA6I4ygacfIzReM1jNrDV81YlTkbnMi9PJ1YaQHAOwybahQ6GG Jjp3rAc+rybWMa4xx8ScsSM7mSiQBOU452Iwu9Cm2NPvU5f3k6VMXOxxsa0HztuU/DV6 hFcieodKislzY9FARY8GpKQhBhL7+eWykh0IIGlHpGlLtSRIRna/Gnh9jpI3rFSwVrig Skyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:cc:to:subject :message-id:date:from:in-reply-to:references:mime-version :dkim-signature; bh=1xKUxRZCD0KmBrrzVGHFaFLA7pcHSE7Vxhv5dQ+ga9E=; b=l5MbM+UIZxEyZyLCM6Of+NByU1YGJeF0P72UkTfIZmjueDB60/tNt94y41UciK7Dd5 krDKb7HEP4adBmPz49noMViVZShk5VJP1+IVO1TbTytcfSf1K54t0aFmlXhBJHri4F1K +p0++5EjvdZPhSEfi5PgVNYQNr7eiw+INThMvtI/Z2MGNEc5OJdS2lyH8gr8kqTGhdsu v2PXRpBeVM2RJ9952eAgE7yf18DlqJEfQqVysWMTQthIJRMCZrhwqtd4O2OSNNPvNRrO o3mXGqxouUCZKRnMP5AE38j1546mlHMGrON/8rlv3lUvt9MYDheMuIsU0qBNP0U8gYEb vW+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b="kB8J/tJS"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h10si16901917pgd.285.2018.12.05.04.59.34; Wed, 05 Dec 2018 04:59:49 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b="kB8J/tJS"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728115AbeLEM6W (ORCPT + 99 others); Wed, 5 Dec 2018 07:58:22 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:40366 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727777AbeLEM6V (ORCPT ); Wed, 5 Dec 2018 07:58:21 -0500 Received: by mail-wm1-f68.google.com with SMTP id q26so12729328wmf.5; Wed, 05 Dec 2018 04:58:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc:content-transfer-encoding; bh=1xKUxRZCD0KmBrrzVGHFaFLA7pcHSE7Vxhv5dQ+ga9E=; b=kB8J/tJSXOl8XBTBu6mMA3VWyw9kOCLjXjVURSalHB4hbaWevFvu7e+JP/KMz6Vfsw Oy5/hD9k8o3PJ/cuw2fOuvh+H/uXvMYIXQlufrZqTR6+MEWmNhfRnOPqahvH2/eYJkPs b7UnQipHTK9ZCo/ffSKoXYV49QWr5unHIIc3HsSdV5mn/VpJ21AL/0r8JdmR/BLQ3aJz pFWSecGhEz1FbvWoHRYTJiGVMIKYiEqaBgUGWphPSbVlJwtXmRlqqZlmvkuLZbS1H6K4 jiTs2h0ygjBiGRDaSsiUlGK+KcigiRqqbSw2W2M/nGMTTP8hhalKyrhnjEehDJsBO63v JfqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc:content-transfer-encoding; bh=1xKUxRZCD0KmBrrzVGHFaFLA7pcHSE7Vxhv5dQ+ga9E=; b=gqsvFvU7silIJ3XBKLfpWCcy20AbC/93ptA1bXoUSAsEtdBuhbSmnE07qEfy4TcAk0 BeT2bA3Gd+fO2GQiy/QLDnlRBlOw6fa52SrF1GJ1vt3AHbLgz/KCrBBzdK0YoFLUEhGc P6D+9TI3Va+35R4FH7Ted5PBgeZW2cHtNtStfZiPiLWnHwDsDV3KOCnUMSTT6vj1nSLp WQnPz4jKtzlFm1dag6Ij+jqH16KjM9frILvQSOncG1UeIatL71uYlLBA+IzI0o+QGx/q FKa2PzxeovLI3eJY7auv1C4D2kar3A30PXO1rPlcnX3rm2Rb0Kz7ZLpDJFdf1SGhC5WH 7LWw== X-Gm-Message-State: AA+aEWbsLb/zrwRfTWLdT+IP7NvLeQMmiLAcoEnNdBJWeXii3vWhjaOa Yj9eaSLm46d/aQPfJrBjkye/qEoZfdmVesUD+p8= X-Received: by 2002:a1c:9692:: with SMTP id y140mr16645135wmd.67.1544014698977; Wed, 05 Dec 2018 04:58:18 -0800 (PST) MIME-Version: 1.0 References: <065643a0-f9aa-a361-715a-03ca978d9228@roeck-us.net> <20181128041555.GE31885@thunk.org> <2547416.7Vy7A2kRpU@siriux> In-Reply-To: From: Andrey Melnikov Date: Wed, 5 Dec 2018 15:58:06 +0300 Message-ID: Subject: Re: ext4 file system corruption with v4.19.3 / v4.19.4 To: jrf@mailbox.org Cc: "Theodore Ts'o" , linux-kernel@vger.kernel.org, linux-ext4@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org =D0=BF=D0=BD, 3 =D0=B4=D0=B5=D0=BA. 2018 =D0=B3. =D0=B2 01:11, Rainer Fiebi= g : > > Am 02.12.18 um 21:19 schrieb Andrey Melnikov: > > =D1=87=D1=82, 29 =D0=BD=D0=BE=D1=8F=D0=B1. 2018 =D0=B3. =D0=B2 01:08, R= ainer Fiebig : > >> > >> Am 28.11.18 um 22:13 schrieb Andrey Melnikov: > >>> =D1=81=D1=80, 28 =D0=BD=D0=BE=D1=8F=D0=B1. 2018 =D0=B3. =D0=B2 18:55,= Rainer Fiebig : > >>>> > >>>> Am Mittwoch, 28. November 2018, 13:02:56 schrieb Andrey Jr. Melnikov= : > >>>>> In gmane.comp.file-systems.ext4 Theodore Y. Ts'o wr= ote: > >>>>>> On Wed, Nov 28, 2018 at 03:16:33AM +0300, Andrey Jr. Melnikov wrot= e: > >>>>>>> Corrupted inodes - always directory, not touched at least year or > >>>>>>> more for writing. Something wrong when updating atime? > >>>>>> > >>>>>> We're not sure. The frustrating thing is that it's not reproducin= g > >>>>>> for me. I run extensive regression tests, and I'm using 4.19 on m= y > >>>>>> development laptop without notcing any problems. If I could repro= duce > >>>>>> it, I could debug it, but since I can't, I need to rely on those w= ho > >>>>>> are seeing the problem to help pinpoint the problem. > >>>>> > >>>>> My workstation hit this bug every time after boot. If you have an i= dea - I > >>>>> may test it. > >>>>> > >>>>>> I'm trying to figure out common factors from those people who are > >>>>>> reporting problems. > >>>>>> > >>>>>> (a) What distribution are you running (it appears that many people > >>>>>> reporting problems are running Ubuntu, but this may be a sampling > >>>>>> issue; lots of people run Ubuntu)? (For the record, I'm using Deb= ian > >>>>>> Testing.) > >>>>> > >>>>> Debian sid but self-build kernel from ubuntu mainline-ppa. > >>>> > >>>> You could try a vanilla 4.19.5 from https://www.kernel.org/ > >>>> and compile it with your current .config. > >>> > >>> mainline-ppa use vanilla kernel. Patches only adds debian specific > >>> build infrastructure. > >>> > >>>> If you still see the errors, at least the Ubuntu-kernel could be rul= ed out. > >>>> > >>>> In addition, if you still see the errors: > >>>> > >>>> - backup your .config in a *different* folder (so that you can later= re-use > >>>> it) > >>>> - do a "make mrproper" (deletes the .config, see above) > >>>> - do a "make defconfig" > >>>> - and compile the kernel with that new .config > >>> > >>> defconfig is great - for abstract hardware in vacuum. > >>> > >>>> If you still have the problem after that, you may want to learn how = to bisect. > >>>> ;) > >>> I'm already know how-to bisect. From kernel 2.0 era. Without git ;) > >>> > >>> This problem simply non-bisectable, when same kernel corrupt FS on my > >>> workstation but normally working on other servers. > >>> And now - FS corrupted again with disabled CONFIG_EXT4_ENCRYPTION. Gr= eat. > >> > >> OK, - and now we are looking forward to *your* ideas how to solve this= . > > > > After four days playing games around git bisect - real winner is > > debian gcc-8.2.0-9. Upgrade it to 8.2.0-10 or use 7.3.0-30 version for > > same kernel + config - does not exhibit ext4 corruption. > > > > I think I hit this https://gcc.gnu.org/bugzilla/show_bug.cgi?id=3D87859 > > with 8.2.0-9 version. > > > Good that it works for you. But others used gcc 5.4.0 or 6.3.0 and were > hit anyway: https://bugzilla.kernel.org/show_bug.cgi?id=3D201685#c165 Depends on workload pattern. 4.19.5 built with 8.2.0-10 and 7.3.0-30 - crashed after 4 hours of usage (previous build crash in 5 min). So my assumption about broken gcc is wrong.