Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753827Ab0LKCPU (ORCPT ); Fri, 10 Dec 2010 21:15:20 -0500 Received: from mail-bw0-f45.google.com ([209.85.214.45]:48120 "EHLO mail-bw0-f45.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750753Ab0LKCPS convert rfc822-to-8bit (ORCPT ); Fri, 10 Dec 2010 21:15:18 -0500 MIME-Version: 1.0 In-Reply-To: References: <20101207182243.GB21112@redhat.com> <20101207193514.GA2921@thunk.org> <20101209180111.GF2921@thunk.org> <20101209201359.GG2921@thunk.org> <20101209231616.GA12515@basil.fritz.box> <1291945065-sup-1838@think> <20101210023852.GB3059@thunk.org> From: Jon Nelson Date: Fri, 10 Dec 2010 20:14:56 -0600 Message-ID: Subject: Re: hunt for 2.6.37 dm-crypt+ext4 corruption? (was: Re: dm-crypt barrier support is effective) To: "Ted Ts'o" , Matt , Chris Mason , Andi Kleen , Jon Nelson , Mike Snitzer , Milan Broz , linux-btrfs , dm-devel , Linux Kernel , htd , htejun , linux-ext4 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8BIT Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2854 Lines: 63 On Fri, Dec 10, 2010 at 10:54 AM, Jon Nelson wrote: > On Fri, Dec 10, 2010 at 8:58 AM, Jon Nelson wrote: >> On Fri, Dec 10, 2010 at 12:52 AM, Jon Nelson wrote: >>> On Thu, Dec 9, 2010 at 8:38 PM, Ted Ts'o wrote: >>>> On Fri, Dec 10, 2010 at 02:53:30AM +0100, Matt wrote: >>>>> >>>>> Try a kernel before 5a87b7a5da250c9be6d757758425dfeaf8ed3179 >>>>> >>>>> from the tests I've done that one showed the least or no corruption if >>>>> you count the empty /etc/env.d/03opengl as an artefact >>>> >>>> Yes, that's a good test.  Also try commit bd2d0210cf.  The patch >>>> series that is most likely to be at fault if there is a regression in >>>> between 5a87b7a5d and bd2d0210cf inclusive. >>>> >>>> I did a lot of testing before submitting it, but that wa a tricky >>>> rewrite.  If you can reproduce the problem reliably, it might be good >>>> to try commit 16828088f9 (the commit before 5a87b7a5d) and commit >>>> bd2d0210cf.  If it reliably reproduces on bd2d0210cf, but is clean on >>>> 16828088f9, then it's my ext4 block i/o submission patches, and we'll >>>> need to either figure out what's going on or back out that set of >>>> changes. >>>> >>>> If that's the case, a bisect of those changes (it's only 6 commits, so >>>> it shouldn't take long) would be most appreciated. >>> >>> I observed the behavior on bd2d0210cf in a qemu-kvm install of >>> openSUSE 11.3 (x86_64) on *totally* different host - an AMD quad-core. >>> >>> I did /not/ observe the behavior on 16828088f9 (yet). I'll run the >>> test a few more times on 1682.. >>> >>> Additionally, I am building a bisected kernel now ( >>> cb20d5188366f04d96d2e07b1240cc92170ade40 ), but won't be able to get >>> back at it for a while. >> >> cb20d5188366f04d96d2e07b1240cc92170ade40 seems OK so far. I'm going to >> try 1de3e3df917459422cb2aecac440febc8879d410 soon. > > Barring false negatives, bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc > appears to be the culprit (according to git bisect). > I will test bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc again, confirm > the behavior, and work backwards to try to reduce the possibility of > false negatives. A few additional notes, in no particular order: - For me, triggering the problem is fairly easy when encryption is involved. - I'm now 81 iterations into testing bd2d0210cf22f2bd0cef72eb97cf94fc7d31d8cc *without* encryption. Out of 81 iterations, I have 4 failures: #16, 40, 62, and 64. I will now try 1de3e3df917459422cb2aecac440febc8879d410 much more extensively. Is this useful information? -- Jon -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/