Subject: CONFIG_SLUB and reproducable general protection faults on 2.6.2x
From: "Nicholas A. Bellinger" <nab@linux-iscsi.org>
To: Bart Van Assche <bart.vanassche@gmail.com>
Cc: Vladislav Bolkhovitin <vst@vlnb.net>,
       FUJITA Tomonori <fujita.tomonori@lab.ntt.co.jp>,
       Mike Christie <michaelc@cs.wisc.edu>, linux-scsi@vger.kernel.org,
       Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
       James Bottomley <James.Bottomley@hansenpartnership.com>,
       Andrew Morton <akpm@linux-foundation.org>,
       Christoph Hellwig <hch@lst.de>, Rik van Riel <riel@redhat.com>,
       Chris Weiss <cweiss@gmail.com>,
       Linus Torvalds <torvalds@linux-foundation.org>
In-Reply-To: <1202874275.14647.52.camel@haakon2.linux-iscsi.org>
References: <e2e108260801230622x42045058v926e84c60f2be7f4@mail.gmail.com>
	 <alpine.LFD.1.00.0802041018390.3034@hp.linux-foundation.org>
	 <1202151989.11265.576.camel@haakon2.linux-iscsi.org>
	 <alpine.LFD.1.00.0802041130140.3034@hp.linux-foundation.org>
	 <20080204224314.113afe7b@core> <47A79A10.4070706@garzik.org>
	 <47A8B29B.8050406@vlnb.net> <47A8B510.8000807@garzik.org>
	 <47A8B757.10101@vlnb.net>
	 <1202256667.2220.83.camel@haakon2.linux-iscsi.org>
	 <e2e108260802120805v4c8e07d6if59c881213c4ae97@mail.gmail.com>
	 <1202874275.14647.52.camel@haakon2.linux-iscsi.org>
Content-Type: text/plain
Date: Tue, 12 Feb 2008 22:18:27 -0800
Message-Id: <1202883507.21178.1.camel@haakon2.linux-iscsi.org>
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3596
Lines: 80

On Tue, 2008-02-12 at 19:57 -0800, Nicholas A. Bellinger wrote:
> Greetings all,
> 
> On Tue, 2008-02-12 at 17:05 +0100, Bart Van Assche wrote:
> > On Feb 6, 2008 1:11 AM, Nicholas A. Bellinger <nab@linux-iscsi.org> wrote:
> > > I have always observed the case with LIO SE/iSCSI target mode ...
> > 
> > Hello Nicholas,
> > 
> > Are you sure that the LIO-SE kernel module source code is ready for
> > inclusion in the mainstream Linux kernel ? As you know I tried to test
> > the LIO-SE iSCSI target. Already while configuring the target I
> > encountered a kernel crash that froze the whole system. I can
> > reproduce this kernel crash easily, and I reported it 11 days ago on
> > the LIO-SE mailing list (February 4, 2008). One of the call stacks I
> > posted shows a crash in mempool_alloc() called from jbd. Or: the crash
> > is most likely the result of memory corruption caused by LIO-SE.
> > 
> 
> So I was able to FINALLY track this down to:
> 
> -# CONFIG_SLUB_DEBUG is not set
> -# CONFIG_SLAB is not set
> -CONFIG_SLUB=y
> +CONFIG_SLAB=y
> 
> in both your and Chris Weiss's configs that was causing the
> reproduceable general protection faults.  I also disabled
> CONFIG_RELOCATABLE and crash dump because I was debugging using kdb in
> x86_64 VM on 2.6.24 with your config.  I am pretty sure you can leave
> this (crash dump) in your config for testing.
> 
> This can take a while to compile and take up alot of space, esp. with
> all of the kernel debug options enabled, which on 2.6.24, really amounts
> to alot of CPU time when building.  Also with your original config, I
> was seeing some strange undefined module objects after Stage 2 Link with
> iscsi_target_mod with modpost with the SLUB the lockups (which are not
> random btw, and are tracked back to __kmalloc())..  Also, at module load
> time with the original config, there where some warning about symbol
> objects (I believe it was SCSI related, same as the ones with modpost).
> 
> In any event, the dozen 1000 loop discovery test is now working fine (as
> well as IPoIB) with the above config change, and you should be ready to
> go for your testing.
> 
> Tomo, Vlad, Andrew and Co:
> 
> Do you have any ideas why this would be the case with LIO-Target..?  Is
> anyone else seeing something similar to this with their target mode
> (mabye its all out of tree code..?) that is having an issue..? I am
> using Debian x86_64 and Bart and Chris are using Ubuntu x86_64 and we
> both have this problem with CONFIG_SLUB on >= 2.6.22 kernel.org
> kernels. 
> 
> Also, I will recompile some of my non x86 machines with the above
> enabled and see if I can reproduce..  Here the Bart's config again:
> 
> http://groups.google.com/group/linux-iscsi-target-dev/browse_thread/thread/30835aede1028188
> 

This is also failing on CONFIG_SLUB on 2.6.24 ppc64.  Since the rest of
the system seems to work fine, my only guess it may be related to the
fact that the module is being compiled out of tree.  I took a quick
glance at what kbuild was using for compiler and linker parameters, but
nothing looked out of the ordinary.

I will take a look with kdb and SLUB re-enabled on x86_64 and see if this
helps shed any light on the issue.  Is anyone else seeing an issue with CONFIG_SLUB..?

Also, I wonder who else aside from Ubuntu is using this by default in
their .config..?


--nab

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/