2007-05-20 03:09:47

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

It's an AM2 5600 chip on AMD 690G chipset on Asus M2A-VM board.

With CONFIG_SLUB it panics. With CONFIG_SLAB all is well. (Same behaviour on
2.6.22-rc1 also)

Unfortunately, it panics so early, it doesn't make it to serial console. So
here's what I have written down (sorry if I missed some important info; I hope
it gives a clue or two):

Process swapper (...)
...
Call Trace:
kernel_init+0x16c/0x2e1
trace_hardirqs_on_thunk+0x35/0x37
trace_hardirqs_on+0x116/0x13a
child_rip+0xa/ox12
restore_args+0x0/0x30
kernel_init+0x0/0x2e1
child_rip+0x0/0x12

Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80
RIP [...] slab_sysfs_init+0x49/0x98
RSP [...]
kernel panic - not syncing: Attempted to kill init!

The .config is attached (with the CONFIG_SLUB).

(It's vanilla 2.6.22-rc2 plus a patch, which is submitted to libata maintainer
already, from Tejun Heo for SB600 AHCI to work on a system with >4GB.)

Any patches welcome.

Thanks


___________________________________________________________________________________
How would you spend $50,000 to create a more sustainable environment in Australia? Go to Yahoo!7 Answers and share your idea.
http://advision.webevents.yahoo.com/aunz/lifestyle/answers/y7ans-babp_reg.html


Attachments:
config-2.6.22-rc2-slub (42.53 kB)
pat938819783

2007-05-20 07:12:38

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Sun, 20 May 2007, Srihari Vijayaraghavan wrote:

> Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80
> RIP [...] slab_sysfs_init+0x49/0x98
> RSP [...]
> kernel panic - not syncing: Attempted to kill init!

sysfs? If you build without CONFIG_SLUB_DEBUG then SLUB will not use
sysfs. You can switch that off in the embedded section.

2007-05-20 07:41:15

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

--- Christoph Lameter <[email protected]> wrote:
> On Sun, 20 May 2007, Srihari Vijayaraghavan wrote:
>
> > Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80
> > RIP [...] slab_sysfs_init+0x49/0x98
> > RSP [...]
> > kernel panic - not syncing: Attempted to kill init!
>
> sysfs? If you build without CONFIG_SLUB_DEBUG then SLUB will not use
> sysfs. You can switch that off in the embedded section.

Fair enough. Would you the section name it appears in 'menuconfig'?

I looked under 'Kernel hacking' section (where the slab debugging appears if
slab is turned on as 'Debug slab memory allocation') & in the 'General setup'
section also. I unable to figure out how to disable slub debugging :-(.

Pls give me a pointer to it.

Thanks

(Won't it be right to disable it by default when a user selects slub such that
a user needs to manually turn on the slub debugging in menuconfig?)



___________________________________________________________________________________
How would you spend $50,000 to create a more sustainable environment in Australia? Go to Yahoo!7 Answers and share your idea.
http://advision.webevents.yahoo.com/aunz/lifestyle/answers/y7ans-babp_reg.html

2007-05-20 13:24:23

by Satyam Sharma

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On 5/20/07, Srihari Vijayaraghavan <[email protected]> wrote:
> --- Christoph Lameter <[email protected]> wrote:
> > On Sun, 20 May 2007, Srihari Vijayaraghavan wrote:
> >
> > > Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80
> > > RIP [...] slab_sysfs_init+0x49/0x98
> > > RSP [...]
> > > kernel panic - not syncing: Attempted to kill init!
> >
> > sysfs? If you build without CONFIG_SLUB_DEBUG then SLUB will not use
> > sysfs. You can switch that off in the embedded section.
>
> Fair enough. Would you the section name it appears in 'menuconfig'?
>
> I looked under 'Kernel hacking' section (where the slab debugging appears if
> slab is turned on as 'Debug slab memory allocation') & in the 'General setup'
> section also. I unable to figure out how to disable slub debugging :-(.

SLUB_DEBUG is both "bool if EMBEDDED" and "depends on SLUB", which
means just getting it to be visible in "menuconfig" is quite a challenge :-)

You first need to go into the choice where you pick the allocators ("Choose
allocator --->" which is the last option in General Setup), and then pick
SLUB as your allocator.

Then, from the General Setup again, turn on the "Configure standard
kernel features (for small systems) --->" just above the previous option
and then go into that menu. Now the option to disable SLUB_DEBUG
(which is y by default) becomes visible, so just deselect it to disable it.

2007-05-20 13:35:27

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

Linux version 2.6.22-rc2-ahci-slub (hari@fedora7) (gcc version 4.1.2 20070502 (Red Hat 4.1.2-12)) #6 SMP Sun May 20 23:05:08 EST 2007
Command line: ro root=LABEL=/1234 console=ttyS0,115200 console=tty0
BIOS-provided physical RAM map:
BIOS-e820: 0000000000000000 - 000000000009f000 (usable)
BIOS-e820: 000000000009f000 - 00000000000a0000 (reserved)
BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
BIOS-e820: 0000000000100000 - 00000000dfee0000 (usable)
BIOS-e820: 00000000dfee0000 - 00000000dfee3000 (ACPI NVS)
BIOS-e820: 00000000dfee3000 - 00000000dfef0000 (ACPI data)
BIOS-e820: 00000000dfef0000 - 00000000dff00000 (reserved)
BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
BIOS-e820: 00000000fec00000 - 0000000100000000 (reserved)
BIOS-e820: 0000000100000000 - 0000000120000000 (usable)
end_pfn_map = 1179648
DMI 2.4 present.
ACPI: RSDP 000F8110, 0024 (r2 ATI )
ACPI: XSDT DFEE30C0, 0044 (r1 ATI ASUSACPI 42302E31 AWRD 0)
ACPI: FACP DFEE83C0, 00F4 (r3 ATI ASUSACPI 42302E31 AWRD 0)
ACPI: DSDT DFEE3240, 511E (r1 ATI ASUSACPI 1000 MSFT 3000000)
ACPI: FACS DFEE0000, 0040
ACPI: SSDT DFEE85C0, 02CC (r1 PTLTD POWERNOW 1 LTP 1)
ACPI: MCFG DFEE8980, 003C (r1 ATI ASUSACPI 42302E31 AWRD 0)
ACPI: APIC DFEE8500, 0068 (r1 ATI ASUSACPI 42302E31 AWRD 0)
Scanning NUMA topology in Northbridge 24
No NUMA configuration found
Faking a node at 0000000000000000-0000000120000000
Bootmem setup node 0 0000000000000000-0000000120000000
Zone PFN ranges:
DMA 0 -> 4096
DMA32 4096 -> 1048576
Normal 1048576 -> 1179648
early_node_map[3] active PFN ranges
0: 0 -> 159
0: 256 -> 917216
0: 1048576 -> 1179648
ATI board detected. Disabling timer routing over 8254.
ACPI: PM-Timer IO Port: 0x4008
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 (Bootup-CPU)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
Processor #1
ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 low level)
Setting APIC routing to flat
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at f1000000 (gap: f0000000:ec00000)
PERCPU: Allocating 33672 bytes of per cpu data
Built 1 zonelists. Total pages: 1030075
Kernel command line: ro root=LABEL=/1234 console=ttyS0,115200 console=tty0
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Marking TSC unstable due to TSCs unsynchronized
time.c: Detected 2799.974 MHz processor.
Console: colour VGA+ 80x25
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES: 8
... MAX_LOCK_DEPTH: 30
... MAX_LOCKDEP_KEYS: 2048
... CLASSHASH_SIZE: 1024
... MAX_LOCKDEP_ENTRIES: 8192
... MAX_LOCKDEP_CHAINS: 16384
... CHAINHASH_SIZE: 8192
memory used by lock dependency info: 1648 kB
per task-struct memory footprint: 1680 bytes
Checking aperture...
CPU 0: aperture @ 8000000 size 32 MB
Aperture too small (32 MB)
No AGP bridge found
Your BIOS doesn't leave a aperture memory hole
Please enable the IOMMU option in the BIOS setup
This costs you 64 MB of RAM
Mapping aperture over 65536 KB of RAM @ 8000000
Memory: 4058408k/4718592k available (2102k kernel code, 134356k reserved, 1229k data, 308k init)
SLUB: Genslabs=23, HWalign=64, Order=0-1, MinObjects=4, Processors=2, Nodes=1
Calibrating delay using timer specific routine.. 5604.66 BogoMIPS (lpj=11209328)
Security Framework v1.0.0 initialized
SELinux: Initializing.
selinux_register_security: Registering secondary module capability
Capability LSM initialized as secondary
Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 0/0 -> Node 0
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
Freeing SMP alternatives: 22k freed
ACPI: Core revision 20070126
Using local APIC timer interrupts.
result 12499895
Detected 12.499 MHz APIC timer.
lockdep: not fixing up alternatives.
Booting processor 1/2 APIC 0x1
Initializing CPU#1
Calibrating delay using timer specific routine.. 5600.19 BogoMIPS (lpj=11200386)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 1024K (64 bytes/line)
CPU 1/1 -> Node 0
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
AMD Athlon(tm) 64 X2 Dual Core Processor 5600+ stepping 03
Brought up 2 CPUs
migration_cost=50
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: Using MMCONFIG at e0000000 - efffffff
PCI: No mmconfig possible on device 00:18
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI: Transparent bridge - 0000:00:14.4
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 11) *0, disabled.
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 10 11) *0, disabled.
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 10 11) *0, disabled.
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 10 11) *0, disabled.
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11) *0, disabled.
ACPI: PCI Interrupt Link [LNK0] (IRQs 3 4 5 6 7 10 *11)
ACPI: PCI Interrupt Link [LNK1] (IRQs 3 4 5 6 7 10 11) *0, disabled.
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 11 devices
ACPI: ACPI bus type pnp unregistered
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq". If it helps, post a report
NetLabel: Initializing
NetLabel: domain hash size = 128
NetLabel: protocols = UNLABELED CIPSOv4
NetLabel: unlabeled traffic allowed by default
PCI-DMA: Disabling AGP.
PCI-DMA: aperture base @ 8000000 size 65536 KB
PCI-DMA: using GART IOMMU.
PCI-DMA: Reserving 64MB of IOMMU area in the AGP aperture
ACPI: RTC can wake from S4
pnp: 00:01: ioport range 0x4100-0x411f has been reserved
pnp: 00:01: ioport range 0x228-0x22f has been reserved
pnp: 00:01: ioport range 0x40b-0x40b has been reserved
pnp: 00:01: ioport range 0x4d6-0x4d6 has been reserved
pnp: 00:01: ioport range 0xc00-0xc01 has been reserved
pnp: 00:01: ioport range 0xc14-0xc14 has been reserved
pnp: 00:01: ioport range 0xc50-0xc52 has been reserved
pnp: 00:01: ioport range 0xc6c-0xc6d has been reserved
pnp: 00:09: iomem range 0xe0000000-0xefffffff could not be reserved
pnp: 00:0a: iomem range 0xf0000-0xf3fff could not be reserved
pnp: 00:0a: iomem range 0xf4000-0xf7fff could not be reserved
pnp: 00:0a: iomem range 0xf8000-0xfbfff could not be reserved
pnp: 00:0a: iomem range 0xfc000-0xfffff could not be reserved
PCI: Bridge: 0000:00:02.0
IO window: d000-dfff
MEM window: fdc00000-fdcfffff
PREFETCH window: f0000000-f7ffffff
PCI: Bridge: 0000:00:07.0
IO window: e000-efff
MEM window: fdb00000-fdbfffff
PREFETCH window: fdf00000-fdffffff
PCI: Bridge: 0000:00:14.4
IO window: c000-cfff
Time: acpi_pm clocksource has been installed.
MEM window: fde00000-fdefffff
PREFETCH window: fdd00000-fddfffff
NET: Registered protocol family 2
IP route cache hash table entries: 131072 (order: 8, 1048576 bytes)
TCP established hash table entries: 524288 (order: 13, 33554432 bytes)
TCP bind hash table entries: 65536 (order: 9, 3670016 bytes)
TCP: Hash tables configured (established 524288 bind 65536)
TCP reno registered
checking if image is initramfs... it is
Freeing initrd memory: 3031k freed
audit: initializing netlink socket (disabled)
audit(1179666942.608:1): initialized
Total HugeTLB memory allocated, 0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
io scheduler noop registered
io scheduler cfq registered (default)
assign_interrupt_mode Found MSI capability
assign_interrupt_mode Found MSI capability
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
ACPI: Fan [FAN] (on)
ACPI: Thermal Zone [THRM] (40 C)
Real Time Clock Driver v1.12ac
Non-volatile memory driver v1.2
Linux agpgart interface v0.102 (c) Dave Jones
Serial: 8250/16550 driver $Revision: 1.90 $ 2 ports, IRQ sharing enabled
00:08: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
RAMDISK driver initialized: 16 RAM disks of 16384K size 4096 blocksize
usbcore: registered new interface driver libusual
PNP: No PS/2 controller found. Probing ports directly.
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
drivers/hid/usbhid/hid-core.c: v2.6:USB HID core driver
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 1
NET: Registered protocol family 17
powernow-k8: Found 2 AMD Athlon(tm) 64 X2 Dual Core Processor 5600+ processors (version 2.00.00)
powernow-k8: 0 : fid 0x14 (2800 MHz), vid 0x8
powernow-k8: 1 : fid 0x12 (2600 MHz), vid 0xa
powernow-k8: 2 : fid 0x10 (2400 MHz), vid 0xc
powernow-k8: 3 : fid 0xe (2200 MHz), vid 0xe
powernow-k8: 4 : fid 0xc (2000 MHz), vid 0x10
powernow-k8: 5 : fid 0xa (1800 MHz), vid 0x10
powernow-k8: 6 : fid 0x2 (1000 MHz), vid 0x12
Freeing unused kernel memory: 308k freed
BUG: spinlock bad magic on CPU#1, init/1
lock: ffff81011ec0a100, .magic: ffff8101, .owner: <none>/-1, .owner_cpu: -1

Call Trace:
[<ffffffff802fc516>] _raw_spin_lock+0x22/0xf6
[<ffffffff8026f684>] vma_adjust+0x219/0x454
[<ffffffff8026f684>] vma_adjust+0x219/0x454
[<ffffffff8026fe0d>] vma_merge+0x147/0x1f4
[<ffffffff80270b55>] do_mmap_pgoff+0x414/0x7c7
[<ffffffff8040aa34>] _spin_unlock_irq+0x24/0x27
[<ffffffff8020f528>] sys_mmap+0xe5/0x110
[<ffffffff80209dde>] system_call+0x7e/0x83

ohci_hcd: 2006 August 04 USB 1.1 'Open' Host Controller (OHCI) Driver
ACPI: PCI Interrupt 0000:00:13.0[A] -> GSI 16 (level, low) -> IRQ 16
ohci_hcd 0000:00:13.0: OHCI Host Controller
ohci_hcd 0000:00:13.0: new USB bus registered, assigned bus number 1
ohci_hcd 0000:00:13.0: irq 16, io mem 0xfe02e000
PM: Adding info for usb:usb1
PM: Adding info for No Bus:usbdev1.1_ep00
usb usb1: configuration #1 chosen from 1 choice
PM: Adding info for usb:1-0:1.0
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
PM: Adding info for No Bus:usbdev1.1_ep81
ACPI: PCI Interrupt 0000:00:13.1[B] -> GSI 17 (level, low) -> IRQ 17
ohci_hcd 0000:00:13.1: OHCI Host Controller
ohci_hcd 0000:00:13.1: new USB bus registered, assigned bus number 2
ohci_hcd 0000:00:13.1: irq 17, io mem 0xfe02d000
PM: Adding info for usb:usb2
PM: Adding info for No Bus:usbdev2.1_ep00
usb usb2: configuration #1 chosen from 1 choice
PM: Adding info for usb:2-0:1.0
hub 2-0:1.0: USB hub found
hub 2-0:1.0: 2 ports detected
PM: Adding info for No Bus:usbdev2.1_ep81
ACPI: PCI Interrupt 0000:00:13.2[C] -> GSI 18 (level, low) -> IRQ 18
ohci_hcd 0000:00:13.2: OHCI Host Controller
ohci_hcd 0000:00:13.2: new USB bus registered, assigned bus number 3
ohci_hcd 0000:00:13.2: irq 18, io mem 0xfe02c000
PM: Adding info for usb:usb3
PM: Adding info for No Bus:usbdev3.1_ep00
usb usb3: configuration #1 chosen from 1 choice
PM: Adding info for usb:3-0:1.0
hub 3-0:1.0: USB hub found
hub 3-0:1.0: 2 ports detected
PM: Adding info for No Bus:usbdev3.1_ep81
ACPI: PCI Interrupt 0000:00:13.3[B] -> GSI 17 (level, low) -> IRQ 17
ohci_hcd 0000:00:13.3: OHCI Host Controller
ohci_hcd 0000:00:13.3: new USB bus registered, assigned bus number 4
ohci_hcd 0000:00:13.3: irq 17, io mem 0xfe02b000
usb 2-1: new low speed USB device using ohci_hcd and address 2
PM: Adding info for usb:usb4
PM: Adding info for No Bus:usbdev4.1_ep00
usb usb4: configuration #1 chosen from 1 choice
PM: Adding info for usb:4-0:1.0
hub 4-0:1.0: USB hub found
hub 4-0:1.0: 2 ports detected
PM: Adding info for No Bus:usbdev4.1_ep81
PM: Adding info for usb:2-1
PM: Adding info for No Bus:usbdev2.2_ep00
usb 2-1: configuration #1 chosen from 1 choice
PM: Adding info for usb:2-1:1.0
ACPI: PCI Interrupt 0000:00:13.4[C] -> <6>input: Logitech USB Receiver as /class/input/input0
input: USB HID v1.10 Keyboard [Logitech USB Receiver] on usb-0000:00:13.1-1
PM: Adding info for No Bus:usbdev2.2_ep81
PM: Adding info for usb:2-1:1.1
drivers/hid/usbhid/hid-core.c: Fixing up Logitech keyboard report descriptor
GSI 18 (level, low) -> IRQ 18
input: Logitech USB Receiver as /class/input/input1
PM: Adding info for No Bus:hiddev0
input,hiddev96: USB HID v1.10 Mouse [Logitech USB Receiver] on usb-0000:00:13.1-1
PM: Adding info for No Bus:usbdev2.2_ep82
ohci_hcd 0000:00:13.4: OHCI Host Controller
ohci_hcd 0000:00:13.4: new USB bus registered, assigned bus number 5
ohci_hcd 0000:00:13.4: irq 18, io mem 0xfe02a000
PM: Adding info for usb:usb5
PM: Adding info for No Bus:usbdev5.1_ep00
usb usb5: configuration #1 chosen from 1 choice
PM: Adding info for usb:5-0:1.0
hub 5-0:1.0: USB hub found
hub 5-0:1.0: 2 ports detected
PM: Adding info for No Bus:usbdev5.1_ep81
ACPI: PCI Interrupt 0000:00:13.5[D] -> GSI 19 (level, low) -> IRQ 19
ehci_hcd 0000:00:13.5: EHCI Host Controller
ehci_hcd 0000:00:13.5: new USB bus registered, assigned bus number 6
ehci_hcd 0000:00:13.5: debug port 1
ehci_hcd 0000:00:13.5: irq 19, io mem 0xfe029000
ehci_hcd 0000:00:13.5: USB 2.0 started, EHCI 1.00, driver 10 Dec 2004
usb 2-1: USB disconnect, address 2
PM: Adding info for usb:usb6
PM: Adding info for No Bus:usbdev6.1_ep00
PM: Removing info for No Bus:usbdev2.2_ep81
PM: Removing info for usb:2-1:1.0
PM: Removing info for No Bus:usbdev2.2_ep82
PM: Removing info for No Bus:hiddev0
PM: Removing info for usb:2-1:1.1
PM: Removing info for No Bus:usbdev2.2_ep00
PM: Removing info for usb:2-1
usb usb6: configuration #1 chosen from 1 choice
PM: Adding info for usb:6-0:1.0
hub 6-0:1.0: USB hub found
hub 6-0:1.0: 10 ports detected
PM: Adding info for No Bus:usbdev6.1_ep81
SCSI subsystem initialized
libata version 2.20 loaded.
ACPI: PCI Interrupt 0000:00:14.1[A] -> GSI 16 (level, low) -> IRQ 16
scsi0 : pata_atiixp
PM: Adding info for No Bus:host0
scsi1 : pata_atiixp
PM: Adding info for No Bus:host1
ata1: PATA max UDMA/100 cmd 0x00000000000101f0 ctl 0x00000000000103f6 bmdma 0x000000000001f900 irq 14
ata2: PATA max UDMA/100 cmd 0x0000000000010170 ctl 0x0000000000010376 bmdma 0x000000000001f908 irq 15
usb 2-1: new low speed USB device using ohci_hcd and address 3
ata1.00: ata_hpa_resize 1: sectors = 234435439, hpa_sectors = 234441648
ata1.00: Host Protected Area detected:
current size: 234435439 sectors
native size: 234441648 sectors
ata1.00: ATA-6: ST3120026A, 3.06, max UDMA/100
ata1.00: 234435439 sectors, multi 1: LBA48
ata1.01: ATAPI, max UDMA/33
ata1.00: ata_hpa_resize 1: sectors = 234435439, hpa_sectors = 234441648
ata1.00: Host Protected Area detected:
current size: 234435439 sectors
native size: 234441648 sectors
ata1.00: configured for UDMA/100
PM: Adding info for usb:2-1
PM: Adding info for No Bus:usbdev2.3_ep00
usb 2-1: configuration #1 chosen from 1 choice
PM: Adding info for usb:2-1:1.0
input: Logitech USB Receiver as /class/input/input2
input: USB HID v1.10 Keyboard [Logitech USB Receiver] on usb-0000:00:13.1-1
PM: Adding info for No Bus:usbdev2.3_ep81
PM: Adding info for usb:2-1:1.1
drivers/hid/usbhid/hid-core.c: Fixing up Logitech keyboard report descriptor
input: Logitech USB Receiver as /class/input/input3
PM: Adding info for No Bus:hiddev0
input,hiddev96: USB HID v1.10 Mouse [Logitech USB Receiver] on usb-0000:00:13.1-1
PM: Adding info for No Bus:usbdev2.3_ep82
ata1.01: configured for UDMA/33
ata2: device not ready (errno=-19), forcing hardreset
ata2: BUG: prereset() requested invalid reset type
PM: Adding info for No Bus:target0:0:0
scsi 0:0:0:0: Direct-Access ATA ST3120026A 3.06 PQ: 0 ANSI: 5
PM: Adding info for scsi:0:0:0:0
sd 0:0:0:0: [sda] 234435439 512-byte hardware sectors (120031 MB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 0:0:0:0: [sda] 234435439 512-byte hardware sectors (120031 MB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sda: sda1 sda2 sda3 sda4 < sda5 sda6 sda7 sda8 sda9 sda10 sda11 sda12 sda13 sda14 >
sd 0:0:0:0: [sda] Attached SCSI disk
PM: Adding info for No Bus:target0:0:1
scsi 0:0:1:0: CD-ROM HL-DT-ST DVDRAM GSA-4163B A102 PQ: 0 ANSI: 5
PM: Adding info for scsi:0:0:1:0
kjournald starting. Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
audit(1179666950.984:2): enforcing=1 old_enforcing=0 auid=4294967295
security: 3 users, 6 roles, 1817 types, 78 bools, 1 sens, 1024 cats
security: 60 classes, 65667 rules
SELinux: Completing initialization.
SELinux: Setting up existing superblocks.
SELinux: initialized (dev sda1, type ext3), uses xattr
SELinux: initialized (dev usbfs, type usbfs), uses genfs_contexts
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev selinuxfs, type selinuxfs), uses genfs_contexts
SELinux: initialized (dev mqueue, type mqueue), uses transition SIDs
SELinux: initialized (dev hugetlbfs, type hugetlbfs), uses genfs_contexts
SELinux: initialized (dev devpts, type devpts), uses transition SIDs
SELinux: initialized (dev inotifyfs, type inotifyfs), uses genfs_contexts
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev futexfs, type futexfs), uses genfs_contexts
SELinux: initialized (dev anon_inodefs, type anon_inodefs), not configured for labeling
SELinux: initialized (dev pipefs, type pipefs), uses task SIDs
SELinux: initialized (dev sockfs, type sockfs), uses task SIDs
SELinux: initialized (dev cpuset, type cpuset), uses genfs_contexts
SELinux: initialized (dev proc, type proc), uses genfs_contexts
SELinux: initialized (dev bdev, type bdev), uses genfs_contexts
SELinux: initialized (dev rootfs, type rootfs), uses genfs_contexts
SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
audit(1179666951.376:3): policy loaded auid=4294967295
PM: Removing info for No Bus:vcs1
PM: Removing info for No Bus:vcsa1
PM: Adding info for No Bus:vcs1
PM: Adding info for No Bus:vcsa1
PM: Removing info for No Bus:vcs1
PM: Removing info for No Bus:vcsa1
PM: Adding info for No Bus:vcs1
PM: Adding info for No Bus:vcsa1
PM: Removing info for No Bus:vcs1
PM: Removing info for No Bus:vcsa1
PM: Adding info for No Bus:vcs1
PM: Adding info for No Bus:vcsa1
PM: Removing info for No Bus:vcs1
PM: Removing info for No Bus:vcsa1
PM: Adding info for No Bus:vcs1
PM: Adding info for No Bus:vcsa1
PM: Removing info for No Bus:vcs1
PM: Removing info for No Bus:vcsa1
PM: Adding info for No Bus:vcs1
PM: Adding info for No Bus:vcsa1
BUG: spinlock lockup on CPU#0, sh/385, ffff81011ca8ffc0

Call Trace:
[<ffffffff802fc5c3>] _raw_spin_lock+0xcf/0xf6
[<ffffffff80273352>] anon_vma_unlink+0x1c/0x68
[<ffffffff80273352>] anon_vma_unlink+0x1c/0x68
[<ffffffff8026ddf4>] free_pgtables+0x69/0xc3
[<ffffffff8026f0c0>] exit_mmap+0x91/0xeb
[<ffffffff802295a1>] mmput+0x45/0xb8
[<ffffffff80288ad8>] flush_old_exec+0x65f/0x941
[<ffffffff80284c04>] vfs_read+0x13f/0x153
[<ffffffff802b5a33>] load_elf_binary+0x0/0x197f
[<ffffffff802b5e93>] load_elf_binary+0x460/0x197f
[<ffffffff80263557>] __alloc_pages+0x72/0x2d4
[<ffffffff802b5a33>] load_elf_binary+0x0/0x197f
[<ffffffff80287d08>] search_binary_handler+0xc4/0x25f
[<ffffffff80289834>] do_execve+0x188/0x231
[<ffffffff80208bf7>] sys_execve+0x36/0x8a
[<ffffffff8020a1d7>] stub_execve+0x67/0xb0


Attachments:
dmesg-2.6.22-slub.txt (19.60 kB)
991354462-dmesg-2.6.22-slub.txt

2007-05-20 13:38:03

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

--- Satyam Sharma <[email protected]> wrote:
> On 5/20/07, Srihari Vijayaraghavan <[email protected]>
> wrote:
> > --- Christoph Lameter <[email protected]> wrote:
> > > On Sun, 20 May 2007, Srihari Vijayaraghavan wrote:
> > >
> > > > Code: 0f ob eb fe 48 8b 1b 48 8b 0x 0f 18 08 48 81 fb 60 cb 51 80
> > > > RIP [...] slab_sysfs_init+0x49/0x98
> > > > RSP [...]
> > > > kernel panic - not syncing: Attempted to kill init!
> > >
> > > sysfs? If you build without CONFIG_SLUB_DEBUG then SLUB will not use
> > > sysfs. You can switch that off in the embedded section.
> >
> > Fair enough. Would you the section name it appears in 'menuconfig'?
> >
> > I looked under 'Kernel hacking' section (where the slab debugging appears
> if
> > slab is turned on as 'Debug slab memory allocation') & in the 'General
> setup'
> > section also. I unable to figure out how to disable slub debugging :-(.
>
> SLUB_DEBUG is both "bool if EMBEDDED" and "depends on SLUB", which
> means just getting it to be visible in "menuconfig" is quite a challenge :-)
>
> You first need to go into the choice where you pick the allocators ("Choose
> allocator --->" which is the last option in General Setup), and then pick
> SLUB as your allocator.
>
> Then, from the General Setup again, turn on the "Configure standard
> kernel features (for small systems) --->" just above the previous option
> and then go into that menu. Now the option to disable SLUB_DEBUG
> (which is y by default) becomes visible, so just deselect it to disable it.

Thanks for that. I've just figured that out a few minutes ago :-). Was a
tricky one indeed. Then I captured and posted the current kernel bug report
while having slub just then.

Thanks



___________________________________________________________________________________
How would you spend $50,000 to create a more sustainable environment in Australia? Go to Yahoo!7 Answers and share your idea.
http://advision.webevents.yahoo.com/aunz/lifestyle/answers/y7ans-babp_reg.html

2007-05-21 16:30:23

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Sun, 20 May 2007, Srihari Vijayaraghavan wrote:

> With no CONFIG_SLUB_DEBUG, things have slightly improved. No more panic. Good.
> Serial console is working. Good. But there is another problem:

Well this indicates that something destroys the sysfs pointer structure
that SLUB is using. Could you reenable CONFIG_SLUB_DEBUG and boot with the
option "slub_debug" on the kernel command line?

Also enable the list debug options in kernel hacking.

This should show up on the console since the failure is in slub_sysfs_init
which is long after the console has been enabled. I wonder why it did not
show so far?

> Freeing unused kernel memory: 308k freed
> BUG: spinlock bad magic on CPU#1, init/1
> lock: ffff81011ec0a100, .magic: ffff8101, .owner: <none>/-1, .owner_cpu: -1
>
> Call Trace:
> [<ffffffff802fc516>] _raw_spin_lock+0x22/0xf6
> [<ffffffff8026f684>] vma_adjust+0x219/0x454
> [<ffffffff8026f684>] vma_adjust+0x219/0x454
> [<ffffffff8026fe0d>] vma_merge+0x147/0x1f4
> [<ffffffff80270b55>] do_mmap_pgoff+0x414/0x7c7
> [<ffffffff8040aa34>] _spin_unlock_irq+0x24/0x27
> [<ffffffff8020f528>] sys_mmap+0xe5/0x110
> [<ffffffff80209dde>] system_call+0x7e/0x83

Hmmmm..... We have seen this before

http://marc.info/?l=linux-kernel&m=117891943401284&w=2

I suspect this is due to an interaction with other debug flags you have
set. What I see in your .config is

CONFIG_DEBUG_RT_MUTEXES=y

Try to switch this off if switching off the lockdep flags does not hel.

CONFIG_DEBUG_PI_LIST=y

Will go off with the above

# CONFIG_RT_MUTEX_TESTER is not set

Leave off

CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y

Try toggling if the disabling lockdep flags does not work.


CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_PROVE_LOCKING=y
CONFIG_LOCKDEP=y

I suspect the above are the main candidates for trouble. Could you
switch those off?


# CONFIG_DEBUG_LOCKDEP is not set
CONFIG_TRACE_IRQFLAGS=y
CONFIG_DEBUG_SPINLOCK_SLEEP=y
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
CONFIG_STACKTRACE=y

# CONFIG_DEBUG_KOBJECT is not set

Debug kobject may be helpful.

CONFIG_DEBUG_BUGVERBOSE=y



2007-05-21 17:41:43

by Hugh Dickins

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, 21 May 2007, Christoph Lameter wrote:
> On Sun, 20 May 2007, Srihari Vijayaraghavan wrote:
>
> > With no CONFIG_SLUB_DEBUG, things have slightly improved. No more panic. Good.
> > Serial console is working. Good. But there is another problem:
>
> > Freeing unused kernel memory: 308k freed
> > BUG: spinlock bad magic on CPU#1, init/1
> > lock: ffff81011ec0a100, .magic: ffff8101, .owner: <none>/-1, .owner_cpu: -1
> >
> > Call Trace:
> > [<ffffffff802fc516>] _raw_spin_lock+0x22/0xf6
> > [<ffffffff8026f684>] vma_adjust+0x219/0x454
> > [<ffffffff8026f684>] vma_adjust+0x219/0x454
> > [<ffffffff8026fe0d>] vma_merge+0x147/0x1f4
> > [<ffffffff80270b55>] do_mmap_pgoff+0x414/0x7c7
> > [<ffffffff8040aa34>] _spin_unlock_irq+0x24/0x27
> > [<ffffffff8020f528>] sys_mmap+0xe5/0x110
> > [<ffffffff80209dde>] system_call+0x7e/0x83
>
> Hmmmm..... We have seen this before
>
> http://marc.info/?l=linux-kernel&m=117891943401284&w=2

Yes, sounded the same to me too: I couldn't reproduce it or see anything
wrong in the code back then. But Srihari's info about CONFIG_DEBUG_SLUB
off has helped a lot: I was then able to reproduce it on my x86_64, and
after a lot of staring at the code, the problem became obvious...

[PATCH] slub: fix size adjustment when SLUB_DEBUG off

SLUB was corrupting the anon_vma's spinlock when not configured for
SLUB_DEBUG: the size adjustment necessary for SLAB_DESTROY_BY_RCU or
constructor was inside the CONFIG_SLUB_DEBUG appropriate for poisoning.

Signed-off-by: Hugh Dickins <[email protected]>
---
mm/slub.c | 18 ++++++++++++++----
1 file changed, 14 insertions(+), 4 deletions(-)

--- 2.6.22-rc2/mm/slub.c 2007-05-21 13:13:20.000000000 +0100
+++ linux/mm/slub.c 2007-05-21 18:05:57.000000000 +0100
@@ -1917,16 +1917,26 @@ static int calculate_sizes(struct kmem_c
*/
s->inuse = size;

+ if ((flags & SLAB_DESTROY_BY_RCU) || s->ctor) {
+ /*
+ * Relocate free pointer after the object if it is not
+ * permitted to overwrite the first word of the object on
+ * kmem_cache_free.
+ *
+ * This is the case if we do RCU or have a constructor.
+ */
+ s->offset = size;
+ size += sizeof(void *);
+ }
+
#ifdef CONFIG_SLUB_DEBUG
- if (((flags & (SLAB_DESTROY_BY_RCU | SLAB_POISON)) ||
- s->ctor)) {
+ if (s->flags & __OBJECT_POISON) {
/*
* Relocate free pointer after the object if it is not
* permitted to overwrite the first word of the object on
* kmem_cache_free.
*
- * This is the case if we do RCU, have a constructor or
- * destructor or are poisoning the objects.
+ * This is the case if we are poisoning the objects.
*/
s->offset = size;
size += sizeof(void *);

2007-05-21 18:25:49

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, 21 May 2007, Hugh Dickins wrote:

> Yes, sounded the same to me too: I couldn't reproduce it or see anything
> wrong in the code back then. But Srihari's info about CONFIG_DEBUG_SLUB
> off has helped a lot: I was then able to reproduce it on my x86_64, and
> after a lot of staring at the code, the problem became obvious...

Right. The #ifdef CONFIG_SLUB_DEBUG is at the wrong location. The best fix
is to moving the #ifdef otherwise the size is still wrong for the
ctor case.


SLUB Debug: Fix object size calculation

The object size calculation is wrong if !CONFIG_SLUB_DEBUG because
the #ifdef CONFIG_SLUB_DEBUG is now switching off the size adjustments
for DESTROY_BY_RCU and ctor.

Signed-off-by: Christoph Lameter <[email protected]>

---
mm/slub.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Index: slub/mm/slub.c
===================================================================
--- slub.orig/mm/slub.c 2007-05-21 11:21:36.000000000 -0700
+++ slub/mm/slub.c 2007-05-21 11:21:49.000000000 -0700
@@ -1943,7 +1943,6 @@ static int calculate_sizes(struct kmem_c
*/
s->inuse = size;

-#ifdef CONFIG_SLUB_DEBUG
if (((flags & (SLAB_DESTROY_BY_RCU | SLAB_POISON)) ||
s->ctor)) {
/*
@@ -1958,6 +1957,7 @@ static int calculate_sizes(struct kmem_c
size += sizeof(void *);
}

+#ifdef CONFIG_SLUB_DEBUG
if (flags & SLAB_STORE_USER)
/*
* Need to store information about allocs and frees after

2007-05-21 18:45:28

by Hugh Dickins

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, 21 May 2007, Christoph Lameter wrote:
> On Mon, 21 May 2007, Hugh Dickins wrote:
>
> > Yes, sounded the same to me too: I couldn't reproduce it or see anything
> > wrong in the code back then. But Srihari's info about CONFIG_DEBUG_SLUB
> > off has helped a lot: I was then able to reproduce it on my x86_64, and
> > after a lot of staring at the code, the problem became obvious...
>
> Right. The #ifdef CONFIG_SLUB_DEBUG is at the wrong location. The best fix
> is to moving the #ifdef otherwise the size is still wrong for the
> ctor case.

? My patch did handle the ctor case.

>
> SLUB Debug: Fix object size calculation
>
> The object size calculation is wrong if !CONFIG_SLUB_DEBUG because
> the #ifdef CONFIG_SLUB_DEBUG is now switching off the size adjustments
> for DESTROY_BY_RCU and ctor.
>
> Signed-off-by: Christoph Lameter <[email protected]>

Yes, I think that should do it too. The reason behind my repeating
the block was to handle the case where SLAB_POISON is passed to
kmem_cache_create, but CONFIG_SLUB_DEBUG is off. But apparently
that case would hit the BUG_ON(flags & ~CREATE_MASK), therefore
your patch is simpler and better. Quite a maze.

Hugh

>
> ---
> mm/slub.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> Index: slub/mm/slub.c
> ===================================================================
> --- slub.orig/mm/slub.c 2007-05-21 11:21:36.000000000 -0700
> +++ slub/mm/slub.c 2007-05-21 11:21:49.000000000 -0700
> @@ -1943,7 +1943,6 @@ static int calculate_sizes(struct kmem_c
> */
> s->inuse = size;
>
> -#ifdef CONFIG_SLUB_DEBUG
> if (((flags & (SLAB_DESTROY_BY_RCU | SLAB_POISON)) ||
> s->ctor)) {
> /*
> @@ -1958,6 +1957,7 @@ static int calculate_sizes(struct kmem_c
> size += sizeof(void *);
> }
>
> +#ifdef CONFIG_SLUB_DEBUG
> if (flags & SLAB_STORE_USER)
> /*
> * Need to store information about allocs and frees after

2007-05-21 18:48:52

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, 21 May 2007, Hugh Dickins wrote:

> On Mon, 21 May 2007, Christoph Lameter wrote:
> > On Mon, 21 May 2007, Hugh Dickins wrote:
> >
> > > Yes, sounded the same to me too: I couldn't reproduce it or see anything
> > > wrong in the code back then. But Srihari's info about CONFIG_DEBUG_SLUB
> > > off has helped a lot: I was then able to reproduce it on my x86_64, and
> > > after a lot of staring at the code, the problem became obvious...
> >
> > Right. The #ifdef CONFIG_SLUB_DEBUG is at the wrong location. The best fix
> > is to moving the #ifdef otherwise the size is still wrong for the
> > ctor case.
>
> ? My patch did handle the ctor case.

True. I was thinking about just checking the problem case that we had
here.

> > SLUB Debug: Fix object size calculation
> >
> > The object size calculation is wrong if !CONFIG_SLUB_DEBUG because
> > the #ifdef CONFIG_SLUB_DEBUG is now switching off the size adjustments
> > for DESTROY_BY_RCU and ctor.
> >
> > Signed-off-by: Christoph Lameter <[email protected]>
>
> Yes, I think that should do it too. The reason behind my repeating
> the block was to handle the case where SLAB_POISON is passed to
> kmem_cache_create, but CONFIG_SLUB_DEBUG is off. But apparently
> that case would hit the BUG_ON(flags & ~CREATE_MASK), therefore
> your patch is simpler and better. Quite a maze.

Would you ack my patch? I do not want to repeat the block.

2007-05-21 19:03:18

by Jens Axboe

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, May 21 2007, Christoph Lameter wrote:
> On Mon, 21 May 2007, Hugh Dickins wrote:
>
> > On Mon, 21 May 2007, Christoph Lameter wrote:
> > > On Mon, 21 May 2007, Hugh Dickins wrote:
> > >
> > > > Yes, sounded the same to me too: I couldn't reproduce it or see anything
> > > > wrong in the code back then. But Srihari's info about CONFIG_DEBUG_SLUB
> > > > off has helped a lot: I was then able to reproduce it on my x86_64, and
> > > > after a lot of staring at the code, the problem became obvious...
> > >
> > > Right. The #ifdef CONFIG_SLUB_DEBUG is at the wrong location. The best fix
> > > is to moving the #ifdef otherwise the size is still wrong for the
> > > ctor case.
> >
> > ? My patch did handle the ctor case.
>
> True. I was thinking about just checking the problem case that we had
> here.
>
> > > SLUB Debug: Fix object size calculation
> > >
> > > The object size calculation is wrong if !CONFIG_SLUB_DEBUG because
> > > the #ifdef CONFIG_SLUB_DEBUG is now switching off the size adjustments
> > > for DESTROY_BY_RCU and ctor.
> > >
> > > Signed-off-by: Christoph Lameter <[email protected]>
> >
> > Yes, I think that should do it too. The reason behind my repeating
> > the block was to handle the case where SLAB_POISON is passed to
> > kmem_cache_create, but CONFIG_SLUB_DEBUG is off. But apparently
> > that case would hit the BUG_ON(flags & ~CREATE_MASK), therefore
> > your patch is simpler and better. Quite a maze.
>
> Would you ack my patch? I do not want to repeat the block.

I can test whatever you want tomorrow morning, it was 100% repeatable
here. So which one, your patch or Hughs?

--
Jens Axboe

2007-05-21 19:04:45

by Hugh Dickins

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, 21 May 2007, Christoph Lameter wrote:
> On Mon, 21 May 2007, Hugh Dickins wrote:
> > On Mon, 21 May 2007, Christoph Lameter wrote:
> > > On Mon, 21 May 2007, Hugh Dickins wrote:
>
> > > SLUB Debug: Fix object size calculation
> > >
> > > The object size calculation is wrong if !CONFIG_SLUB_DEBUG because
> > > the #ifdef CONFIG_SLUB_DEBUG is now switching off the size adjustments
> > > for DESTROY_BY_RCU and ctor.
> > >
> > > Signed-off-by: Christoph Lameter <[email protected]>

Acked-by: Hugh Dickins <[email protected]>

2007-05-21 19:07:03

by Hugh Dickins

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, 21 May 2007, Jens Axboe wrote:
>
> I can test whatever you want tomorrow morning, it was 100% repeatable
> here. So which one, your patch or Hughs?

Great, thanks: Christoph's please - I'm sure he'll agree!

Hugh

2007-05-21 19:12:17

by Jens Axboe

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, May 21 2007, Hugh Dickins wrote:
> On Mon, 21 May 2007, Jens Axboe wrote:
> >
> > I can test whatever you want tomorrow morning, it was 100% repeatable
> > here. So which one, your patch or Hughs?
>
> Great, thanks: Christoph's please - I'm sure he'll agree!

Alright, will do!

--
Jens Axboe

2007-05-21 19:13:18

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, 21 May 2007, Jens Axboe wrote:

> I can test whatever you want tomorrow morning, it was 100% repeatable
> here. So which one, your patch or Hughs?

This was an effect of me suggesting to switch off CONFIG_SLUB_DEBUG. If
you did not run without CONFIG_SLUB_DEBUG then you were not affected by
the problem. And the earlier case that you reported was a version of SLUB
that did not have the capability to switch off SLUB_DEBUG.

Is there any way we can boot with "slub_debug" on the kernel command line
and get early printk output? Looks like an early failure that may have to
do with other kernel debug options being on (lockdep or so?)

2007-05-21 19:50:55

by Jens Axboe

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, May 21 2007, Christoph Lameter wrote:
> On Mon, 21 May 2007, Jens Axboe wrote:
>
> > I can test whatever you want tomorrow morning, it was 100% repeatable
> > here. So which one, your patch or Hughs?
>
> This was an effect of me suggesting to switch off CONFIG_SLUB_DEBUG. If
> you did not run without CONFIG_SLUB_DEBUG then you were not affected by
> the problem. And the earlier case that you reported was a version of SLUB
> that did not have the capability to switch off SLUB_DEBUG.
>
> Is there any way we can boot with "slub_debug" on the kernel command line
> and get early printk output? Looks like an early failure that may have to
> do with other kernel debug options being on (lockdep or so?)

Sure, I will try that as well. For reference, the last kernel tested on
that box was HEAD 129a84de2347002f09721cda3155ccfd19fade40.

--
Jens Axboe

2007-05-21 19:54:57

by Hugh Dickins

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, 21 May 2007, Christoph Lameter wrote:
> On Mon, 21 May 2007, Jens Axboe wrote:
>
> > I can test whatever you want tomorrow morning, it was 100% repeatable
> > here. So which one, your patch or Hughs?
>
> This was an effect of me suggesting to switch off CONFIG_SLUB_DEBUG. If
> you did not run without CONFIG_SLUB_DEBUG then you were not affected by
> the problem. And the earlier case that you reported was a version of SLUB
> that did not have the capability to switch off SLUB_DEBUG.

Are you sure? If CONFIG_EMBEDDED=y then -rc1 and -rc2 give you the
choice to turn it off.

> Is there any way we can boot with "slub_debug" on the kernel command line
> and get early printk output? Looks like an early failure that may have to
> do with other kernel debug options being on (lockdep or so?)

2007-05-21 19:57:38

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, 21 May 2007, Hugh Dickins wrote:

> > the problem. And the earlier case that you reported was a version of SLUB
> > that did not have the capability to switch off SLUB_DEBUG.
>
> Are you sure? If CONFIG_EMBEDDED=y then -rc1 and -rc2 give you the
> choice to turn it off.

Right. That went in indeed shortly before the rc1 release.

So if both had disabled SLUB_DEBUG then this could be the reason for the
failure. However, CONFIG_SLUB_DEBUG was set in the .config that was
included when this bug was reported. So I think we still have not found
the real problem yet.

2007-05-21 22:27:22

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

Ok booting SLUB without "slub_debug" and having

CONFIG_DEBUG_LOCK_ALLOC
CONFIG_LOCKDEP

will trigger the problem.

So I guess the issue is that lockdep does a slab alloc while we get the
slab lock during slab alloc?

2007-05-21 23:46:18

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, 21 May 2007, Christoph Lameter wrote:

> So I guess the issue is that lockdep does a slab alloc while we get the
> slab lock during slab alloc?

Lockdep is not available on IA64 where I would be able to figure it out
using a simulator. x86_64 early printk support seems to be broken?
No output with earlyprintk=serial.

2007-05-22 07:10:19

by Jens Axboe

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Mon, May 21 2007, Christoph Lameter wrote:
> Ok booting SLUB without "slub_debug" and having
>
> CONFIG_DEBUG_LOCK_ALLOC
> CONFIG_LOCKDEP
>
> will trigger the problem.
>
> So I guess the issue is that lockdep does a slab alloc while we get the
> slab lock during slab alloc?

If I have CONFIG_SLUB_DEBUG set, then box boots fine. If I don't have it
set, then it hard hangs sometime after boot:

[...]
ReiserFS: sda2: Using r5 hash to sort names
VFS: Mounted root (reiserfs filesystem) readonly.
Freeing unused kernel memory: 180k freed

which is usually followed by messages bringing up the network link, etc.
Let me know which tests you want run, or which patches to try.

--
Jens Axboe

2007-05-22 08:26:51

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

Just another data point: with the flick of CONFIG_SMP, I'm in control of the
hangs/crashes ;-).

Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's quite stable
actually (having completed a dozen kernel compile sessions so far).

I suspected this after seeing spinlock issues on both cores of my CPU in my
earlier report. Just a stab in the dark with some unexpected luck :-).

(Tested on http://www.kernel.org/hg/linux-2.6 changeset at 57106:3f42ad5de7cd.
That's the latest hg commit there.)

Thanks

PS: For the record everything except sysrq support is disabled in the kernel
hacking section. Also the CONFIG_SLAB_DEBUG is off. Kernel preemption is also
off, after proving it had no influence. Just to keep the test case simple.



___________________________________________________________________________________
How would you spend $50,000 to create a more sustainable environment in Australia? Go to Yahoo!7 Answers and share your idea.
http://advision.webevents.yahoo.com/aunz/lifestyle/answers/y7ans-babp_reg.html

2007-05-22 09:26:01

by Ingo Molnar

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub


* Srihari Vijayaraghavan <[email protected]> wrote:

> Just another data point: with the flick of CONFIG_SMP, I'm in control
> of the hangs/crashes ;-).
>
> Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's
> quite stable actually (having completed a dozen kernel compile
> sessions so far).
>
> I suspected this after seeing spinlock issues on both cores of my CPU
> in my earlier report. Just a stab in the dark with some unexpected
> luck :-).

could you enable CONFIG_PROVE_LOCKING - does it spit out any warning
into the syslog?

Ingo

2007-05-22 11:01:40

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

--- Ingo Molnar <[email protected]> wrote:
> * Srihari Vijayaraghavan <[email protected]> wrote:
> > Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's
> > quite stable actually (having completed a dozen kernel compile
> > sessions so far).

[...]

> could you enable CONFIG_PROVE_LOCKING - does it spit out any warning
> into the syslog?

Compiled slub with SMP & CONFIG_PROVE_LOCKING. No luck. It still hangs solid
after the second spinlock lockup call trace.

Here's the relevant sections of the kernel logs:

...
Freeing unused kernel memory: 228k freed
BUG: spinlock bad magic on CPU#1, init/1
lock: ffff81011f5f1100, .magic: ffff8101, .owner: <none>/-1, .owner_cpu: -1

Call Trace:
[<ffffffff802f326a>] _raw_spin_lock+0x22/0xf6
[<ffffffff8026b2d5>] vma_adjust+0x21c/0x446
[<ffffffff8026b2d5>] vma_adjust+0x21c/0x446
[<ffffffff8026b9d4>] vma_merge+0x10c/0x195
[<ffffffff8026c757>] do_mmap_pgoff+0x3f5/0x794
[<ffffffff803fff0c>] _spin_unlock_irq+0x24/0x27
[<ffffffff8020f414>] sys_mmap+0xe5/0x110
[<ffffffff80209dde>] system_call+0x7e/0x83
...
PM: Adding info for No Bus:vcsa1
BUG: spinlock lockup on CPU#1, hostname/369, ffff81011f5f1fc0

Call Trace:
[<ffffffff802f3317>] _raw_spin_lock+0xcf/0xf6
[<ffffffff8026ec9c>] anon_vma_unlink+0x1c/0x68
[<ffffffff8026ec9c>] anon_vma_unlink+0x1c/0x68
[<ffffffff80269aa0>] free_pgtables+0x69/0xc4
[<ffffffff8026ad0e>] exit_mmap+0x91/0xeb
[<ffffffff80228cea>] mmput+0x2c/0x9f
[<ffffffff8022df72>] do_exit+0x22e/0x82e
[<ffffffff8022e5f4>] sys_exit_group+0x0/0xe
[<ffffffff80209dde>] system_call+0x7e/0x83


Surprisingly, with CONFIG_SMP=n, CONFIG_PROVE_LOCKING produces this with slub
(then hangs solid):

Freeing unused kernel memory: 188k freed
BUG: spinlock lockup on CPU#0, init/1, ffff81011e9d3160

Call Trace:
[<ffffffff802eca20>] _raw_spin_lock+0xca/0xe8
[<ffffffff80265d6d>] vma_adjust+0x218/0x442
[<ffffffff80265d6d>] vma_adjust+0x218/0x442
[<ffffffff8026646b>] vma_merge+0x10c/0x195
[<ffffffff802671d5>] do_mmap_pgoff+0x3f5/0x790
[<ffffffff803f6e84>] _spin_unlock_irq+0x24/0x27
[<ffffffff8020ead0>] sys_mmap+0xe5/0x110
[<ffffffff80209cce>] system_call+0x7e/0x83

To recap:
1. No problems with slub on CONFIG_SMP=n & CONFIG_PROVE_LOCKING=n
2. Problem with slub on CONFIG_SMP=n & CONFIG_PROVE_LOCKING=y (perhaps a. some
locking issue when slub is activated or b. something is wrong with 'prove
locking' mechanism when slub is activated or c. something else I don't see)
3. Problem with slub on CONFIG_SMP=y (even without CONFIG_PROVE_LOCKING=y)

Thanks



___________________________________________________________________________________
How would you spend $50,000 to create a more sustainable environment in Australia? Go to Yahoo!7 Answers and share your idea.
http://advision.webevents.yahoo.com/aunz/lifestyle/answers/y7ans-babp_reg.html

2007-05-22 11:21:17

by Ingo Molnar

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub


* Srihari Vijayaraghavan <[email protected]> wrote:

> Compiled slub with SMP & CONFIG_PROVE_LOCKING. No luck. It still hangs
> solid after the second spinlock lockup call trace.

hm. This suggests that the spinlock got corrupted - otherwise lockdep
would have complained about the lockup before the spinlock-debug code
had its chance.

> Surprisingly, with CONFIG_SMP=n, CONFIG_PROVE_LOCKING produces this
> with slub (then hangs solid):

yes - PROVE_LOCKING reactivates spinlocks even on UP. At least this
suggests that you'd have gotten the hang even with maxcpus=1 - i.e. the
spinlock corruption is not caused by some genuine SMP race.

furthermore, PROVE_LOCKING also turns on DEBUG_LOCK_ALLOC, so we now
know that it's most likely not use-after-free type of corruption.

Ingo

2007-05-22 11:26:49

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

--- Ingo Molnar <[email protected]> wrote:
[...]
> yes - PROVE_LOCKING reactivates spinlocks even on UP. At least this
> suggests that you'd have gotten the hang even with maxcpus=1 - i.e. the
> spinlock corruption is not caused by some genuine SMP race.

You're right on the mark there: even with maxcpus=1 or nosmp (or with both),
with PROVE_LOCKING it hung with slub. I didn't mention that so not to confuse
the matter, but you rightly figured that out too.

Thanks



___________________________________________________________________________________
How would you spend $50,000 to create a more sustainable environment in Australia? Go to Yahoo!7 Answers and share your idea.
http://advision.webevents.yahoo.com/aunz/lifestyle/answers/y7ans-babp_reg.html

2007-05-22 13:01:34

by Hugh Dickins

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Tue, 22 May 2007, Srihari Vijayaraghavan wrote:
> --- Ingo Molnar <[email protected]> wrote:
> > * Srihari Vijayaraghavan <[email protected]> wrote:
> > > Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's
> > > quite stable actually (having completed a dozen kernel compile
> > > sessions so far).
>
> [...]
>
> > could you enable CONFIG_PROVE_LOCKING - does it spit out any warning
> > into the syslog?
>
> Compiled slub with SMP & CONFIG_PROVE_LOCKING. No luck. It still hangs solid
> after the second spinlock lockup call trace.
>
> Here's the relevant sections of the kernel logs:
>
> ...
> Freeing unused kernel memory: 228k freed
> BUG: spinlock bad magic on CPU#1, init/1
> lock: ffff81011f5f1100, .magic: ffff8101, .owner: <none>/-1, .owner_cpu: -1
>
> Call Trace:
> [<ffffffff802f326a>] _raw_spin_lock+0x22/0xf6
> [<ffffffff8026b2d5>] vma_adjust+0x21c/0x446
> [<ffffffff8026b2d5>] vma_adjust+0x21c/0x446
> [<ffffffff8026b9d4>] vma_merge+0x10c/0x195
> [<ffffffff8026c757>] do_mmap_pgoff+0x3f5/0x794
> [<ffffffff803fff0c>] _spin_unlock_irq+0x24/0x27
> [<ffffffff8020f414>] sys_mmap+0xe5/0x110
> [<ffffffff80209dde>] system_call+0x7e/0x83
> ...
> PM: Adding info for No Bus:vcsa1
> BUG: spinlock lockup on CPU#1, hostname/369, ffff81011f5f1fc0
>
> Call Trace:
> [<ffffffff802f3317>] _raw_spin_lock+0xcf/0xf6
> [<ffffffff8026ec9c>] anon_vma_unlink+0x1c/0x68
> [<ffffffff8026ec9c>] anon_vma_unlink+0x1c/0x68
> [<ffffffff80269aa0>] free_pgtables+0x69/0xc4
> [<ffffffff8026ad0e>] exit_mmap+0x91/0xeb
> [<ffffffff80228cea>] mmput+0x2c/0x9f
> [<ffffffff8022df72>] do_exit+0x22e/0x82e
> [<ffffffff8022e5f4>] sys_exit_group+0x0/0xe
> [<ffffffff80209dde>] system_call+0x7e/0x83
>
>
> Surprisingly, with CONFIG_SMP=n, CONFIG_PROVE_LOCKING produces this with slub
> (then hangs solid):
>
> Freeing unused kernel memory: 188k freed
> BUG: spinlock lockup on CPU#0, init/1, ffff81011e9d3160
>
> Call Trace:
> [<ffffffff802eca20>] _raw_spin_lock+0xca/0xe8
> [<ffffffff80265d6d>] vma_adjust+0x218/0x442
> [<ffffffff80265d6d>] vma_adjust+0x218/0x442
> [<ffffffff8026646b>] vma_merge+0x10c/0x195
> [<ffffffff802671d5>] do_mmap_pgoff+0x3f5/0x790
> [<ffffffff803f6e84>] _spin_unlock_irq+0x24/0x27
> [<ffffffff8020ead0>] sys_mmap+0xe5/0x110
> [<ffffffff80209cce>] system_call+0x7e/0x83
>
> To recap:
> 1. No problems with slub on CONFIG_SMP=n & CONFIG_PROVE_LOCKING=n
> 2. Problem with slub on CONFIG_SMP=n & CONFIG_PROVE_LOCKING=y (perhaps a. some
> locking issue when slub is activated or b. something is wrong with 'prove
> locking' mechanism when slub is activated or c. something else I don't see)
> 3. Problem with slub on CONFIG_SMP=y (even without CONFIG_PROVE_LOCKING=y)

You've made no mention of trying the patch I sent yesterday, or better,
the patch Christoph replied with to replace it. Please clarify whether
you're getting the above after applying one of those patches - thanks.

Hugh

2007-05-22 15:39:06

by Jens Axboe

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Tue, May 22 2007, Hugh Dickins wrote:
> On Tue, 22 May 2007, Srihari Vijayaraghavan wrote:
> > --- Ingo Molnar <[email protected]> wrote:
> > > * Srihari Vijayaraghavan <[email protected]> wrote:
> > > > Yup, with CONFIG_SMP=n, I'm unable to reproduce the problem. It's
> > > > quite stable actually (having completed a dozen kernel compile
> > > > sessions so far).
> >
> > [...]
> >
> > > could you enable CONFIG_PROVE_LOCKING - does it spit out any warning
> > > into the syslog?
> >
> > Compiled slub with SMP & CONFIG_PROVE_LOCKING. No luck. It still hangs solid
> > after the second spinlock lockup call trace.
> >
> > Here's the relevant sections of the kernel logs:
> >
> > ...
> > Freeing unused kernel memory: 228k freed
> > BUG: spinlock bad magic on CPU#1, init/1
> > lock: ffff81011f5f1100, .magic: ffff8101, .owner: <none>/-1, .owner_cpu: -1
> >
> > Call Trace:
> > [<ffffffff802f326a>] _raw_spin_lock+0x22/0xf6
> > [<ffffffff8026b2d5>] vma_adjust+0x21c/0x446
> > [<ffffffff8026b2d5>] vma_adjust+0x21c/0x446
> > [<ffffffff8026b9d4>] vma_merge+0x10c/0x195
> > [<ffffffff8026c757>] do_mmap_pgoff+0x3f5/0x794
> > [<ffffffff803fff0c>] _spin_unlock_irq+0x24/0x27
> > [<ffffffff8020f414>] sys_mmap+0xe5/0x110
> > [<ffffffff80209dde>] system_call+0x7e/0x83
> > ...
> > PM: Adding info for No Bus:vcsa1
> > BUG: spinlock lockup on CPU#1, hostname/369, ffff81011f5f1fc0
> >
> > Call Trace:
> > [<ffffffff802f3317>] _raw_spin_lock+0xcf/0xf6
> > [<ffffffff8026ec9c>] anon_vma_unlink+0x1c/0x68
> > [<ffffffff8026ec9c>] anon_vma_unlink+0x1c/0x68
> > [<ffffffff80269aa0>] free_pgtables+0x69/0xc4
> > [<ffffffff8026ad0e>] exit_mmap+0x91/0xeb
> > [<ffffffff80228cea>] mmput+0x2c/0x9f
> > [<ffffffff8022df72>] do_exit+0x22e/0x82e
> > [<ffffffff8022e5f4>] sys_exit_group+0x0/0xe
> > [<ffffffff80209dde>] system_call+0x7e/0x83
> >
> >
> > Surprisingly, with CONFIG_SMP=n, CONFIG_PROVE_LOCKING produces this with slub
> > (then hangs solid):
> >
> > Freeing unused kernel memory: 188k freed
> > BUG: spinlock lockup on CPU#0, init/1, ffff81011e9d3160
> >
> > Call Trace:
> > [<ffffffff802eca20>] _raw_spin_lock+0xca/0xe8
> > [<ffffffff80265d6d>] vma_adjust+0x218/0x442
> > [<ffffffff80265d6d>] vma_adjust+0x218/0x442
> > [<ffffffff8026646b>] vma_merge+0x10c/0x195
> > [<ffffffff802671d5>] do_mmap_pgoff+0x3f5/0x790
> > [<ffffffff803f6e84>] _spin_unlock_irq+0x24/0x27
> > [<ffffffff8020ead0>] sys_mmap+0xe5/0x110
> > [<ffffffff80209cce>] system_call+0x7e/0x83
> >
> > To recap:
> > 1. No problems with slub on CONFIG_SMP=n & CONFIG_PROVE_LOCKING=n
> > 2. Problem with slub on CONFIG_SMP=n & CONFIG_PROVE_LOCKING=y (perhaps a. some
> > locking issue when slub is activated or b. something is wrong with 'prove
> > locking' mechanism when slub is activated or c. something else I don't see)
> > 3. Problem with slub on CONFIG_SMP=y (even without CONFIG_PROVE_LOCKING=y)
>
> You've made no mention of trying the patch I sent yesterday, or better,
> the patch Christoph replied with to replace it. Please clarify whether
> you're getting the above after applying one of those patches - thanks.

Christophs patch works for me!

--
Jens Axboe

2007-05-22 18:30:27

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Tue, 22 May 2007, Jens Axboe wrote:

> > You've made no mention of trying the patch I sent yesterday, or better,
> > the patch Christoph replied with to replace it. Please clarify whether
> > you're getting the above after applying one of those patches - thanks.
>
> Christophs patch works for me!

So you are able to run slub with the patch and

1. CONFIG_NUMA set (CONFIG_SMP works fine here too)

2. CONFIG_LOCKDEP

3. CONFIG_DEBUG_LOCK_ALLOCS

?

That fails here.

2007-05-22 20:12:54

by Jens Axboe

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Tue, May 22 2007, Christoph Lameter wrote:
> On Tue, 22 May 2007, Jens Axboe wrote:
>
> > > You've made no mention of trying the patch I sent yesterday, or better,
> > > the patch Christoph replied with to replace it. Please clarify whether
> > > you're getting the above after applying one of those patches - thanks.
> >
> > Christophs patch works for me!
>
> So you are able to run slub with the patch and
>
> 1. CONFIG_NUMA set (CONFIG_SMP works fine here too)
>
> 2. CONFIG_LOCKDEP
>
> 3. CONFIG_DEBUG_LOCK_ALLOCS
>
> ?
>
> That fails here.

I didn't say that. Previously a kernel without SLUB_DEBUG set would not
work, with the patch it does. That is all I tested so far.

--
Jens Axboe

2007-05-22 22:15:50

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

--- Hugh Dickins <[email protected]> wrote:
> On Tue, 22 May 2007, Srihari Vijayaraghavan wrote:

[...]

> > Compiled slub with SMP & CONFIG_PROVE_LOCKING. No luck. It still hangs
> solid

[...]

> You've made no mention of trying the patch I sent yesterday, or better,
> the patch Christoph replied with to replace it. Please clarify whether
> you're getting the above after applying one of those patches - thanks.

Sorry about that :-(. Was a busy day & skipped a few emails :-(.

Indeed with Christoph's patch (over the hg commit 57109:745e2df8ec60), without
CONFIG_SLUB_DEBUG, but with CONFIG_PROVE_LOCKING only under the kernel
debugging section, on CONFIG_SMP the slub kerenel is quite stable. (It's
running some stress tests now, but things are looking good.)

(If you want me to test it with other slub or kernel debug options please let
me know. It just takes a lot of time to eliminate the variables, if there are
problems.)

Thanks folks.



___________________________________________________________________________________
How would you spend $50,000 to create a more sustainable environment in Australia? Go to Yahoo!7 Answers and share your idea.
http://advision.webevents.yahoo.com/aunz/lifestyle/answers/y7ans-babp_reg.html

2007-05-22 22:26:13

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Wed, 23 May 2007, Srihari Vijayaraghavan wrote:

> (If you want me to test it with other slub or kernel debug options please let
> me know. It just takes a lot of time to eliminate the variables, if there are
> problems.)

Yup. compile with

CONFIG_NUMA

CONFIG_LOCKDEP

CONFIG_DEBUG_LOCK_ALLOCS

and then try to boot without slub_debug.

If that fails then boot with slub_nomerge

2007-05-23 07:13:50

by Jens Axboe

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Tue, May 22 2007, Christoph Lameter wrote:
> On Wed, 23 May 2007, Srihari Vijayaraghavan wrote:
>
> > (If you want me to test it with other slub or kernel debug options please let
> > me know. It just takes a lot of time to eliminate the variables, if there are
> > problems.)
>
> Yup. compile with
>
> CONFIG_NUMA
>
> CONFIG_LOCKDEP
>
> CONFIG_DEBUG_LOCK_ALLOCS
>
> and then try to boot without slub_debug.

That works for me with the patch, .config attached.

--
Jens Axboe


Attachments:
(No filename) (487.00 B)
.config (27.35 kB)
Download all attachments

2007-05-23 10:18:32

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

--- Christoph Lameter <[email protected]> wrote:
> On Wed, 23 May 2007, Srihari Vijayaraghavan wrote:
[...]
> Yup. compile with
>
> CONFIG_NUMA
>
> CONFIG_LOCKDEP
>
> CONFIG_DEBUG_LOCK_ALLOCS

(All the tests in this email was conducted on top of your patch)

Yup done that. The resulting kernel (without CONFIG_SLUB_DEBUG=y) is quite
stable; passed a few iterations of kernel compile tests successfully.

> and then try to boot without slub_debug.

I guess you mean with CONFIG_SLUB_CONFIG=y? If so, I built another kernel with
CONFIG_SLUB_DEBUG=y (plus all of the above) & tested it. It panics by default,
but with slub_nomerge it works just fine (tested under moderate load).

(the panic message produced by CONFIG_SLUB_DEBUG=y was the exact same call
trace as my very first email in this email thread with slightly different
address on a couple of functions, but rest remains the same)

I'm personally very happy that slub works stably without slub debug options,
because that's what I'd run in a production env. Thanks to your patch, slub is
quite stable without the slub debug for me :-)). But it'd to nice to have a
working slub debug for test env., as you'd undoubtedly be aware of, of course
:-). Just my humble opinion.

> If that fails then boot with slub_nomerge

Yup, I had to use slub_nomerge; without that CONFIG_SLUB_DEBUG=y kernel
panics. (I haven't tested the UP case though. I did try nosmp & maxcpus=1, but
they had no effect on the panic. Do you want me to test UP case for
CONFIG_SLUB_DEBUG=y without slub_nomerge?)

Thanks



___________________________________________________________________________________
How would you spend $50,000 to create a more sustainable environment in Australia? Go to Yahoo!7 Answers and share your idea.
http://advision.webevents.yahoo.com/aunz/lifestyle/answers/y7ans-babp_reg.html

2007-05-23 17:04:36

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Wed, 23 May 2007, Jens Axboe wrote:

> That works for me with the patch, .config attached.

Hmmmm... That means the .config sent initially here was bogus.

2007-05-23 17:10:28

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Wed, 23 May 2007, Srihari Vijayaraghavan wrote:

> > and then try to boot without slub_debug.
>
> I guess you mean with CONFIG_SLUB_CONFIG=y? If so, I built another kernel with
> CONFIG_SLUB_DEBUG=y (plus all of the above) & tested it. It panics by default,
> but with slub_nomerge it works just fine (tested under moderate load).
>
> (the panic message produced by CONFIG_SLUB_DEBUG=y was the exact same call
> trace as my very first email in this email thread with slightly different
> address on a couple of functions, but rest remains the same)

Ahh... At least we are getting to the original problem.

> I'm personally very happy that slub works stably without slub debug options,
> because that's what I'd run in a production env. Thanks to your patch, slub is
> quite stable without the slub debug for me :-)). But it'd to nice to have a
> working slub debug for test env., as you'd undoubtedly be aware of, of course
> :-). Just my humble opinion.
>
> > If that fails then boot with slub_nomerge

So lockdep has issues with slab merging? If locks are tracking within
slabs then I imagine that lockdep gets confused if we put them together.

> Yup, I had to use slub_nomerge; without that CONFIG_SLUB_DEBUG=y kernel
> panics. (I haven't tested the UP case though. I did try nosmp & maxcpus=1, but
> they had no effect on the panic. Do you want me to test UP case for
> CONFIG_SLUB_DEBUG=y without slub_nomerge?)

Yes.

2007-05-23 17:19:31

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Wed, 23 May 2007, Srihari Vijayaraghavan wrote:

> I'm personally very happy that slub works stably without slub debug options,
> because that's what I'd run in a production env. Thanks to your patch, slub is
> quite stable without the slub debug for me :-)). But it'd to nice to have a
> working slub debug for test env., as you'd undoubtedly be aware of, of course
> :-). Just my humble opinion.

Just switching off lockdep should make you happy until we figure out what
is wrong?

2007-05-23 22:24:01

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.22-rc2
# Thu May 24 07:46:01 2007
#
CONFIG_X86_64=y
CONFIG_64BIT=y
CONFIG_X86=y
CONFIG_GENERIC_TIME=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_ZONE_DMA32=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_SEMAPHORE_SLEEPERS=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_RWSEM_GENERIC_SPINLOCK=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_X86_CMPXCHG=y
CONFIG_EARLY_PRINTK=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_DMI=y
CONFIG_AUDIT_ARCH=y
CONFIG_GENERIC_BUG=y
# CONFIG_ARCH_HAS_ILOG2_U32 is not set
# CONFIG_ARCH_HAS_ILOG2_U64 is not set
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"

#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32

#
# General setup
#
CONFIG_LOCALVERSION="-hg-slub"
# CONFIG_LOCALVERSION_AUTO is not set
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_IPC_NS=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
# CONFIG_TASKSTATS is not set
CONFIG_UTS_NS=y
CONFIG_AUDIT=y
CONFIG_AUDITSYSCALL=y
# CONFIG_IKCONFIG is not set
CONFIG_LOG_BUF_SHIFT=17
# CONFIG_CPUSETS is not set
CONFIG_SYSFS_DEPRECATED=y
CONFIG_RELAY=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE=""
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_SYSCTL=y
CONFIG_EMBEDDED=y
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
CONFIG_KALLSYMS_EXTRA_PASS=y
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_ANON_INODES=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_SLUB_DEBUG=y
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLOB is not set
CONFIG_RT_MUTEXES=y
# CONFIG_TINY_SHMEM is not set
CONFIG_BASE_SMALL=0

#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
CONFIG_MODVERSIONS=y
CONFIG_MODULE_SRCVERSION_ALL=y
CONFIG_KMOD=y
CONFIG_STOP_MACHINE=y

#
# Block layer
#
CONFIG_BLOCK=y
CONFIG_BLK_DEV_IO_TRACE=y

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
# CONFIG_IOSCHED_AS is not set
# CONFIG_IOSCHED_DEADLINE is not set
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_AS is not set
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"

#
# Processor type and features
#
CONFIG_X86_PC=y
# CONFIG_X86_VSMP is not set
CONFIG_MK8=y
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_GENERIC_CPU is not set
CONFIG_X86_L1_CACHE_BYTES=64
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_INTERNODE_CACHE_BYTES=64
CONFIG_X86_TSC=y
CONFIG_X86_GOOD_APIC=y
# CONFIG_MICROCODE is not set
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_MTRR=y
CONFIG_SMP=y
# CONFIG_SCHED_SMT is not set
CONFIG_SCHED_MC=y
CONFIG_PREEMPT_NONE=y
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set
CONFIG_PREEMPT_BKL=y
# CONFIG_NUMA is not set
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_SELECT_MEMORY_MODEL=y
# CONFIG_FLATMEM_MANUAL is not set
# CONFIG_DISCONTIGMEM_MANUAL is not set
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_HAVE_MEMORY_PRESENT=y
# CONFIG_SPARSEMEM_STATIC is not set
CONFIG_SPARSEMEM_EXTREME=y
# CONFIG_MEMORY_HOTPLUG is not set
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_RESOURCES_64BIT=y
CONFIG_ZONE_DMA_FLAG=1
CONFIG_NR_CPUS=2
# CONFIG_HOTPLUG_CPU is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_IOMMU=y
# CONFIG_CALGARY_IOMMU is not set
CONFIG_SWIOTLB=y
CONFIG_X86_MCE=y
# CONFIG_X86_MCE_INTEL is not set
CONFIG_X86_MCE_AMD=y
CONFIG_KEXEC=y
# CONFIG_CRASH_DUMP is not set
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_START=0x200000
# CONFIG_SECCOMP is not set
# CONFIG_CC_STACKPROTECTOR is not set
# CONFIG_HZ_100 is not set
CONFIG_HZ_250=y
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=250
CONFIG_K8_NB=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_ISA_DMA_API=y
CONFIG_GENERIC_PENDING_IRQ=y

#
# Power management options
#
CONFIG_PM=y
CONFIG_PM_LEGACY=y
CONFIG_PM_DEBUG=y
# CONFIG_DISABLE_CONSOLE_SUSPEND is not set
# CONFIG_PM_SYSFS_DEPRECATED is not set

#
# ACPI (Advanced Configuration and Power Interface) Support
#
CONFIG_ACPI=y
CONFIG_ACPI_PROCFS=y
CONFIG_ACPI_AC=m
CONFIG_ACPI_BATTERY=m
CONFIG_ACPI_BUTTON=m
CONFIG_ACPI_VIDEO=m
CONFIG_ACPI_FAN=y
CONFIG_ACPI_DOCK=m
CONFIG_ACPI_BAY=m
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_ASUS is not set
CONFIG_ACPI_TOSHIBA=m
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_SYSTEM=y
CONFIG_X86_PM_TIMER=y
CONFIG_ACPI_CONTAINER=y
CONFIG_ACPI_SBS=m

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=y
CONFIG_CPU_FREQ_DEBUG=y
CONFIG_CPU_FREQ_STAT=m
CONFIG_CPU_FREQ_STAT_DETAILS=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_POWERSAVE=m
CONFIG_CPU_FREQ_GOV_USERSPACE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=m
CONFIG_CPU_FREQ_GOV_CONSERVATIVE=m

#
# CPUFreq processor drivers
#
CONFIG_X86_POWERNOW_K8=y
CONFIG_X86_POWERNOW_K8_ACPI=y
# CONFIG_X86_SPEEDSTEP_CENTRINO is not set
CONFIG_X86_ACPI_CPUFREQ=m

#
# shared options
#
# CONFIG_X86_ACPI_CPUFREQ_PROC_INTF is not set
# CONFIG_X86_P4_CLOCKMOD is not set
# CONFIG_X86_SPEEDSTEP_LIB is not set

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCIEPORTBUS=y
CONFIG_HOTPLUG_PCI_PCIE=m
# CONFIG_HOTPLUG_PCI_PCIE_POLL_EVENT_MODE is not set
CONFIG_PCIEAER=y
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
# CONFIG_PCI_DEBUG is not set
CONFIG_HT_IRQ=y

#
# PCCARD (PCMCIA/CardBus) support
#
# CONFIG_PCCARD is not set
CONFIG_HOTPLUG_PCI=y
CONFIG_HOTPLUG_PCI_FAKE=m
CONFIG_HOTPLUG_PCI_ACPI=m
CONFIG_HOTPLUG_PCI_ACPI_IBM=m
# CONFIG_HOTPLUG_PCI_CPCI is not set
CONFIG_HOTPLUG_PCI_SHPC=m

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
CONFIG_BINFMT_MISC=y
CONFIG_IA32_EMULATION=y
# CONFIG_IA32_AOUT is not set
CONFIG_COMPAT=y
CONFIG_SYSVIPC_COMPAT=y

#
# Networking
#
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=y
CONFIG_PACKET_MMAP=y
CONFIG_UNIX=y
CONFIG_XFRM=y
CONFIG_XFRM_USER=y
# CONFIG_XFRM_SUB_POLICY is not set
CONFIG_XFRM_MIGRATE=y
CONFIG_NET_KEY=m
# CONFIG_NET_KEY_MIGRATE is not set
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_ASK_IP_FIB_HASH=y
# CONFIG_IP_FIB_TRIE is not set
CONFIG_IP_FIB_HASH=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y
# CONFIG_IP_ROUTE_MULTIPATH_CACHED is not set
CONFIG_IP_ROUTE_VERBOSE=y
# CONFIG_IP_PNP is not set
CONFIG_NET_IPIP=m
CONFIG_NET_IPGRE=m
CONFIG_NET_IPGRE_BROADCAST=y
CONFIG_IP_MROUTE=y
CONFIG_IP_PIMSM_V1=y
CONFIG_IP_PIMSM_V2=y
# CONFIG_ARPD is not set
CONFIG_SYN_COOKIES=y
CONFIG_INET_AH=m
CONFIG_INET_ESP=m
CONFIG_INET_IPCOMP=m
CONFIG_INET_XFRM_TUNNEL=m
CONFIG_INET_TUNNEL=m
CONFIG_INET_XFRM_MODE_TRANSPORT=m
CONFIG_INET_XFRM_MODE_TUNNEL=m
CONFIG_INET_XFRM_MODE_BEET=m
CONFIG_INET_DIAG=m
CONFIG_INET_TCP_DIAG=m
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set

#
# IP: Virtual Server Configuration
#
# CONFIG_IP_VS is not set
# CONFIG_IPV6 is not set
# CONFIG_INET6_XFRM_TUNNEL is not set
# CONFIG_INET6_TUNNEL is not set
CONFIG_NETLABEL=y
CONFIG_NETWORK_SECMARK=y
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
CONFIG_BRIDGE_NETFILTER=y

#
# Core Netfilter Configuration
#
CONFIG_NETFILTER_NETLINK=m
CONFIG_NETFILTER_NETLINK_QUEUE=m
CONFIG_NETFILTER_NETLINK_LOG=m
CONFIG_NF_CONNTRACK_ENABLED=m
CONFIG_NF_CONNTRACK=m
CONFIG_NF_CT_ACCT=y
CONFIG_NF_CONNTRACK_MARK=y
CONFIG_NF_CONNTRACK_SECMARK=y
CONFIG_NF_CONNTRACK_EVENTS=y
CONFIG_NF_CT_PROTO_GRE=m
CONFIG_NF_CT_PROTO_SCTP=m
CONFIG_NF_CONNTRACK_AMANDA=m
CONFIG_NF_CONNTRACK_FTP=m
CONFIG_NF_CONNTRACK_H323=m
CONFIG_NF_CONNTRACK_IRC=m
CONFIG_NF_CONNTRACK_NETBIOS_NS=m
CONFIG_NF_CONNTRACK_PPTP=m
CONFIG_NF_CONNTRACK_SANE=m
CONFIG_NF_CONNTRACK_SIP=m
CONFIG_NF_CONNTRACK_TFTP=m
CONFIG_NF_CT_NETLINK=m
CONFIG_NETFILTER_XTABLES=m
CONFIG_NETFILTER_XT_TARGET_CLASSIFY=m
CONFIG_NETFILTER_XT_TARGET_CONNMARK=m
CONFIG_NETFILTER_XT_TARGET_DSCP=m
CONFIG_NETFILTER_XT_TARGET_MARK=m
CONFIG_NETFILTER_XT_TARGET_NFQUEUE=m
CONFIG_NETFILTER_XT_TARGET_NFLOG=m
CONFIG_NETFILTER_XT_TARGET_NOTRACK=m
CONFIG_NETFILTER_XT_TARGET_SECMARK=m
CONFIG_NETFILTER_XT_TARGET_CONNSECMARK=m
CONFIG_NETFILTER_XT_TARGET_TCPMSS=m
CONFIG_NETFILTER_XT_MATCH_COMMENT=m
CONFIG_NETFILTER_XT_MATCH_CONNBYTES=m
CONFIG_NETFILTER_XT_MATCH_CONNMARK=m
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=m
CONFIG_NETFILTER_XT_MATCH_DCCP=m
CONFIG_NETFILTER_XT_MATCH_DSCP=m
CONFIG_NETFILTER_XT_MATCH_ESP=m
CONFIG_NETFILTER_XT_MATCH_HELPER=m
CONFIG_NETFILTER_XT_MATCH_LENGTH=m
CONFIG_NETFILTER_XT_MATCH_LIMIT=m
CONFIG_NETFILTER_XT_MATCH_MAC=m
CONFIG_NETFILTER_XT_MATCH_MARK=m
CONFIG_NETFILTER_XT_MATCH_POLICY=m
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=m
# CONFIG_NETFILTER_XT_MATCH_PHYSDEV is not set
CONFIG_NETFILTER_XT_MATCH_PKTTYPE=m
CONFIG_NETFILTER_XT_MATCH_QUOTA=m
CONFIG_NETFILTER_XT_MATCH_REALM=m
CONFIG_NETFILTER_XT_MATCH_SCTP=m
CONFIG_NETFILTER_XT_MATCH_STATE=m
CONFIG_NETFILTER_XT_MATCH_STATISTIC=m
CONFIG_NETFILTER_XT_MATCH_STRING=m
CONFIG_NETFILTER_XT_MATCH_TCPMSS=m
CONFIG_NETFILTER_XT_MATCH_HASHLIMIT=m

#
# IP: Netfilter Configuration
#
CONFIG_NF_CONNTRACK_IPV4=m
# CONFIG_NF_CONNTRACK_PROC_COMPAT is not set
CONFIG_IP_NF_QUEUE=m
CONFIG_IP_NF_IPTABLES=m
CONFIG_IP_NF_MATCH_IPRANGE=m
CONFIG_IP_NF_MATCH_TOS=m
CONFIG_IP_NF_MATCH_RECENT=m
CONFIG_IP_NF_MATCH_ECN=m
CONFIG_IP_NF_MATCH_AH=m
CONFIG_IP_NF_MATCH_TTL=m
CONFIG_IP_NF_MATCH_OWNER=m
CONFIG_IP_NF_MATCH_ADDRTYPE=m
CONFIG_IP_NF_FILTER=m
CONFIG_IP_NF_TARGET_REJECT=m
CONFIG_IP_NF_TARGET_LOG=m
CONFIG_IP_NF_TARGET_ULOG=m
CONFIG_NF_NAT=m
CONFIG_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=m
CONFIG_IP_NF_TARGET_REDIRECT=m
CONFIG_IP_NF_TARGET_NETMAP=m
CONFIG_IP_NF_TARGET_SAME=m
CONFIG_NF_NAT_SNMP_BASIC=m
CONFIG_NF_NAT_PROTO_GRE=m
CONFIG_NF_NAT_FTP=m
CONFIG_NF_NAT_IRC=m
CONFIG_NF_NAT_TFTP=m
CONFIG_NF_NAT_AMANDA=m
CONFIG_NF_NAT_PPTP=m
CONFIG_NF_NAT_H323=m
CONFIG_NF_NAT_SIP=m
CONFIG_IP_NF_MANGLE=m
CONFIG_IP_NF_TARGET_TOS=m
CONFIG_IP_NF_TARGET_ECN=m
CONFIG_IP_NF_TARGET_TTL=m
CONFIG_IP_NF_TARGET_CLUSTERIP=m
CONFIG_IP_NF_RAW=m
CONFIG_IP_NF_ARPTABLES=m
CONFIG_IP_NF_ARPFILTER=m
CONFIG_IP_NF_ARP_MANGLE=m

#
# Bridge: Netfilter Configuration
#
# CONFIG_BRIDGE_NF_EBTABLES is not set

#
# DCCP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_DCCP is not set

#
# SCTP Configuration (EXPERIMENTAL)
#
# CONFIG_IP_SCTP is not set

#
# TIPC Configuration (EXPERIMENTAL)
#
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
CONFIG_BRIDGE=m
CONFIG_VLAN_8021Q=m
# CONFIG_DECNET is not set
CONFIG_LLC=m
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set

#
# QoS and/or fair queueing
#
# CONFIG_NET_SCHED is not set
CONFIG_NET_CLS_ROUTE=y

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not set
CONFIG_FIB_RULES=y

#
# Wireless
#
# CONFIG_CFG80211 is not set
# CONFIG_WIRELESS_EXT is not set
# CONFIG_MAC80211 is not set
# CONFIG_IEEE80211 is not set
# CONFIG_RFKILL is not set

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=y
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_SYS_HYPERVISOR is not set

#
# Connector - unified userspace <-> kernelspace linker
#
CONFIG_CONNECTOR=y
CONFIG_PROC_EVENTS=y
# CONFIG_MTD is not set

#
# Parallel port support
#
# CONFIG_PARPORT is not set

#
# Plug and Play support
#
CONFIG_PNP=y
# CONFIG_PNP_DEBUG is not set

#
# Protocols
#
CONFIG_PNPACPI=y

#
# Block devices
#
# CONFIG_BLK_DEV_FD is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=m
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_NBD=m
# CONFIG_BLK_DEV_SX8 is not set
CONFIG_BLK_DEV_UB=m
CONFIG_BLK_DEV_RAM=y
CONFIG_BLK_DEV_RAM_COUNT=16
CONFIG_BLK_DEV_RAM_SIZE=16384
CONFIG_BLK_DEV_RAM_BLOCKSIZE=4096
CONFIG_CDROM_PKTCDVD=m
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
# CONFIG_ATA_OVER_ETH is not set

#
# Misc devices
#
# CONFIG_IBM_ASM is not set
# CONFIG_PHANTOM is not set
# CONFIG_SGI_IOC4 is not set
# CONFIG_TIFM_CORE is not set
# CONFIG_ASUS_LAPTOP is not set
# CONFIG_MSI_LAPTOP is not set
# CONFIG_SONY_LAPTOP is not set
# CONFIG_THINKPAD_ACPI is not set
# CONFIG_BLINK is not set
# CONFIG_IDE is not set

#
# SCSI device support
#
CONFIG_RAID_ATTRS=m
CONFIG_SCSI=m
CONFIG_SCSI_TGT=m
# CONFIG_SCSI_NETLINK is not set
CONFIG_SCSI_PROC_FS=y

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=m
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=m
CONFIG_BLK_DEV_SR_VENDOR=y
CONFIG_CHR_DEV_SG=m
CONFIG_CHR_DEV_SCH=m

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_CONSTANTS=y
CONFIG_SCSI_LOGGING=y
CONFIG_SCSI_SCAN_ASYNC=y
CONFIG_SCSI_WAIT_SCAN=m

#
# SCSI Transports
#
CONFIG_SCSI_SPI_ATTRS=m
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_ATTRS is not set
# CONFIG_SCSI_SAS_LIBSAS is not set

#
# SCSI low-level drivers
#
# CONFIG_ISCSI_TCP is not set
# CONFIG_BLK_DEV_3W_XXXX_RAID is not set
# CONFIG_SCSI_3W_9XXX is not set
# CONFIG_SCSI_ACARD is not set
# CONFIG_SCSI_AACRAID is not set
# CONFIG_SCSI_AIC7XXX is not set
# CONFIG_SCSI_AIC7XXX_OLD is not set
# CONFIG_SCSI_AIC79XX is not set
# CONFIG_SCSI_AIC94XX is not set
# CONFIG_SCSI_ARCMSR is not set
# CONFIG_MEGARAID_NEWGEN is not set
# CONFIG_MEGARAID_LEGACY is not set
# CONFIG_MEGARAID_SAS is not set
# CONFIG_SCSI_HPTIOP is not set
# CONFIG_SCSI_BUSLOGIC is not set
# CONFIG_SCSI_DMX3191D is not set
# CONFIG_SCSI_EATA is not set
# CONFIG_SCSI_FUTURE_DOMAIN is not set
# CONFIG_SCSI_GDTH is not set
# CONFIG_SCSI_IPS is not set
# CONFIG_SCSI_INITIO is not set
# CONFIG_SCSI_INIA100 is not set
# CONFIG_SCSI_STEX is not set
# CONFIG_SCSI_SYM53C8XX_2 is not set
# CONFIG_SCSI_IPR is not set
# CONFIG_SCSI_QLOGIC_1280 is not set
# CONFIG_SCSI_QLA_FC is not set
# CONFIG_SCSI_QLA_ISCSI is not set
# CONFIG_SCSI_LPFC is not set
# CONFIG_SCSI_DC395x is not set
# CONFIG_SCSI_DC390T is not set
# CONFIG_SCSI_DEBUG is not set
# CONFIG_SCSI_ESP_CORE is not set
# CONFIG_SCSI_SRP is not set
CONFIG_ATA=m
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_ACPI=y
CONFIG_SATA_AHCI=m
# CONFIG_SATA_SVW is not set
# CONFIG_ATA_PIIX is not set
# CONFIG_SATA_MV is not set
# CONFIG_SATA_NV is not set
# CONFIG_PDC_ADMA is not set
# CONFIG_SATA_QSTOR is not set
# CONFIG_SATA_PROMISE is not set
# CONFIG_SATA_SX4 is not set
# CONFIG_SATA_SIL is not set
# CONFIG_SATA_SIL24 is not set
# CONFIG_SATA_SIS is not set
# CONFIG_SATA_ULI is not set
# CONFIG_SATA_VIA is not set
# CONFIG_SATA_VITESSE is not set
# CONFIG_SATA_INIC162X is not set
# CONFIG_PATA_ALI is not set
# CONFIG_PATA_AMD is not set
# CONFIG_PATA_ARTOP is not set
CONFIG_PATA_ATIIXP=m
# CONFIG_PATA_CMD640_PCI is not set
# CONFIG_PATA_CMD64X is not set
# CONFIG_PATA_CS5520 is not set
# CONFIG_PATA_CS5530 is not set
# CONFIG_PATA_CYPRESS is not set
# CONFIG_PATA_EFAR is not set
CONFIG_ATA_GENERIC=m
# CONFIG_PATA_HPT366 is not set
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
# CONFIG_PATA_HPT3X3 is not set
# CONFIG_PATA_IT821X is not set
# CONFIG_PATA_IT8213 is not set
# CONFIG_PATA_JMICRON is not set
# CONFIG_PATA_TRIFLEX is not set
# CONFIG_PATA_MARVELL is not set
# CONFIG_PATA_MPIIX is not set
# CONFIG_PATA_OLDPIIX is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NS87410 is not set
# CONFIG_PATA_OPTI is not set
# CONFIG_PATA_OPTIDMA is not set
# CONFIG_PATA_PDC_OLD is not set
# CONFIG_PATA_RADISYS is not set
# CONFIG_PATA_RZ1000 is not set
# CONFIG_PATA_SC1200 is not set
# CONFIG_PATA_SERVERWORKS is not set
# CONFIG_PATA_PDC2027X is not set
# CONFIG_PATA_SIL680 is not set
# CONFIG_PATA_SIS is not set
# CONFIG_PATA_VIA is not set
# CONFIG_PATA_WINBOND is not set
# CONFIG_PATA_PLATFORM is not set

#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_LINEAR=m
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
# CONFIG_MD_RAID10 is not set
CONFIG_MD_RAID456=m
CONFIG_MD_RAID5_RESHAPE=y
CONFIG_MD_MULTIPATH=m
# CONFIG_MD_FAULTY is not set
CONFIG_BLK_DEV_DM=m
CONFIG_DM_DEBUG=y
CONFIG_DM_CRYPT=m
# CONFIG_DM_SNAPSHOT is not set
# CONFIG_DM_MIRROR is not set
# CONFIG_DM_ZERO is not set
# CONFIG_DM_MULTIPATH is not set
# CONFIG_DM_DELAY is not set

#
# Fusion MPT device support
#
# CONFIG_FUSION is not set
# CONFIG_FUSION_SPI is not set
# CONFIG_FUSION_FC is not set
# CONFIG_FUSION_SAS is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_FIREWIRE is not set
# CONFIG_IEEE1394 is not set

#
# I2O device support
#
CONFIG_I2O=m
# CONFIG_I2O_LCT_NOTIFY_ON_CHANGES is not set
CONFIG_I2O_EXT_ADAPTEC=y
CONFIG_I2O_EXT_ADAPTEC_DMA64=y
CONFIG_I2O_CONFIG=m
CONFIG_I2O_CONFIG_OLD_IOCTL=y
CONFIG_I2O_BUS=m
CONFIG_I2O_BLOCK=m
# CONFIG_I2O_SCSI is not set
CONFIG_I2O_PROC=m
# CONFIG_MACINTOSH_DRIVERS is not set

#
# Network device support
#
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
# CONFIG_BONDING is not set
# CONFIG_EQUALIZER is not set
CONFIG_TUN=m
# CONFIG_NET_SB1000 is not set

#
# ARCnet devices
#
# CONFIG_ARCNET is not set

#
# Ethernet (10 or 100Mbit)
#
# CONFIG_NET_ETHERNET is not set
CONFIG_NETDEV_1000=y
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
CONFIG_R8169=m
CONFIG_R8169_NAPI=y
CONFIG_R8169_VLAN=y
# CONFIG_SIS190 is not set
# CONFIG_SKGE is not set
# CONFIG_SKY2 is not set
# CONFIG_SK98LIN is not set
# CONFIG_TIGON3 is not set
# CONFIG_BNX2 is not set
# CONFIG_QLA3XXX is not set
# CONFIG_ATL1 is not set
# CONFIG_NETDEV_10000 is not set

#
# Token Ring devices
#
# CONFIG_TR is not set

#
# Wireless LAN
#
# CONFIG_WLAN_PRE80211 is not set
# CONFIG_WLAN_80211 is not set

#
# USB Network Adapters
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_USBNET_MII is not set
# CONFIG_USB_USBNET is not set
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
# CONFIG_SHAPER is not set
# CONFIG_NETCONSOLE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set

#
# ISDN subsystem
#
# CONFIG_ISDN is not set

#
# Telephony Support
#
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y
# CONFIG_INPUT_FF_MEMLESS is not set

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
# CONFIG_INPUT_MOUSEDEV_PSAUX is not set
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_TSDEV is not set
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_STOWAWAY is not set
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
CONFIG_MOUSE_PS2_LIFEBOOK=y
CONFIG_MOUSE_PS2_TRACKPOINT=y
# CONFIG_MOUSE_PS2_TOUCHKIT is not set
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_APPLETOUCH is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
CONFIG_INPUT_MISC=y
CONFIG_INPUT_PCSPKR=m
# CONFIG_INPUT_ATLAS_BTNS is not set
# CONFIG_INPUT_ATI_REMOTE is not set
# CONFIG_INPUT_ATI_REMOTE2 is not set
# CONFIG_INPUT_KEYSPAN_REMOTE is not set
# CONFIG_INPUT_POWERMATE is not set
# CONFIG_INPUT_YEALINK is not set
CONFIG_INPUT_UINPUT=m
# CONFIG_INPUT_POLLDEV is not set

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_SERPORT=y
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
CONFIG_SERIO_RAW=m
# CONFIG_GAMEPORT is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_VT_HW_CONSOLE_BINDING=y
CONFIG_SERIAL_NONSTANDARD=y
# CONFIG_COMPUTONE is not set
# CONFIG_ROCKETPORT is not set
# CONFIG_CYCLADES is not set
# CONFIG_DIGIEPCA is not set
# CONFIG_MOXA_INTELLIO is not set
# CONFIG_MOXA_SMARTIO is not set
# CONFIG_MOXA_SMARTIO_NEW is not set
# CONFIG_ISI is not set
# CONFIG_SYNCLINK is not set
# CONFIG_SYNCLINKMP is not set
# CONFIG_SYNCLINK_GT is not set
# CONFIG_N_HDLC is not set
# CONFIG_SPECIALIX is not set
# CONFIG_SX is not set
# CONFIG_RIO is not set
# CONFIG_STALDRV is not set

#
# Serial drivers
#
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_PNP=y
CONFIG_SERIAL_8250_NR_UARTS=2
CONFIG_SERIAL_8250_RUNTIME_UARTS=2
CONFIG_SERIAL_8250_EXTENDED=y
CONFIG_SERIAL_8250_MANY_PORTS=y
CONFIG_SERIAL_8250_SHARE_IRQ=y
CONFIG_SERIAL_8250_DETECT_IRQ=y
# CONFIG_SERIAL_8250_RSA is not set

#
# Non-8250 serial port support
#
CONFIG_SERIAL_CORE=y
CONFIG_SERIAL_CORE_CONSOLE=y
# CONFIG_SERIAL_JSM is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_LEGACY_PTYS is not set

#
# IPMI
#
CONFIG_IPMI_HANDLER=m
# CONFIG_IPMI_PANIC_EVENT is not set
CONFIG_IPMI_DEVICE_INTERFACE=m
CONFIG_IPMI_SI=m
CONFIG_IPMI_WATCHDOG=m
CONFIG_IPMI_POWEROFF=m
# CONFIG_WATCHDOG is not set
# CONFIG_HW_RANDOM is not set
CONFIG_NVRAM=y
CONFIG_RTC=y
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
CONFIG_AGP=y
CONFIG_AGP_AMD64=y
# CONFIG_AGP_INTEL is not set
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_VIA is not set
CONFIG_DRM=m
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
CONFIG_DRM_RADEON=m
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_SIS is not set
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
# CONFIG_MWAVE is not set
# CONFIG_PC8736x_GPIO is not set
# CONFIG_RAW_DRIVER is not set
CONFIG_HPET=y
# CONFIG_HPET_RTC_IRQ is not set
# CONFIG_HPET_MMAP is not set
# CONFIG_HANGCHECK_TIMER is not set

#
# TPM devices
#
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
CONFIG_DEVPORT=y
CONFIG_I2C=m
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_CHARDEV=m

#
# I2C Algorithms
#
CONFIG_I2C_ALGOBIT=m
# CONFIG_I2C_ALGOPCF is not set
# CONFIG_I2C_ALGOPCA is not set

#
# I2C Hardware Bus support
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
# CONFIG_I2C_AMD8111 is not set
# CONFIG_I2C_I801 is not set
# CONFIG_I2C_I810 is not set
CONFIG_I2C_PIIX4=m
CONFIG_I2C_ISA=m
# CONFIG_I2C_NFORCE2 is not set
# CONFIG_I2C_OCORES is not set
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_PROSAVAGE is not set
# CONFIG_I2C_SAVAGE4 is not set
# CONFIG_I2C_SIMTEC is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_STUB is not set
# CONFIG_I2C_TINY_USB is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set
# CONFIG_I2C_VOODOO3 is not set

#
# Miscellaneous I2C Chip support
#
# CONFIG_SENSORS_DS1337 is not set
# CONFIG_SENSORS_DS1374 is not set
# CONFIG_SENSORS_EEPROM is not set
# CONFIG_SENSORS_PCF8574 is not set
# CONFIG_SENSORS_PCA9539 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_SENSORS_MAX6875 is not set
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_I2C_DEBUG_CHIP is not set

#
# SPI support
#
# CONFIG_SPI is not set
# CONFIG_SPI_MASTER is not set

#
# Dallas's 1-wire bus
#
# CONFIG_W1 is not set
CONFIG_HWMON=m
CONFIG_HWMON_VID=m
# CONFIG_SENSORS_ABITUGURU is not set
# CONFIG_SENSORS_AD7418 is not set
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
# CONFIG_SENSORS_ADM1026 is not set
# CONFIG_SENSORS_ADM1029 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ADM9240 is not set
CONFIG_SENSORS_K8TEMP=m
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_ATXP1 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_F71805F is not set
# CONFIG_SENSORS_FSCHER is not set
# CONFIG_SENSORS_FSCPOS is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_GL520SM is not set
# CONFIG_SENSORS_CORETEMP is not set
CONFIG_SENSORS_IT87=m
# CONFIG_SENSORS_LM63 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM77 is not set
# CONFIG_SENSORS_LM78 is not set
# CONFIG_SENSORS_LM80 is not set
# CONFIG_SENSORS_LM83 is not set
# CONFIG_SENSORS_LM85 is not set
# CONFIG_SENSORS_LM87 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_LM92 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_MAX6650 is not set
# CONFIG_SENSORS_PC87360 is not set
# CONFIG_SENSORS_PC87427 is not set
# CONFIG_SENSORS_SIS5595 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_SMSC47M192 is not set
# CONFIG_SENSORS_SMSC47B397 is not set
# CONFIG_SENSORS_VIA686A is not set
# CONFIG_SENSORS_VT1211 is not set
# CONFIG_SENSORS_VT8231 is not set
# CONFIG_SENSORS_W83781D is not set
# CONFIG_SENSORS_W83791D is not set
# CONFIG_SENSORS_W83792D is not set
# CONFIG_SENSORS_W83793 is not set
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83627HF is not set
# CONFIG_SENSORS_W83627EHF is not set
# CONFIG_SENSORS_HDAPS is not set
# CONFIG_SENSORS_APPLESMC is not set
# CONFIG_HWMON_DEBUG_CHIP is not set

#
# Multifunction device drivers
#
# CONFIG_MFD_SM501 is not set

#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set
# CONFIG_DVB_CORE is not set
# CONFIG_DAB is not set

#
# Graphics support
#
CONFIG_BACKLIGHT_LCD_SUPPORT=y
CONFIG_BACKLIGHT_CLASS_DEVICE=y
CONFIG_LCD_CLASS_DEVICE=m
# CONFIG_BACKLIGHT_PROGEAR is not set

#
# Display device support
#
# CONFIG_DISPLAY_SUPPORT is not set
# CONFIG_VGASTATE is not set
CONFIG_FB=y
# CONFIG_FIRMWARE_EDID is not set
CONFIG_FB_DDC=m
CONFIG_FB_CFB_FILLRECT=m
CONFIG_FB_CFB_COPYAREA=m
CONFIG_FB_CFB_IMAGEBLIT=m
# CONFIG_FB_SYS_FILLRECT is not set
# CONFIG_FB_SYS_COPYAREA is not set
# CONFIG_FB_SYS_IMAGEBLIT is not set
# CONFIG_FB_SYS_FOPS is not set
CONFIG_FB_DEFERRED_IO=y
# CONFIG_FB_SVGALIB is not set
# CONFIG_FB_MACMODES is not set
# CONFIG_FB_BACKLIGHT is not set
CONFIG_FB_MODE_HELPERS=y
# CONFIG_FB_TILEBLITTING is not set

#
# Frame buffer hardware drivers
#
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
# CONFIG_FB_VESA is not set
# CONFIG_FB_HECUBA is not set
# CONFIG_FB_HGA is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_NVIDIA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_FB_LE80578 is not set
# CONFIG_FB_INTEL is not set
# CONFIG_FB_MATROX is not set
CONFIG_FB_RADEON=m
CONFIG_FB_RADEON_I2C=y
# CONFIG_FB_RADEON_BACKLIGHT is not set
# CONFIG_FB_RADEON_DEBUG is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_S3 is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_VT8623 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_ARK is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_GEODE is not set
# CONFIG_FB_VIRTUAL is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_VGACON_SOFT_SCROLLBACK=y
CONFIG_VGACON_SOFT_SCROLLBACK_SIZE=64
CONFIG_VIDEO_SELECT=y
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
CONFIG_LOGO=y
# CONFIG_LOGO_LINUX_MONO is not set
# CONFIG_LOGO_LINUX_VGA16 is not set
CONFIG_LOGO_LINUX_CLUT224=y

#
# Sound
#
CONFIG_SOUND=m

#
# Advanced Linux Sound Architecture
#
CONFIG_SND=m
CONFIG_SND_TIMER=m
CONFIG_SND_PCM=m
CONFIG_SND_SEQUENCER=m
CONFIG_SND_SEQ_DUMMY=m
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=m
CONFIG_SND_PCM_OSS=m
CONFIG_SND_PCM_OSS_PLUGINS=y
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_RTCTIMER=m
CONFIG_SND_SEQ_RTCTIMER_DEFAULT=y
CONFIG_SND_DYNAMIC_MINORS=y
# CONFIG_SND_SUPPORT_OLD_API is not set
CONFIG_SND_VERBOSE_PROCFS=y
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set

#
# Generic devices
#
# CONFIG_SND_DUMMY is not set
# CONFIG_SND_VIRMIDI is not set
# CONFIG_SND_MTPAV is not set
# CONFIG_SND_SERIAL_U16550 is not set
# CONFIG_SND_MPU401 is not set

#
# PCI devices
#
# CONFIG_SND_AD1889 is not set
# CONFIG_SND_ALS300 is not set
# CONFIG_SND_ALS4000 is not set
# CONFIG_SND_ALI5451 is not set
# CONFIG_SND_ATIIXP is not set
# CONFIG_SND_ATIIXP_MODEM is not set
# CONFIG_SND_AU8810 is not set
# CONFIG_SND_AU8820 is not set
# CONFIG_SND_AU8830 is not set
# CONFIG_SND_AZT3328 is not set
# CONFIG_SND_BT87X is not set
# CONFIG_SND_CA0106 is not set
# CONFIG_SND_CMIPCI is not set
# CONFIG_SND_CS4281 is not set
# CONFIG_SND_CS46XX is not set
# CONFIG_SND_DARLA20 is not set
# CONFIG_SND_GINA20 is not set
# CONFIG_SND_LAYLA20 is not set
# CONFIG_SND_DARLA24 is not set
# CONFIG_SND_GINA24 is not set
# CONFIG_SND_LAYLA24 is not set
# CONFIG_SND_MONA is not set
# CONFIG_SND_MIA is not set
# CONFIG_SND_ECHO3G is not set
# CONFIG_SND_INDIGO is not set
# CONFIG_SND_INDIGOIO is not set
# CONFIG_SND_INDIGODJ is not set
# CONFIG_SND_EMU10K1 is not set
# CONFIG_SND_EMU10K1X is not set
# CONFIG_SND_ENS1370 is not set
# CONFIG_SND_ENS1371 is not set
# CONFIG_SND_ES1938 is not set
# CONFIG_SND_ES1968 is not set
# CONFIG_SND_FM801 is not set
CONFIG_SND_HDA_INTEL=m
# CONFIG_SND_HDSP is not set
# CONFIG_SND_HDSPM is not set
# CONFIG_SND_ICE1712 is not set
# CONFIG_SND_ICE1724 is not set
# CONFIG_SND_INTEL8X0 is not set
# CONFIG_SND_INTEL8X0M is not set
# CONFIG_SND_KORG1212 is not set
# CONFIG_SND_MAESTRO3 is not set
# CONFIG_SND_MIXART is not set
# CONFIG_SND_NM256 is not set
# CONFIG_SND_PCXHR is not set
# CONFIG_SND_RIPTIDE is not set
# CONFIG_SND_RME32 is not set
# CONFIG_SND_RME96 is not set
# CONFIG_SND_RME9652 is not set
# CONFIG_SND_SONICVIBES is not set
# CONFIG_SND_TRIDENT is not set
# CONFIG_SND_VIA82XX is not set
# CONFIG_SND_VIA82XX_MODEM is not set
# CONFIG_SND_VX222 is not set
# CONFIG_SND_YMFPCI is not set

#
# USB devices
#
# CONFIG_SND_USB_AUDIO is not set
# CONFIG_SND_USB_USX2Y is not set
# CONFIG_SND_USB_CAIAQ is not set

#
# System on Chip audio support
#
# CONFIG_SND_SOC is not set

#
# Open Sound System
#
# CONFIG_SOUND_PRIME is not set

#
# HID Devices
#
CONFIG_HID=y
# CONFIG_HID_DEBUG is not set

#
# USB Input Devices
#
CONFIG_USB_HID=m
# CONFIG_USB_HIDINPUT_POWERBOOK is not set
# CONFIG_HID_FF is not set
CONFIG_USB_HIDDEV=y

#
# USB HID Boot Protocol drivers
#
# CONFIG_USB_KBD is not set
# CONFIG_USB_MOUSE is not set

#
# USB support
#
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=m
# CONFIG_USB_DEBUG is not set

#
# Miscellaneous USB options
#
CONFIG_USB_DEVICEFS=y
# CONFIG_USB_DEVICE_CLASS is not set
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_SUSPEND is not set
# CONFIG_USB_OTG is not set

#
# USB Host Controller Drivers
#
CONFIG_USB_EHCI_HCD=m
# CONFIG_USB_EHCI_SPLIT_ISO is not set
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
# CONFIG_USB_EHCI_TT_NEWSCHED is not set
# CONFIG_USB_EHCI_BIG_ENDIAN_MMIO is not set
# CONFIG_USB_ISP116X_HCD is not set
CONFIG_USB_OHCI_HCD=m
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
# CONFIG_USB_UHCI_HCD is not set
# CONFIG_USB_SL811_HCD is not set

#
# USB Device Class drivers
#
# CONFIG_USB_ACM is not set
# CONFIG_USB_PRINTER is not set

#
# NOTE: USB_STORAGE enables SCSI, and 'SCSI disk support'
#

#
# may also be needed; see USB_STORAGE Help for more information
#
CONFIG_USB_STORAGE=m
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_DATAFAB is not set
CONFIG_USB_STORAGE_FREECOM=y
CONFIG_USB_STORAGE_DPCM=y
# CONFIG_USB_STORAGE_USBAT is not set
# CONFIG_USB_STORAGE_SDDR09 is not set
# CONFIG_USB_STORAGE_SDDR55 is not set
# CONFIG_USB_STORAGE_JUMPSHOT is not set
# CONFIG_USB_STORAGE_ALAUDA is not set
CONFIG_USB_STORAGE_KARMA=y
CONFIG_USB_LIBUSUAL=y

#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set
# CONFIG_USB_MON is not set

#
# USB port drivers
#

#
# USB Serial Converter support
#
# CONFIG_USB_SERIAL is not set

#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_ADUTUX is not set
# CONFIG_USB_AUERSWALD is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_BERRY_CHARGE is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYPRESS_CY7C63 is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_PHIDGET is not set
# CONFIG_USB_IDMOUSE is not set
# CONFIG_USB_FTDI_ELAN is not set
# CONFIG_USB_APPLEDISPLAY is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
# CONFIG_USB_TRANCEVIBRATOR is not set
# CONFIG_USB_IOWARRIOR is not set
# CONFIG_USB_TEST is not set

#
# USB DSL modem support
#

#
# USB Gadget Support
#
# CONFIG_USB_GADGET is not set
# CONFIG_MMC is not set

#
# LED devices
#
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=y

#
# LED drivers
#

#
# LED Triggers
#
CONFIG_LEDS_TRIGGERS=y
CONFIG_LEDS_TRIGGER_TIMER=m
CONFIG_LEDS_TRIGGER_HEARTBEAT=m

#
# InfiniBand support
#
# CONFIG_INFINIBAND is not set

#
# EDAC - error detection and reporting (RAS) (EXPERIMENTAL)
#
CONFIG_EDAC=y

#
# Reporting subsystems
#
# CONFIG_EDAC_DEBUG is not set
CONFIG_EDAC_MM_EDAC=m
# CONFIG_EDAC_E752X is not set
CONFIG_EDAC_POLL=y

#
# Real Time Clock
#
CONFIG_RTC_LIB=m
CONFIG_RTC_CLASS=m

#
# RTC interfaces
#
CONFIG_RTC_INTF_SYSFS=y
CONFIG_RTC_INTF_PROC=y
CONFIG_RTC_INTF_DEV=y
# CONFIG_RTC_INTF_DEV_UIE_EMUL is not set
# CONFIG_RTC_DRV_TEST is not set

#
# I2C RTC drivers
#
# CONFIG_RTC_DRV_DS1307 is not set
# CONFIG_RTC_DRV_DS1672 is not set
# CONFIG_RTC_DRV_MAX6900 is not set
# CONFIG_RTC_DRV_RS5C372 is not set
# CONFIG_RTC_DRV_ISL1208 is not set
# CONFIG_RTC_DRV_X1205 is not set
# CONFIG_RTC_DRV_PCF8563 is not set
# CONFIG_RTC_DRV_PCF8583 is not set

#
# SPI RTC drivers
#

#
# Platform RTC drivers
#
CONFIG_RTC_DRV_CMOS=m
# CONFIG_RTC_DRV_DS1553 is not set
# CONFIG_RTC_DRV_DS1742 is not set
# CONFIG_RTC_DRV_M48T86 is not set
# CONFIG_RTC_DRV_V3020 is not set

#
# on-CPU RTC drivers
#

#
# DMA Engine support
#
CONFIG_DMA_ENGINE=y

#
# DMA Clients
#
CONFIG_NET_DMA=y

#
# DMA Devices
#
# CONFIG_INTEL_IOATDMA is not set

#
# Virtualization
#
CONFIG_KVM=m
# CONFIG_KVM_INTEL is not set
CONFIG_KVM_AMD=m

#
# Firmware Drivers
#
# CONFIG_EDD is not set
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set

#
# File systems
#
CONFIG_EXT2_FS=m
CONFIG_EXT2_FS_XATTR=y
CONFIG_EXT2_FS_POSIX_ACL=y
CONFIG_EXT2_FS_SECURITY=y
CONFIG_EXT2_FS_XIP=y
CONFIG_FS_XIP=y
CONFIG_EXT3_FS=m
CONFIG_EXT3_FS_XATTR=y
CONFIG_EXT3_FS_POSIX_ACL=y
CONFIG_EXT3_FS_SECURITY=y
# CONFIG_EXT4DEV_FS is not set
CONFIG_JBD=m
# CONFIG_JBD_DEBUG is not set
CONFIG_FS_MBCACHE=m
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
CONFIG_FS_POSIX_ACL=y
# CONFIG_XFS_FS is not set
# CONFIG_GFS2_FS is not set
# CONFIG_OCFS2_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
CONFIG_INOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_QUOTA=y
# CONFIG_QFMT_V1 is not set
CONFIG_QFMT_V2=m
CONFIG_QUOTACTL=y
CONFIG_DNOTIFY=y
CONFIG_AUTOFS_FS=m
CONFIG_AUTOFS4_FS=m
CONFIG_FUSE_FS=m
CONFIG_GENERIC_ACL=y

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=m
CONFIG_JOLIET=y
CONFIG_ZISOFS=y
CONFIG_UDF_FS=m
CONFIG_UDF_NLS=y

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="ascii"
CONFIG_NTFS_FS=m
# CONFIG_NTFS_DEBUG is not set
CONFIG_NTFS_RW=y

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_PROC_SYSCTL=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_RAMFS=y
CONFIG_CONFIGFS_FS=m

#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
CONFIG_ECRYPT_FS=m
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set

#
# Network File Systems
#
CONFIG_NFS_FS=m
CONFIG_NFS_V3=y
CONFIG_NFS_V3_ACL=y
CONFIG_NFS_V4=y
CONFIG_NFS_DIRECTIO=y
CONFIG_NFSD=m
CONFIG_NFSD_V2_ACL=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V3_ACL=y
CONFIG_NFSD_V4=y
CONFIG_NFSD_TCP=y
CONFIG_LOCKD=m
CONFIG_LOCKD_V4=y
CONFIG_EXPORTFS=m
CONFIG_NFS_ACL_SUPPORT=m
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=m
CONFIG_SUNRPC_GSS=m
# CONFIG_SUNRPC_BIND34 is not set
CONFIG_RPCSEC_GSS_KRB5=m
CONFIG_RPCSEC_GSS_SPKM3=m
# CONFIG_SMB_FS is not set
CONFIG_CIFS=m
# CONFIG_CIFS_STATS is not set
CONFIG_CIFS_WEAK_PW_HASH=y
CONFIG_CIFS_XATTR=y
CONFIG_CIFS_POSIX=y
# CONFIG_CIFS_DEBUG2 is not set
# CONFIG_CIFS_EXPERIMENTAL is not set
# CONFIG_NCP_FS is not set
# CONFIG_CODA_FS is not set
# CONFIG_AFS_FS is not set
# CONFIG_9P_FS is not set

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y

#
# Native Language Support
#
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="utf8"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ASCII=y
# CONFIG_NLS_ISO8859_1 is not set
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
# CONFIG_NLS_ISO8859_15 is not set
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
CONFIG_NLS_UTF8=m

#
# Distributed Lock Manager
#
# CONFIG_DLM is not set

#
# Instrumentation Support
#
# CONFIG_PROFILING is not set
# CONFIG_KPROBES is not set

#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_PRINTK_TIME is not set
# CONFIG_ENABLE_MUST_CHECK is not set
CONFIG_MAGIC_SYSRQ=y
# CONFIG_UNUSED_SYMBOLS is not set
CONFIG_DEBUG_FS=y
# CONFIG_HEADERS_CHECK is not set
CONFIG_DEBUG_KERNEL=y
CONFIG_DEBUG_SHIRQ=y
CONFIG_DETECT_SOFTLOCKUP=y
# CONFIG_SCHEDSTATS is not set
# CONFIG_TIMER_STATS is not set
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_PI_LIST=y
CONFIG_RT_MUTEX_TESTER=y
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_PROVE_LOCKING=y
CONFIG_LOCKDEP=y
# CONFIG_DEBUG_LOCKDEP is not set
CONFIG_TRACE_IRQFLAGS=y
CONFIG_DEBUG_SPINLOCK_SLEEP=y
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
CONFIG_STACKTRACE=y
# CONFIG_DEBUG_KOBJECT is not set
# CONFIG_DEBUG_BUGVERBOSE is not set
# CONFIG_DEBUG_INFO is not set
# CONFIG_DEBUG_VM is not set
# CONFIG_DEBUG_LIST is not set
# CONFIG_FRAME_POINTER is not set
CONFIG_FORCED_INLINING=y
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_FAULT_INJECTION is not set
# CONFIG_DEBUG_RODATA is not set
# CONFIG_IOMMU_DEBUG is not set
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set

#
# Security options
#
CONFIG_KEYS=y
CONFIG_KEYS_DEBUG_PROC_KEYS=y
CONFIG_SECURITY=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_NETWORK_XFRM=y
CONFIG_SECURITY_CAPABILITIES=y
# CONFIG_SECURITY_ROOTPLUG is not set
CONFIG_SECURITY_SELINUX=y
CONFIG_SECURITY_SELINUX_BOOTPARAM=y
CONFIG_SECURITY_SELINUX_BOOTPARAM_VALUE=1
CONFIG_SECURITY_SELINUX_DISABLE=y
CONFIG_SECURITY_SELINUX_DEVELOP=y
CONFIG_SECURITY_SELINUX_AVC_STATS=y
CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE=1
# CONFIG_SECURITY_SELINUX_ENABLE_SECMARK_DEFAULT is not set
# CONFIG_SECURITY_SELINUX_POLICYDB_VERSION_MAX is not set

#
# Cryptographic options
#
CONFIG_CRYPTO=y
CONFIG_CRYPTO_ALGAPI=m
CONFIG_CRYPTO_ABLKCIPHER=m
CONFIG_CRYPTO_BLKCIPHER=m
CONFIG_CRYPTO_HASH=m
CONFIG_CRYPTO_MANAGER=m
CONFIG_CRYPTO_HMAC=m
CONFIG_CRYPTO_XCBC=m
CONFIG_CRYPTO_NULL=m
CONFIG_CRYPTO_MD4=m
CONFIG_CRYPTO_MD5=m
CONFIG_CRYPTO_SHA1=m
CONFIG_CRYPTO_SHA256=m
CONFIG_CRYPTO_SHA512=m
CONFIG_CRYPTO_WP512=m
CONFIG_CRYPTO_TGR192=m
CONFIG_CRYPTO_GF128MUL=m
CONFIG_CRYPTO_ECB=m
CONFIG_CRYPTO_CBC=m
CONFIG_CRYPTO_PCBC=m
CONFIG_CRYPTO_LRW=m
CONFIG_CRYPTO_CRYPTD=m
CONFIG_CRYPTO_DES=m
CONFIG_CRYPTO_FCRYPT=m
CONFIG_CRYPTO_BLOWFISH=m
CONFIG_CRYPTO_TWOFISH=m
CONFIG_CRYPTO_TWOFISH_COMMON=m
CONFIG_CRYPTO_TWOFISH_X86_64=m
CONFIG_CRYPTO_SERPENT=m
CONFIG_CRYPTO_AES=m
CONFIG_CRYPTO_AES_X86_64=m
CONFIG_CRYPTO_CAST5=m
CONFIG_CRYPTO_CAST6=m
CONFIG_CRYPTO_TEA=m
CONFIG_CRYPTO_ARC4=m
CONFIG_CRYPTO_KHAZAD=m
CONFIG_CRYPTO_ANUBIS=m
CONFIG_CRYPTO_DEFLATE=m
CONFIG_CRYPTO_MICHAEL_MIC=m
CONFIG_CRYPTO_CRC32C=m
CONFIG_CRYPTO_CAMELLIA=m
# CONFIG_CRYPTO_TEST is not set

#
# Hardware crypto devices
#

#
# Library routines
#
CONFIG_BITREVERSE=y
CONFIG_CRC_CCITT=m
CONFIG_CRC16=m
CONFIG_CRC_ITU_T=m
CONFIG_CRC32=y
CONFIG_LIBCRC32C=m
CONFIG_ZLIB_INFLATE=m
CONFIG_ZLIB_DEFLATE=m
CONFIG_TEXTSEARCH=y
CONFIG_TEXTSEARCH_KMP=m
CONFIG_TEXTSEARCH_BM=m
CONFIG_TEXTSEARCH_FSM=m
CONFIG_PLIST=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y


Attachments:
slub-paratmeters-patch.txt (352.00 B)
1357051567-slub-paratmeters-patch.txt
config-2.6.22-rc2-hg-slub.txt (41.49 kB)
1615403583-config-2.6.22-rc2-hg-slub.txt
Download all attachments

2007-05-23 22:32:48

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Thu, 24 May 2007, Srihari Vijayaraghavan wrote:

> I'm stunned. Honestly, I have no possible explanations for this behaviour. Do
> you? I need more time to work out (until otherwise you might know a reason).

Hmmm... Bad. We have conflicting reports and no clear way to trigger the
bug. This may actually only trigger depending on how certain objects is
place in memory.

Could you boot with slub_debug and then run

slabinfo -v

to validate all slabs? If there is anything wrong with an object then it
should show in the syslog.

slabinfo.c can be found at Documentation/vm/slabinfo.c

> Oh, while at it, I created this patch (to ask you more info on what
> slub_nomerge is for):

See Documentation/vm/slub.txt which covers most of those.

slub_nomerge is not mentioned. It removes an optimization and that is only
useful for debugging strange cornercases. I hope to remove it someday.

2007-05-24 07:27:14

by Jens Axboe

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Wed, May 23 2007, Christoph Lameter wrote:
> On Wed, 23 May 2007, Jens Axboe wrote:
>
> > That works for me with the patch, .config attached.
>
> Hmmmm... That means the .config sent initially here was bogus.

?

Considering we're trying to help you fix bugs in your code, you could be
considerably more helpful.

--
Jens Axboe

2007-05-24 13:12:44

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

Name Objects Objsize Space Slabs/Part/Cpu O/S O %Fr %Ef Flg
Acpi-Namespace 635 32 69.6K 17/1/0 39 0 5 29 PZFU
Acpi-Operand 1975 64 274.4K 67/4/0 30 0 5 46 PZFU
Acpi-Parse 0 40 8.1K 2/2/0 36 0 100 0 PZFU
Acpi-ParseExt 0 64 8.1K 2/2/0 30 0 100 0 PZFU
Acpi-State 0 80 8.1K 2/2/0 26 0 100 0 PZFU
anon_vma 1636 64 233.4K 57/4/2 30 0 7 44 PZFU
arp_cache 2 300 4.0K 1/1/0 10 0 100 14 APZFU
avc_node 503 72 81.9K 20/10/2 28 0 50 44 PZFU
avtab_node 110636 24 10.7M 2635/1/0 42 0 0 24 PZFU
bdev_cache 36 1344 65.5K 8/2/0 5 1 25 73 APaZFU
bio 2 104 8.1K 2/0/2 21 0 0 2 APZFU
biovec-1 2 16 8.1K 2/0/2 46 0 0 0 APZFU
biovec-128 2 2048 16.3K 2/2/0 3 1 100 25 APZFU
biovec-16 2 256 8.1K 2/1/1 10 0 50 6 APZFU
biovec-256 2 4096 16.3K 2/0/0 1 1 0 50 APZFU
biovec-4 2 64 8.1K 2/0/2 21 0 0 1 APZFU
biovec-64 2 1024 16.3K 2/1/1 7 1 50 12 APZFU
blkdev_ioc 42 64 8.1K 2/1/1 30 0 50 32 PZFU
blkdev_queue 21 1752 49.1K 6/1/0 4 1 16 74 PZFU
blkdev_requests 16 280 16.3K 4/1/2 11 0 25 27 PZFU
buffer_head 5315 104 954.3K 233/4/2 23 0 1 57 PaZFU
cfq_io_context 45 152 16.3K 4/3/1 18 0 75 41 PZFU
cfq_queue 52 144 16.3K 4/3/1 18 0 75 45 PZFU
dentry 21166 240 6.6M 1629/2/2 13 0 0 76 PaZFU
dnotify_cache 1 40 4.0K 1/1/0 36 0 100 0 PZFU
ext3_inode_cache 8357 1376 13.6M 1672/2/2 5 1 0 83 PaZFU
ext3_xattr 842 88 143.3K 35/0/2 25 0 0 51 PaZFU
fasync_cache 2 24 4.0K 1/1/0 42 0 100 1 PZFU
file_lock_cache 4 216 8.1K 2/2/0 14 0 100 10 PZFU
files_cache 60 768 69.6K 17/3/2 4 0 17 66 APZFU
filp 2190 328 1.0M 245/0/2 9 0 0 71 APZFU
fs_cache 59 112 16.3K 4/2/2 21 0 50 40 APZFU
hugetlbfs_inode_cache 1 1008 8.1K 1/1/0 7 1 100 12 PZFU
idr_layer_cache 194 528 135.1K 33/1/0 6 0 3 75 PZFU
inet_peer_cache 1 64 4.0K 1/1/0 21 0 100 1 APZFU
inode_cache 5403 1008 6.3M 773/2/2 7 1 0 86 PaZFU
inotify_event_cache 0 40 8.1K 2/2/0 36 0 100 0 PZFU
inotify_watch_cache 81 72 16.3K 4/3/0 28 0 75 35 PZFU
ip_dst_cache 43 320 20.4K 5/0/2 9 0 0 67 APZFU
ip_fib_alias 10 48 8.1K 2/2/0 32 0 100 5 APZFU
ip_fib_hash 10 40 8.1K 2/2/0 32 0 100 4 APZFU
jbd_4k 0 4096 16.3K 2/0/2 1 1 0 0 PZFU
journal_handle 0 24 8.1K 2/0/2 42 0 0 0 PZFU
journal_head 28 96 12.2K 3/1/2 24 0 33 21 PZFU
key_jar 10 216 8.1K 2/2/0 12 0 100 26 APZFU
kmalloc-1024 297 1024 360.4K 44/2/2 7 1 4 84 PZFU
kmalloc-128 100 128 24.5K 6/3/0 20 0 50 52 PZFU
kmalloc-131072 1 131072 262.1K 1/0/0 1 6 0 50 PZFU
kmalloc-16 53453 16 4.7M 1164/6/2 46 0 0 17 PZFU
kmalloc-16384 12 16384 458.7K 14/0/2 1 3 0 42 PZFU
kmalloc-192 999 192 278.5K 68/1/2 15 0 1 68 PZFU
kmalloc-2048 607 2048 1.6M 204/3/2 3 1 1 74 PZFU
kmalloc-256 244 256 90.1K 22/3/2 12 0 13 69 PZFU
kmalloc-32 11176 32 1.1M 288/2/2 39 0 0 30 PZFU
kmalloc-32768 1 32768 65.5K 1/0/0 1 4 0 50 PZFU
kmalloc-4096 41 4096 352.2K 43/0/2 1 1 0 47 PZFU
kmalloc-512 297 512 184.3K 45/4/2 7 0 8 82 PZFU
kmalloc-64 1825 64 262.1K 64/9/2 30 0 14 44 PZFU
kmalloc-8 5574 8 454.6K 111/1/2 51 0 0 9 PZFU
kmalloc-8192 10 8192 196.6K 12/0/2 1 2 0 41 PZFU
kmalloc-96 873 96 155.6K 38/3/2 24 0 7 53 PZFU
kmalloc_dma-512 0 512 8.1K 2/0/2 7 0 0 0 dPZFU
mm_struct 59 1096 90.1K 11/2/2 6 1 18 71 APZFU
mnt_cache 27 208 12.2K 3/2/0 12 0 66 45 APZFU
mqueue_inode_cache 1 1376 8.1K 1/1/0 5 1 100 16 APZFU
names_cache 0 4096 16.3K 2/0/2 1 1 0 0 APZFU
pid 181 64 28.6K 7/2/2 30 0 28 40 PZFU
proc_inode_cache 364 1040 442.3K 54/4/2 7 1 7 85 PaZFU
radix_tree_node 4267 552 2.9M 712/1/2 6 0 0 80 PZFU
RAW 3 1088 8.1K 1/1/0 6 1 100 39 APZFU
revoke_record 0 32 8.1K 2/2/0 39 0 100 0 APZFU
revoke_table 4 16 8.1K 2/2/0 46 0 100 0 PZFU
scsi_cmd_cache 6 384 8.1K 2/0/2 8 0 0 28 APZFU
selinux_inode_security 14854 168 3.5M 875/3/2 17 0 0 69 PZFU
sgpool-128 2 4096 24.5K 3/1/0 1 1 33 33 APZFU
sgpool-16 2 512 8.1K 2/0/2 6 0 0 12 APZFU
sgpool-32 2 1024 16.3K 2/1/1 7 1 50 12 APZFU
sgpool-64 2 2048 16.3K 2/2/0 3 1 100 25 APZFU
sgpool-8 2 256 8.1K 2/0/2 10 0 0 6 APZFU
shmem_inode_cache 524 1256 729.0K 89/2/1 6 1 2 90 PZFU
sighand_cache 107 2120 311.2K 38/4/2 3 1 10 72 APZFU
signal_cache 107 784 118.7K 29/5/2 4 0 17 70 APZFU
sigqueue 0 160 8.1K 2/0/2 17 0 0 0 PZFU
skbuff_fclone_cache 0 452 8.1K 2/0/2 7 0 0 0 APZFU
skbuff_head_cache 257 224 98.3K 24/12/2 12 0 50 58 APZFU
sock_inode_cache 168 1128 253.9K 31/5/2 6 1 16 74 APaZFU
sysfs_dir_cache 8336 80 1.3M 322/0/2 26 0 0 50 PZFU
task_struct 181 3600 761.8K 93/5/2 2 1 5 85 PZFU
TCP 6 2032 40.9K 5/2/2 3 1 40 29 APZFU
tcp_bind_bucket 11 32 8.1K 2/0/2 39 0 0 4 APZFU
tw_sock_TCP 5 144 4.0K 1/0/1 16 0 0 17 APZFU
UDP 0 1104 16.3K 2/1/1 6 1 50 0 APZFU
uid_cache 4 80 8.1K 2/2/0 21 0 100 3 APZFU
UNIX 144 1216 212.9K 26/4/2 6 1 15 82 APZFU
vm_area_struct 5321 168 1.2M 317/9/2 17 0 2 68 PZFU


Attachments:
slabinfo-2.6-hg-e3a5033ec4d6.txt (7.83 kB)
2200734171-slabinfo-2.6-hg-e3a5033ec4d6.txt

2007-05-24 16:55:45

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Thu, 24 May 2007, Jens Axboe wrote:

> On Wed, May 23 2007, Christoph Lameter wrote:
> > On Wed, 23 May 2007, Jens Axboe wrote:
> >
> > > That works for me with the patch, .config attached.
> >
> > Hmmmm... That means the .config sent initially here was bogus.
>
> ?
>
> Considering we're trying to help you fix bugs in your code, you could be
> considerably more helpful.

I am trying figure out what is wrong and somehow that is not helpful? The
data that we have right now about the bug does not give us a clear
picture. One explanation that would have made it consistent would be that
the .config posted was not reflecting the actual kernel in use.



2007-05-24 16:58:19

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Thu, 24 May 2007, Srihari Vijayaraghavan wrote:

> slabinfo -v produces this error message:
> Cannot write to Acpi-Namespace/validate

You need to be root to do this. Sorry.


2007-05-24 21:30:25

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

Name Objects Objsize Space Slabs/Part/Cpu O/S O %Fr %Ef Flg
Acpi-Namespace 635 32 69.6K 17/1/0 39 0 5 29 PZFU
Acpi-Operand 1975 64 274.4K 67/4/0 30 0 5 46 PZFU
Acpi-Parse 0 40 8.1K 2/2/0 36 0 100 0 PZFU
Acpi-ParseExt 0 64 8.1K 2/2/0 30 0 100 0 PZFU
Acpi-State 0 80 8.1K 2/2/0 26 0 100 0 PZFU
anon_vma 1714 64 237.5K 58/2/2 30 0 3 46 PZFU
arp_cache 1 300 4.0K 1/1/0 10 0 100 7 APZFU
avc_node 498 72 86.0K 21/15/1 28 0 71 41 PZFU
avtab_node 110636 24 10.7M 2635/1/0 42 0 0 24 PZFU
bdev_cache 36 1344 65.5K 8/1/0 5 1 12 73 APaZFU
bio 2 104 8.1K 2/0/2 21 0 0 2 APZFU
biovec-1 2 16 8.1K 2/0/2 46 0 0 0 APZFU
biovec-128 2 2048 16.3K 2/2/0 3 1 100 25 APZFU
biovec-16 2 256 8.1K 2/2/0 10 0 100 6 APZFU
biovec-256 2 4096 16.3K 2/0/0 1 1 0 50 APZFU
biovec-4 2 64 8.1K 2/2/0 21 0 100 1 APZFU
biovec-64 2 1024 16.3K 2/2/0 7 1 100 12 APZFU
blkdev_ioc 50 64 12.2K 3/3/0 30 0 100 26 PZFU
blkdev_queue 21 1752 49.1K 6/1/0 4 1 16 74 PZFU
blkdev_requests 16 280 16.3K 4/2/2 11 0 50 27 PZFU
buffer_head 5420 104 970.7K 237/3/2 23 0 1 58 PaZFU
cfq_io_context 51 152 16.3K 4/3/0 18 0 75 47 PZFU
cfq_queue 58 144 20.4K 5/4/0 18 0 80 40 PZFU
dentry 21791 240 6.8M 1677/0/2 13 0 0 76 PaZFU
dnotify_cache 1 40 4.0K 1/1/0 36 0 100 0 PZFU
ext3_inode_cache 8478 1376 13.9M 1697/1/2 5 1 0 83 PaZFU
ext3_xattr 823 88 139.2K 34/1/1 25 0 2 52 PaZFU
fasync_cache 2 24 4.0K 1/1/0 42 0 100 1 PZFU
file_lock_cache 7 216 8.1K 2/2/0 14 0 100 18 PZFU
files_cache 67 768 77.8K 19/3/2 4 0 15 66 APZFU
filp 2531 328 1.1M 283/0/2 9 0 0 71 APZFU
fs_cache 66 112 16.3K 4/2/2 21 0 50 45 APZFU
hugetlbfs_inode_cache 1 1008 8.1K 1/1/0 7 1 100 12 PZFU
idr_layer_cache 194 528 135.1K 33/2/0 6 0 6 75 PZFU
inode_cache 5617 1008 6.5M 804/0/2 7 1 0 85 PaZFU
inotify_event_cache 0 40 8.1K 2/2/0 36 0 100 0 PZFU
inotify_watch_cache 81 72 16.3K 4/3/0 28 0 75 35 PZFU
ip_dst_cache 14 320 20.4K 5/5/0 9 0 100 21 APZFU
ip_fib_alias 14 48 8.1K 2/2/0 32 0 100 8 APZFU
ip_fib_hash 14 40 8.1K 2/2/0 32 0 100 6 APZFU
jbd_4k 0 4096 16.3K 2/0/2 1 1 0 0 PZFU
journal_handle 0 24 8.1K 2/0/2 42 0 0 0 PZFU
journal_head 12 96 8.1K 2/0/2 24 0 0 14 PZFU
key_jar 15 216 12.2K 3/2/0 12 0 66 26 APZFU
kmalloc-1024 307 1024 368.6K 45/1/2 7 1 2 85 PZFU
kmalloc-128 108 128 28.6K 7/3/0 20 0 42 48 PZFU
kmalloc-131072 1 131072 262.1K 1/0/0 1 6 0 50 PZFU
kmalloc-16 53888 16 4.8M 1173/3/2 46 0 0 17 PZFU
kmalloc-16384 13 16384 491.5K 15/0/2 1 3 0 43 PZFU
kmalloc-192 1036 192 286.7K 70/0/2 15 0 0 69 PZFU
kmalloc-2048 642 2048 1.7M 218/10/2 3 1 4 73 PZFU
kmalloc-256 245 256 94.2K 23/4/2 12 0 17 66 PZFU
kmalloc-32 11251 32 1.1M 290/2/2 39 0 0 30 PZFU
kmalloc-32768 1 32768 131.0K 2/1/0 1 4 50 25 PZFU
kmalloc-4096 43 4096 368.6K 45/0/2 1 1 0 47 PZFU
kmalloc-512 321 512 204.8K 50/6/2 7 0 12 80 PZFU
kmalloc-64 1885 64 266.2K 65/8/2 30 0 12 45 PZFU
kmalloc-8 5697 8 458.7K 112/2/2 51 0 1 9 PZFU
kmalloc-8192 11 8192 212.9K 13/0/2 1 2 0 42 PZFU
kmalloc-96 905 96 159.7K 39/1/2 24 0 2 54 PZFU
kmalloc_dma-512 0 512 8.1K 2/0/2 7 0 0 0 dPZFU
mm_struct 66 1096 106.4K 13/3/2 6 1 23 67 APZFU
mnt_cache 27 208 12.2K 3/2/0 12 0 66 45 APZFU
mqueue_inode_cache 1 1376 8.1K 1/1/0 5 1 100 16 APZFU
names_cache 0 4096 16.3K 2/0/2 1 1 0 0 APZFU
nf_nat:help 0 360 4.0K 1/0/1 9 0 0 0 PZFU
pid 198 64 32.7K 8/2/2 30 0 25 38 PZFU
proc_inode_cache 398 1040 475.1K 58/3/1 7 1 5 87 PaZFU
radix_tree_node 4425 552 3.0M 738/3/1 6 0 0 80 PZFU
RAW 3 1088 16.3K 2/2/0 6 1 100 19 APZFU
revoke_record 0 32 8.1K 2/2/0 39 0 100 0 APZFU
revoke_table 4 16 8.1K 2/2/0 46 0 100 0 PZFU
scsi_cmd_cache 6 384 8.1K 2/0/2 8 0 0 28 APZFU
selinux_inode_security 15234 168 3.6M 898/3/2 17 0 0 69 PZFU
sgpool-128 2 4096 24.5K 3/1/0 1 1 33 33 APZFU
sgpool-16 2 512 8.1K 2/0/2 6 0 0 12 APZFU
sgpool-32 2 1024 16.3K 2/2/0 7 1 100 12 APZFU
sgpool-64 2 2048 16.3K 2/2/0 3 1 100 25 APZFU
sgpool-8 2 256 8.1K 2/0/2 10 0 0 6 APZFU
shmem_inode_cache 525 1256 720.8K 88/3/0 6 1 3 91 PZFU
sighand_cache 114 2120 327.6K 40/4/2 3 1 10 73 APZFU
signal_cache 114 784 126.9K 31/4/2 4 0 12 70 APZFU
sigqueue 0 160 8.1K 2/0/2 17 0 0 0 PZFU
skbuff_fclone_cache 1 452 8.1K 2/2/0 7 0 100 5 APZFU
skbuff_head_cache 257 224 98.3K 24/10/2 12 0 41 58 APZFU
sock_inode_cache 178 1128 270.3K 33/6/1 6 1 18 74 APaZFU
sysfs_dir_cache 8919 80 1.4M 344/1/1 26 0 0 50 PZFU
task_struct 195 3600 827.3K 101/6/2 2 1 5 84 PZFU
TCP 5 2032 24.5K 3/2/0 3 1 66 41 APZFU
tcp_bind_bucket 4 32 8.1K 2/2/0 39 0 100 1 APZFU
UDP 3 1104 16.3K 2/1/1 6 1 50 20 APZFU
uid_cache 6 80 8.1K 2/2/0 21 0 100 5 APZFU
UNIX 149 1216 212.9K 26/2/1 6 1 7 85 APZFU
vm_area_struct 5621 168 1.3M 334/9/2 17 0 2 69 PZFU


Attachments:
slabinfo_without_v.txt (7.75 kB)
244129750-slabinfo_without_v.txt

2007-05-24 22:19:55

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Fri, 25 May 2007, Srihari Vijayaraghavan wrote:

> I know slabinfo_without_v might not be good enough for you. For completeness,
> I attach it here anyway.

Right. "slabinfo" output is useless. You need to specify -v for it to have
any effect.

slabinfo -v

can detect object corruptions without the kernel having to touch the
object. It will scan all objects and it will only work if run as root.
Any corrupted objects will be reported to the syslog.

2007-05-24 22:39:38

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

--- Christoph Lameter <[email protected]> wrote:
> slabinfo -v
[...]
> Any corrupted objects will be reported to the syslog.

(Only only could have made that sentence better.)

Sorry I didn't know slabinfo -v produces nothing on stdout, stderr, rather all
findings reported to syslog. Right? You don't have to answer this question, if
my assumptions are right.

If so, when ran it produced nothing in the syslog.

Sorry mate, I'm unable to spend a lot of time on slub debugging. I'll
nonetheless keep slub as my preferred memory allocator for the rest of the
2.6.22-rc process. If anything interesting turns up that I could contribute
during this phase, you'll be the first know I promise.

Thanks



___________________________________________________________________________________
How would you spend $50,000 to create a more sustainable environment in Australia? Go to Yahoo!7 Answers and share your idea.
http://advision.webevents.yahoo.com/aunz/lifestyle/answers/y7ans-babp_reg.html

2007-05-25 06:15:10

by Jens Axboe

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Thu, May 24 2007, Christoph Lameter wrote:
> On Thu, 24 May 2007, Jens Axboe wrote:
>
> > On Wed, May 23 2007, Christoph Lameter wrote:
> > > On Wed, 23 May 2007, Jens Axboe wrote:
> > >
> > > > That works for me with the patch, .config attached.
> > >
> > > Hmmmm... That means the .config sent initially here was bogus.
> >
> > ?
> >
> > Considering we're trying to help you fix bugs in your code, you could be
> > considerably more helpful.
>
> I am trying figure out what is wrong and somehow that is not helpful? The
> data that we have right now about the bug does not give us a clear

You are doing what you are supposed to do, fixing your broken code. We
are the ones that are helping you do that, by reporting the problem and
testing solutions. I think you have your attributions backwards.

> picture. One explanation that would have made it consistent would be that
> the .config posted was not reflecting the actual kernel in use.

The .config I sent is the one that the last boot of the machine was
based on. The kernel is 2.6.22-rc2 with the patch moving the #ifdef from
you, and it works for me with or without CONFIG_SLUB_DEBUG set.

--
Jens Axboe

2007-05-25 13:40:37

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Fri, 25 May 2007, Jens Axboe wrote:

> The .config I sent is the one that the last boot of the machine was
> based on. The kernel is 2.6.22-rc2 with the patch moving the #ifdef from
> you, and it works for me with or without CONFIG_SLUB_DEBUG set.

I was not talking about your config but the .config that started this
thread.


2007-05-25 14:03:16

by Jens Axboe

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

On Fri, May 25 2007, Christoph Lameter wrote:
> On Fri, 25 May 2007, Jens Axboe wrote:
>
> > The .config I sent is the one that the last boot of the machine was
> > based on. The kernel is 2.6.22-rc2 with the patch moving the #ifdef from
> > you, and it works for me with or without CONFIG_SLUB_DEBUG set.
>
> I was not talking about your config but the .config that started this
> thread.

OK, that also wasn't clear.

I'll repeat, if you want me to test anything further, please let that be
known explicitly. I have reported what works and doesn't, as far as I'm
concerned the patch works for me without incident.

--
Jens Axboe

2007-05-26 01:52:40

by Christoph Lameter

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

I finally figured out the second issue. Took some time to get that figure
out. Sorry. But now all the bug reports make sense.

Here is the fix


SLUB: Fix NUMA / SYSFS bootstrap issue

The kmem_cache_node cache is very special because it is needed for
NUMA bootstrap. Under certain conditions (like for example if lockdep is
enabled and significantly increases the size of spinlock_t) the structure
may become exactly the size as one of the larger caches in the kmalloc array.

That early during bootstrap we cannot perform merging properly. The unique
id for the kmem_cache_node cache will match one of the kmalloc array. Sysfs
will complain about a duplicate directory entry. All of this occurs while
the console is not yet fully operational. Thus boot may appear to be
silently failing.

The kmem_cache_node cache is very special. During early boostrap the main
allocation function is not operational yet and so we have to run our
own small special alloc function during early boot. It is also special in
that it is never freed.

We really do not want any merging on that cache. Set the refcount -1 and
forbid merging of slabs that have a negative refcount.

Signed-off-by: Christoph Lameter <[email protected]>

---
mm/slub.c | 7 +++++++
1 file changed, 7 insertions(+)

Index: slub/mm/slub.c
===================================================================
--- slub.orig/mm/slub.c 2007-05-25 18:28:42.000000000 -0700
+++ slub/mm/slub.c 2007-05-25 18:29:46.000000000 -0700
@@ -2473,6 +2473,7 @@ void __init kmem_cache_init(void)
*/
create_kmalloc_cache(&kmalloc_caches[0], "kmem_cache_node",
sizeof(struct kmem_cache_node), GFP_KERNEL);
+ kmalloc_caches[0].refcount = -1;
#endif

/* Able to allocate the per node structures */
@@ -2520,6 +2521,12 @@ static int slab_unmergeable(struct kmem_
if (s->ctor)
return 1;

+ /*
+ * We may have set a slab to be unmergeable during bootstrap.
+ */
+ if (s->refcount < 0)
+ return 1;
+
return 0;
}

2007-05-26 02:49:40

by Srihari Vijayaraghavan

[permalink] [raw]
Subject: Re: [PROBLEM] 2.6.22-rc2 panics on x86-64 with slub

--- Christoph Lameter <[email protected]> wrote:
> I finally figured out the second issue. Took some time to get that figure
> out. Sorry. But now all the bug reports make sense.

[...]

Impressive Christoph. Indeed, this fixes my problem on latest -git (its hg
equivalent :-)). Well done.

(Tested with CONFIG_SLAB_DEBUG without slub_debug; all was well. Then with
slub_debug & slabinfo -v also; again all was well. I had many LOCKDEP and its
relatives also compiled in.)

Now hoping it'll be in 2.6.22-rc3 :-)).

> Signed-off-by: Christoph Lameter <[email protected]>
>
> ---
> mm/slub.c | 7 +++++++
> 1 file changed, 7 insertions(+)
>
> Index: slub/mm/slub.c
> ===================================================================
> --- slub.orig/mm/slub.c 2007-05-25 18:28:42.000000000 -0700
> +++ slub/mm/slub.c 2007-05-25 18:29:46.000000000 -0700
> @@ -2473,6 +2473,7 @@ void __init kmem_cache_init(void)
> */
> create_kmalloc_cache(&kmalloc_caches[0], "kmem_cache_node",
> sizeof(struct kmem_cache_node), GFP_KERNEL);
> + kmalloc_caches[0].refcount = -1;
> #endif
>
> /* Able to allocate the per node structures */
> @@ -2520,6 +2521,12 @@ static int slab_unmergeable(struct kmem_
> if (s->ctor)
> return 1;
>
> + /*
> + * We may have set a slab to be unmergeable during bootstrap.
> + */
> + if (s->refcount < 0)
> + return 1;
> +
> return 0;
> }



___________________________________________________________________________________
How would you spend $50,000 to create a more sustainable environment in Australia? Go to Yahoo!7 Answers and share your idea.
http://advision.webevents.yahoo.com/aunz/lifestyle/answers/y7ans-babp_reg.html