2011-05-15 22:25:11

by Nix

[permalink] [raw]
Subject: [BISECTED] 2.6.39rc: kobject-related reboot after RAID array initialization(?) post-QUEUE_FLAG_REENTER-removal

After this change:

commit c21e6beba8835d09bb80e34961430b13e60381c5
Author: Jens Axboe <[email protected]>
Date: Tue Apr 19 13:32:46 2011 +0200

block: get rid of QUEUE_FLAG_REENTER

We are currently using this flag to check whether it's safe
to call into ->request_fn(). If it is set, we punt to kblockd.
But we get a lot of false positives and excessive punts to
kblockd, which hurts performance.

The only real abuser of this infrastructure is SCSI. So export
the async queue run and convert SCSI over to use that. There's
room for improvement in that SCSI need not always use the async
call, but this fixes our performance issue and they can fix that
up in due time.

Signed-off-by: Jens Axboe <[email protected]>

my system panics and reboots in early userspace. It is slightly
difficult to figure out where -- the reboot happens so fast -- but it is
either triggered by

/sbin/mdadm --assemble --scan --auto=md

(with mdadm v2.6.9, yes, I know, it's quite old but it works)

or by

/sbin/lvm vgscan --ignorelockingfailure --mknodes

(most probably the former, since I don't see any sign of lvm running in
the text that blinks up right before the reboot, and the oops below
mentions md1, not anything lvmish.

netconsole reports this (ignore the fact that md1 is resyncing, that's
because of previous instances of this bug!):

[ 6.773532] md: md0 stopped.
[ 6.976368] md: bind<sdb1>
[ 6.978284] md: bind<sda1>
[ 6.980162] bio: create slab <bio-1> at 1
[ 6.981992] md/raid1:md0: active with 2 out of 2 mirrors
[ 6.983745] md0: detected capacity change from 0 to 271319040
[ 6.987345] md: md1 stopped.
[ 6.989411] md0: unknown partition table
[ 7.000464] md: bind<sdb3>
[ 7.002247] md: bind<sda3>
[ 7.003998] md/raid1:md1: not clean -- starting background reconstruction
[ 7.005669] md/raid1:md1: active with 2 out of 2 mirrors
[ 7.007330] md1: detected capacity change from 0 to 486936436736
[ 7.008982] md: resync of RAID array md1
[ 7.008984] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
[ 7.008985] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync.
[ 7.008988] md: using 128k window, over a total of 475523864 blocks.
[ 7.008990] md: resuming resync of md1 from checkpoint.
[ 7.176568] md1: unknown partition table
[ 7.350823] general protection fault: 0000 [#1] PREEMPT SMP
[ 7.353166] last sysfs file: /sys/devices/virtual/block/md1/dev
[ 7.355496] CPU 1
[ 7.355514] Modules linked in:
[ 7.360073]
[ 7.362310] Pid: 0, comm: kworker/0:0 Not tainted 2.6.39-rc4-00119-g584f790-dirty #11
System manufacturer System Product Name /P6T
[ 7.364629] RIP: 0010:[<ffffffff8122bb01>] [<ffffffff8122bb01>] kobject_put+0x11/0x4b
[ 7.366921] RSP: 0018:ffff88033fc0e510 EFLAGS: 00010202
[ 7.369178] RAX: 0000000400000008 RBX: 3d9e2838ffff8813 RCX: 0000000000000003
[ 7.371417] RDX: ffff8803396feec8 RSI: ffff8803391ea800 RDI: 3d9e2838ffff8813
[ 7.373621] RBP: ffff88033fc0e520 R08: ffff88033fc0e530 R09: 00000000000003e8
[ 7.375827] R10: 0000000001887509 R11: 0000000200000000 R12: ffff8803391ea800
[ 7.378040] R13: ffff8803396fee00 R14: ffff88033d9e2848 R15: 0000000000001055
[ 7.380265] FS: 0000000000000000(0000) GS:ffff88033fc40000(0000) knlGS:0000000000000000
[ 7.382514] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 7.384765] CR2: 00000000004051d0 CR3: 000000033a22c000 CR4: 00000000000006e0
[ 7.387037] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 7.389325] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 7.391610] Process kworker/0:0 (pid: 0, threadinfo ffff88033e256000, task ffff88033e254300)
[ 7.393914] Stack:
[ 7.396196] ffff88033fc0e530 ffff88033d9e2800 ffff88033fc0e530 ffffffff81367f19
[ 7.398544] ffff88033fc0e580 ffffffff81381614 ffff88033a2669c0 3d9e2838ffff8803
[ 7.400876] 0000000000000053 ffff8803396fee00 0000000000000202 0000000000000246
[ 7.403207] Call Trace:
[ 7.405481] Code: 89 de 48 c7 c7 d8 ee 7d 81 31 c0 e8 c8 7b 33 00 e8
9d 79 33 00 5b 41 5c c9 c3 55 48 89 e5 53 48 89 fb 48 83 ec 08 48 85 ff
74 36 <f6> 47 3c 01 75 20 49 89 f8 48 8b 0f 48 c7 c2 ed ee 7d 81 be 53

[ 7.411141] RIP [<ffffffff8122bb01>] kobject_put+0x11/0x4b
[ 7.413725] RSP <ffff88033fc0e510>
[ 7.416289] ---[ end trace 2a57282106bd5f52 ]---
[ 7.418831] Kernel panic - not syncing: Fatal exception in interrupt
[ 7.421364] Pid: 0, comm: kworker/0:0 Tainted: G D 2.6.39-rc4-00119-g584f790-dirty #11
[ 7.423926] Call Trace:

(There is no call trace, ever. I guess it doesn't have time to get over
the network before the panic?)

My .config (this one happens to have TuxOnIce built in, but this is not
a requirement: the reboot happens in a pristine kernel too):

CONFIG_64BIT=y
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HAS_DEFAULT_IDLE=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_HAVE_CPUMASK_OF_CPU_MAP=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_ZONE_DMA32=y
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_AUDIT_ARCH=y
CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
CONFIG_HAVE_INTEL_TXT=y
CONFIG_X86_64_SMP=y
CONFIG_X86_HT=y
CONFIG_ARCH_HWEIGHT_CFLAGS="-fcall-saved-rdi -fcall-saved-rsi -fcall-saved-rdx -fcall-saved-rcx -fcall-saved-r8 -fcall-saved-r9 -fcall-saved-r10 -fcall-saved-r11"
CONFIG_ARCH_CPU_PROBE_RELEASE=y
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
CONFIG_CONSTRUCTORS=y
CONFIG_HAVE_IRQ_WORK=y
CONFIG_IRQ_WORK=y
CONFIG_EXPERIMENTAL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_CROSS_COMPILE=""
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
CONFIG_KERNEL_LZMA=y
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
CONFIG_BSD_PROCESS_ACCT=y
CONFIG_TASKSTATS=y
CONFIG_TASK_DELAY_ACCT=y
CONFIG_TASK_XACCT=y
CONFIG_TASK_IO_ACCOUNTING=y
CONFIG_AUDIT=y
CONFIG_HAVE_GENERIC_HARDIRQS=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_HAVE_SPARSE_IRQ=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_IRQ_FORCED_THREADING=y
CONFIG_SPARSE_IRQ=y
CONFIG_TREE_PREEMPT_RCU=y
CONFIG_PREEMPT_RCU=y
CONFIG_RCU_FANOUT=8
CONFIG_LOG_BUF_SHIFT=18
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
CONFIG_CGROUPS=y
CONFIG_CGROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
CONFIG_NAMESPACES=y
CONFIG_SCHED_AUTOGROUP=y
CONFIG_RELAY=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_INITRAMFS_SOURCE="usr/initramfs.mutilate"
CONFIG_INITRAMFS_ROOT_UID=99
CONFIG_INITRAMFS_ROOT_GID=101
CONFIG_RD_GZIP=y
CONFIG_RD_BZIP2=y
CONFIG_RD_LZMA=y
CONFIG_RD_XZ=y
CONFIG_RD_LZO=y
CONFIG_INITRAMFS_COMPRESSION_LZMA=y
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_PCSPKR_PLATFORM=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_HAVE_PERF_EVENTS=y
CONFIG_PERF_EVENTS=y
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_PCI_QUIRKS=y
CONFIG_SLAB=y
CONFIG_TRACEPOINTS=y
CONFIG_HAVE_OPROFILE=y
CONFIG_JUMP_LABEL=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_ATTRS=y
CONFIG_USE_GENERIC_SMP_HELPERS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_DMA_API_DEBUG=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y
CONFIG_SLABINFO=y
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
CONFIG_BLK_DEV_BSG=y
CONFIG_BLOCK_COMPAT=y
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_DEADLINE=m
CONFIG_IOSCHED_CFQ=y
CONFIG_DEFAULT_CFQ=y
CONFIG_DEFAULT_IOSCHED="cfq"
CONFIG_PREEMPT_NOTIFIERS=y
CONFIG_MUTEX_SPIN_ON_OWNER=y
CONFIG_FREEZER=y
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_SMP=y
CONFIG_X86_SUPPORTS_MEMORY_FAILURE=y
CONFIG_SCHED_OMIT_FRAME_POINTER=y
CONFIG_NO_BOOTMEM=y
CONFIG_MCORE2=y
CONFIG_X86_INTERNODE_CACHE_SHIFT=6
CONFIG_X86_CMPXCHG=y
CONFIG_CMPXCHG_LOCAL=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_XADD=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_P6_NOP=y
CONFIG_X86_TSC=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_CPU_SUP_INTEL=y
CONFIG_CPU_SUP_AMD=y
CONFIG_CPU_SUP_CENTAUR=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_DMI=y
CONFIG_GART_IOMMU=y
CONFIG_SWIOTLB=y
CONFIG_IOMMU_HELPER=y
CONFIG_IOMMU_API=y
CONFIG_NR_CPUS=8
CONFIG_SCHED_SMT=y
CONFIG_SCHED_MC=y
CONFIG_PREEMPT=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_MCE=y
CONFIG_X86_MCE_INTEL=y
CONFIG_X86_MCE_THRESHOLD=y
CONFIG_X86_THERMAL_VECTOR=y
CONFIG_MICROCODE=m
CONFIG_MICROCODE_INTEL=y
CONFIG_MICROCODE_OLD_INTERFACE=y
CONFIG_X86_MSR=m
CONFIG_X86_CPUID=y
CONFIG_ARCH_PHYS_ADDR_T_64BIT=y
CONFIG_ARCH_DMA_ADDR_T_64BIT=y
CONFIG_DIRECT_GBPAGES=y
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_HAVE_MEMORY_PRESENT=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER=y
CONFIG_SPARSEMEM_VMEMMAP=y
CONFIG_HAVE_MEMBLOCK=y
CONFIG_PAGEFLAGS_EXTENDED=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_COMPACTION=y
CONFIG_MIGRATION=y
CONFIG_PHYS_ADDR_T_64BIT=y
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_MMU_NOTIFIER=y
CONFIG_KSM=y
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
CONFIG_TRANSPARENT_HUGEPAGE=y
CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS=y
CONFIG_X86_RESERVE_LOW=64
CONFIG_MTRR=y
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_SCHED_HRTICK=y
CONFIG_PHYSICAL_START=0x1000000
CONFIG_PHYSICAL_ALIGN=0x1000000
CONFIG_HOTPLUG_CPU=y
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
CONFIG_ARCH_HIBERNATION_HEADER=y
CONFIG_HIBERNATE_CALLBACKS=y
CONFIG_HIBERNATION=y
CONFIG_PM_STD_PARTITION=""
CONFIG_TOI_CORE=y
CONFIG_TOI_SWAP=y
CONFIG_TOI_CRYPTO=y
CONFIG_TOI_USERUI=y
CONFIG_TOI_USERUI_DEFAULT_PATH="/usr/sbin/tuxoniceui_text"
CONFIG_TOI_DEFAULT_IMAGE_SIZE_LIMIT=-2
CONFIG_TOI_REPLACE_SWSUSP=y
CONFIG_TOI_IGNORE_LATE_INITCALL=y
CONFIG_TOI_DEFAULT_WAIT=-1
CONFIG_TOI_DEFAULT_EXTRA_PAGES_ALLOWANCE=50000
CONFIG_TOI_CHECKSUM=y
CONFIG_TOI=y
CONFIG_PM_SLEEP=y
CONFIG_PM_SLEEP_SMP=y
CONFIG_PM_RUNTIME=y
CONFIG_PM=y
CONFIG_ACPI=y
CONFIG_ACPI_SLEEP=y
CONFIG_ACPI_PROC_EVENT=y
CONFIG_ACPI_BUTTON=y
CONFIG_ACPI_FAN=y
CONFIG_ACPI_PROCESSOR=y
CONFIG_ACPI_HOTPLUG_CPU=y
CONFIG_ACPI_THERMAL=y
CONFIG_ACPI_CUSTOM_DSDT_FILE=""
CONFIG_ACPI_BLACKLIST_YEAR=0
CONFIG_X86_PM_TIMER=y
CONFIG_ACPI_CONTAINER=y
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=y
CONFIG_CPU_FREQ_STAT=y
CONFIG_CPU_FREQ_STAT_DETAILS=y
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
CONFIG_X86_ACPI_CPUFREQ=y
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y
CONFIG_I7300_IDLE_IOAT_CHANNEL=y
CONFIG_I7300_IDLE=y
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_DOMAINS=y
CONFIG_DMAR=y
CONFIG_DMAR_DEFAULT_ON=y
CONFIG_DMAR_FLOPPY_WA=y
CONFIG_PCIEPORTBUS=y
CONFIG_PCIEAER=y
CONFIG_PCIEASPM=y
CONFIG_PCIE_PME=y
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
CONFIG_PCI_IOV=y
CONFIG_PCI_IOAPIC=y
CONFIG_PCI_LABEL=y
CONFIG_ISA_DMA_API=y
CONFIG_AMD_NB=y
CONFIG_BINFMT_ELF=y
CONFIG_COMPAT_BINFMT_ELF=y
CONFIG_CORE_DUMP_DEFAULT_ELF_HEADERS=y
CONFIG_BINFMT_MISC=y
CONFIG_IA32_EMULATION=y
CONFIG_COMPAT=y
CONFIG_COMPAT_FOR_U64_ALIGNMENT=y
CONFIG_SYSVIPC_COMPAT=y
CONFIG_HAVE_TEXT_POKE_SMP=y
CONFIG_NET=y
CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_INET_LRO=y
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
CONFIG_IPV6=y
CONFIG_IPV6_PRIVACY=y
CONFIG_RPS=y
CONFIG_RFS_ACCEL=y
CONFIG_XPS=y
CONFIG_BT=y
CONFIG_BT_L2CAP=y
CONFIG_BT_SCO=y
CONFIG_BT_RFCOMM=y
CONFIG_BT_HCIBTUSB=y
CONFIG_UEVENT_HELPER_PATH=""
CONFIG_PREVENT_FIRMWARE_BUILD=y
CONFIG_FW_LOADER=y
CONFIG_FIRMWARE_IN_KERNEL=y
CONFIG_EXTRA_FIRMWARE="radeon/R600_rlc.bin radeon/R700_rlc.bin"
CONFIG_EXTRA_FIRMWARE_DIR="/usr/src/linux/linux-firmware"
CONFIG_ARCH_NO_SYSDEV_OPS=y
CONFIG_PNP=y
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
CONFIG_BLK_DEV_LOOP=y
CONFIG_BLK_DEV_CRYPTOLOOP=m
CONFIG_BLK_DEV_NBD=m
CONFIG_CDROM_PKTCDVD=y
CONFIG_CDROM_PKTCDVD_BUFFERS=16
CONFIG_MISC_DEVICES=y
CONFIG_HAVE_IDE=y
CONFIG_SCSI_MOD=y
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
CONFIG_BLK_DEV_SD=y
CONFIG_BLK_DEV_SR=y
CONFIG_SCSI_MULTI_LUN=y
CONFIG_SCSI_SCAN_ASYNC=y
CONFIG_SCSI_WAIT_SCAN=m
CONFIG_SCSI_LOWLEVEL=y
CONFIG_ATA=y
CONFIG_ATA_VERBOSE_ERROR=y
CONFIG_ATA_ACPI=y
CONFIG_SATA_PMP=y
CONFIG_SATA_AHCI=y
CONFIG_ATA_SFF=y
CONFIG_ATA_BMDMA=y
CONFIG_PATA_JMICRON=y
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_RAID1=y
CONFIG_BLK_DEV_DM=y
CONFIG_DM_CRYPT=y
CONFIG_DM_SNAPSHOT=y
CONFIG_DM_MIRROR=y
CONFIG_DM_ZERO=y
CONFIG_FIREWIRE=m
CONFIG_FIREWIRE_OHCI=m
CONFIG_FIREWIRE_OHCI_DEBUG=y
CONFIG_FIREWIRE_SBP2=m
CONFIG_NETDEVICES=y
CONFIG_DUMMY=m
CONFIG_MACVLAN=y
CONFIG_MACVTAP=y
CONFIG_MII=y
CONFIG_NETDEV_1000=y
CONFIG_R8169=y
CONFIG_NETCONSOLE=y
CONFIG_NETCONSOLE_DYNAMIC=y
CONFIG_NETPOLL=y
CONFIG_NET_POLL_CONTROLLER=y
CONFIG_INPUT=y
CONFIG_INPUT_MOUSEDEV=y
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1680
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=1050
CONFIG_INPUT_JOYDEV=y
CONFIG_INPUT_EVDEV=y
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
CONFIG_INPUT_MOUSE=y
CONFIG_MOUSE_PS2=y
CONFIG_MOUSE_PS2_ALPS=y
CONFIG_MOUSE_PS2_LOGIPS2PP=y
CONFIG_MOUSE_PS2_SYNAPTICS=y
CONFIG_MOUSE_PS2_LIFEBOOK=y
CONFIG_MOUSE_PS2_TRACKPOINT=y
CONFIG_INPUT_JOYSTICK=y
CONFIG_JOYSTICK_ANALOG=y
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
CONFIG_SERIO_LIBPS2=y
CONFIG_GAMEPORT=y
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
CONFIG_UNIX98_PTYS=y
CONFIG_SERIAL_8250=y
CONFIG_FIX_EARLYCON_MEM=y
CONFIG_SERIAL_8250_PCI=y
CONFIG_SERIAL_8250_PNP=y
CONFIG_SERIAL_8250_NR_UARTS=4
CONFIG_SERIAL_8250_RUNTIME_UARTS=4
CONFIG_SERIAL_CORE=y
CONFIG_NVRAM=m
CONFIG_HPET=y
CONFIG_HPET_MMAP=y
CONFIG_DEVPORT=y
CONFIG_I2C=y
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
CONFIG_I2C_CHARDEV=y
CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_ALGOBIT=y
CONFIG_I2C_I801=y
CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
CONFIG_POWER_SUPPLY=y
CONFIG_HWMON=y
CONFIG_SENSORS_CORETEMP=y
CONFIG_SENSORS_PKGTEMP=y
CONFIG_SENSORS_ATK0110=y
CONFIG_THERMAL=y
CONFIG_THERMAL_HWMON=y
CONFIG_SSB_POSSIBLE=y
CONFIG_MFD_SUPPORT=y
CONFIG_MEDIA_SUPPORT=y
CONFIG_VIDEO_DEV=y
CONFIG_VIDEO_V4L2_COMMON=y
CONFIG_VIDEO_MEDIA=y
CONFIG_MEDIA_TUNER=y
CONFIG_MEDIA_TUNER_SIMPLE=y
CONFIG_MEDIA_TUNER_TDA8290=y
CONFIG_MEDIA_TUNER_TDA827X=y
CONFIG_MEDIA_TUNER_TDA18271=y
CONFIG_MEDIA_TUNER_TDA9887=y
CONFIG_MEDIA_TUNER_TEA5761=y
CONFIG_MEDIA_TUNER_TEA5767=y
CONFIG_MEDIA_TUNER_MT20XX=y
CONFIG_MEDIA_TUNER_XC2028=y
CONFIG_MEDIA_TUNER_XC5000=y
CONFIG_MEDIA_TUNER_MC44S803=y
CONFIG_VIDEO_V4L2=y
CONFIG_VIDEO_CAPTURE_DRIVERS=y
CONFIG_VIDEO_HELPER_CHIPS_AUTO=y
CONFIG_V4L_USB_DRIVERS=y
CONFIG_USB_VIDEO_CLASS=y
CONFIG_USB_VIDEO_CLASS_INPUT_EVDEV=y
CONFIG_VGA_ARB=y
CONFIG_VGA_ARB_MAX_GPUS=2
CONFIG_DRM=y
CONFIG_DRM_KMS_HELPER=y
CONFIG_DRM_TTM=y
CONFIG_DRM_RADEON=y
CONFIG_DRM_RADEON_KMS=y
CONFIG_FB=y
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
CONFIG_BACKLIGHT_CLASS_DEVICE=y
CONFIG_VGA_CONSOLE=y
CONFIG_VGACON_SOFT_SCROLLBACK=y
CONFIG_VGACON_SOFT_SCROLLBACK_SIZE=512
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY=y
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
CONFIG_SOUND=y
CONFIG_SOUND_OSS_CORE=y
CONFIG_SND=y
CONFIG_SND_TIMER=y
CONFIG_SND_PCM=y
CONFIG_SND_JACK=y
CONFIG_SND_SEQUENCER=y
CONFIG_SND_SEQ_DUMMY=m
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=y
CONFIG_SND_PCM_OSS=y
CONFIG_SND_PCM_OSS_PLUGINS=y
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_HRTIMER=y
CONFIG_SND_SEQ_HRTIMER_DEFAULT=y
CONFIG_SND_DYNAMIC_MINORS=y
CONFIG_SND_VERBOSE_PROCFS=y
CONFIG_SND_VMASTER=y
CONFIG_SND_DMA_SGBUF=y
CONFIG_SND_PCI=y
CONFIG_SND_HDA_INTEL=y
CONFIG_SND_HDA_INPUT_JACK=y
CONFIG_SND_HDA_GENERIC=y
CONFIG_SND_HDA_POWER_SAVE=y
CONFIG_SND_HDA_POWER_SAVE_DEFAULT=0
CONFIG_HID_SUPPORT=y
CONFIG_HID=y
CONFIG_USB_HID=y
CONFIG_HID_A4TECH=y
CONFIG_HID_APPLE=y
CONFIG_HID_BELKIN=y
CONFIG_HID_CHERRY=y
CONFIG_HID_CHICONY=y
CONFIG_HID_CYPRESS=y
CONFIG_HID_EZKEY=y
CONFIG_HID_KYE=y
CONFIG_HID_KENSINGTON=y
CONFIG_HID_LOGITECH=y
CONFIG_HID_MICROSOFT=y
CONFIG_HID_MONTEREY=y
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=y
CONFIG_USB_DEVICEFS=y
CONFIG_USB_DYNAMIC_MINORS=y
CONFIG_USB_EHCI_HCD=y
CONFIG_USB_UHCI_HCD=y
CONFIG_USB_STORAGE=y
CONFIG_RTC_LIB=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_HCTOSYS=y
CONFIG_RTC_HCTOSYS_DEVICE="rtc0"
CONFIG_RTC_INTF_SYSFS=y
CONFIG_RTC_INTF_PROC=y
CONFIG_RTC_INTF_DEV=y
CONFIG_RTC_DRV_CMOS=y
CONFIG_DMADEVICES=y
CONFIG_INTEL_IOATDMA=y
CONFIG_DMA_ENGINE=y
CONFIG_NET_DMA=y
CONFIG_DCA=y
CONFIG_FIRMWARE_MEMMAP=y
CONFIG_DMIID=y
CONFIG_EXT4_FS=y
CONFIG_EXT4_USE_FOR_EXT23=y
CONFIG_EXT4_FS_XATTR=y
CONFIG_EXT4_FS_POSIX_ACL=y
CONFIG_JBD2=y
CONFIG_FS_MBCACHE=y
CONFIG_FS_POSIX_ACL=y
CONFIG_EXPORTFS=y
CONFIG_FILE_LOCKING=y
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_FANOTIFY=y
CONFIG_QUOTA=y
CONFIG_QUOTA_NETLINK_INTERFACE=y
CONFIG_PRINT_QUOTA_WARNING=y
CONFIG_QUOTA_TREE=y
CONFIG_QFMT_V2=y
CONFIG_QUOTACTL=y
CONFIG_QUOTACTL_COMPAT=y
CONFIG_FUSE_FS=y
CONFIG_CUSE=y
CONFIG_GENERIC_ACL=y
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
CONFIG_UDF_FS=y
CONFIG_UDF_NLS=y
CONFIG_FAT_FS=m
CONFIG_MSDOS_FS=m
CONFIG_VFAT_FS=m
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_PROC_FS=y
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
CONFIG_TMPFS_POSIX_ACL=y
CONFIG_HUGETLBFS=y
CONFIG_HUGETLB_PAGE=y
CONFIG_CONFIGFS_FS=y
CONFIG_NETWORK_FILESYSTEMS=y
CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
CONFIG_NFS_V3_ACL=y
CONFIG_NFSD=y
CONFIG_NFSD_V2_ACL=y
CONFIG_NFSD_V3=y
CONFIG_NFSD_V3_ACL=y
CONFIG_LOCKD=y
CONFIG_LOCKD_V4=y
CONFIG_NFS_ACL_SUPPORT=y
CONFIG_NFS_COMMON=y
CONFIG_SUNRPC=y
CONFIG_PARTITION_ADVANCED=y
CONFIG_MSDOS_PARTITION=y
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=y
CONFIG_NLS_ASCII=m
CONFIG_NLS_ISO8859_1=y
CONFIG_NLS_ISO8859_15=m
CONFIG_NLS_UTF8=m
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_PRINTK_TIME=y
CONFIG_DEFAULT_MESSAGE_LOGLEVEL=4
CONFIG_ENABLE_WARN_DEPRECATED=y
CONFIG_ENABLE_MUST_CHECK=y
CONFIG_FRAME_WARN=1024
CONFIG_STRIP_ASM_SYMS=y
CONFIG_DEBUG_FS=y
CONFIG_DEBUG_KERNEL=y
CONFIG_LOCKUP_DETECTOR=y
CONFIG_HARDLOCKUP_DETECTOR=y
CONFIG_BOOTPARAM_HARDLOCKUP_PANIC_VALUE=0
CONFIG_BOOTPARAM_SOFTLOCKUP_PANIC_VALUE=0
CONFIG_DETECT_HUNG_TASK=y
CONFIG_BOOTPARAM_HUNG_TASK_PANIC_VALUE=0
CONFIG_SCHED_DEBUG=y
CONFIG_SCHEDSTATS=y
CONFIG_TIMER_STATS=y
CONFIG_STACKTRACE=y
CONFIG_DEBUG_BUGVERBOSE=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_MEMORY_INIT=y
CONFIG_ARCH_WANT_FRAME_POINTERS=y
CONFIG_FRAME_POINTER=y
CONFIG_LATENCYTOP=y
CONFIG_SYSCTL_SYSCALL_CHECK=y
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_NOP_TRACER=y
CONFIG_HAVE_FTRACE_NMI_ENTER=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST=y
CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_RING_BUFFER=y
CONFIG_FTRACE_NMI_ENTER=y
CONFIG_EVENT_TRACING=y
CONFIG_CONTEXT_SWITCH_TRACER=y
CONFIG_TRACING=y
CONFIG_GENERIC_TRACER=y
CONFIG_TRACING_SUPPORT=y
CONFIG_FTRACE=y
CONFIG_FUNCTION_TRACER=y
CONFIG_BRANCH_PROFILE_NONE=y
CONFIG_BLK_DEV_IO_TRACE=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_ARCH_KGDB=y
CONFIG_HAVE_ARCH_KMEMCHECK=y
CONFIG_STRICT_DEVMEM=y
CONFIG_X86_VERBOSE_BOOTUP=y
CONFIG_EARLY_PRINTK=y
CONFIG_DEBUG_RODATA=y
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
CONFIG_IO_DELAY_0X80=y
CONFIG_DEFAULT_IO_DELAY_TYPE=0
CONFIG_SECURITY=y
CONFIG_SECURITYFS=y
CONFIG_SECURITY_NETWORK=y
CONFIG_SECURITY_PATH=y
CONFIG_SECURITY_APPARMOR=y
CONFIG_SECURITY_APPARMOR_BOOTPARAM_VALUE=0
CONFIG_DEFAULT_SECURITY_DAC=y
CONFIG_DEFAULT_SECURITY=""
CONFIG_ASYNC_TX_DISABLE_PQ_VAL_DMA=y
CONFIG_ASYNC_TX_DISABLE_XOR_VAL_DMA=y
CONFIG_CRYPTO=y
CONFIG_CRYPTO_ALGAPI=y
CONFIG_CRYPTO_ALGAPI2=y
CONFIG_CRYPTO_AEAD2=y
CONFIG_CRYPTO_BLKCIPHER=y
CONFIG_CRYPTO_BLKCIPHER2=y
CONFIG_CRYPTO_HASH=y
CONFIG_CRYPTO_HASH2=y
CONFIG_CRYPTO_RNG2=y
CONFIG_CRYPTO_PCOMP2=y
CONFIG_CRYPTO_MANAGER=y
CONFIG_CRYPTO_MANAGER2=y
CONFIG_CRYPTO_MANAGER_DISABLE_TESTS=y
CONFIG_CRYPTO_WORKQUEUE=y
CONFIG_CRYPTO_CBC=y
CONFIG_CRYPTO_MD4=y
CONFIG_CRYPTO_LZO=y
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_APIC_ARCHITECTURE=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_ASYNC_PF=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
CONFIG_KVM_INTEL=y
CONFIG_VHOST_NET=y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_RING=y
CONFIG_VIRTIO_PCI=m
CONFIG_VIRTIO_BALLOON=y
CONFIG_BINARY_PRINTF=y
CONFIG_BITREVERSE=y
CONFIG_GENERIC_FIND_FIRST_BIT=y
CONFIG_GENERIC_FIND_NEXT_BIT=y
CONFIG_GENERIC_FIND_LAST_BIT=y
CONFIG_CRC16=y
CONFIG_CRC_ITU_T=y
CONFIG_CRC32=y
CONFIG_ZLIB_INFLATE=y
CONFIG_LZO_COMPRESS=y
CONFIG_LZO_DECOMPRESS=y
CONFIG_XZ_DEC=y
CONFIG_XZ_DEC_X86=y
CONFIG_XZ_DEC_POWERPC=y
CONFIG_XZ_DEC_IA64=y
CONFIG_XZ_DEC_ARM=y
CONFIG_XZ_DEC_ARMTHUMB=y
CONFIG_XZ_DEC_SPARC=y
CONFIG_XZ_DEC_BCJ=y
CONFIG_DECOMPRESS_GZIP=y
CONFIG_DECOMPRESS_BZIP2=y
CONFIG_DECOMPRESS_LZMA=y
CONFIG_DECOMPRESS_XZ=y
CONFIG_DECOMPRESS_LZO=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y
CONFIG_CPU_RMAP=y
CONFIG_NLATTR=y

--
NULL && (void)


2011-05-15 23:21:27

by NeilBrown

[permalink] [raw]
Subject: Re: [BISECTED] 2.6.39rc: kobject-related reboot after RAID array initialization(?) post-QUEUE_FLAG_REENTER-removal

On Sun, 15 May 2011 23:05:32 +0100 Nix <[email protected]> wrote:

> After this change:
>
> commit c21e6beba8835d09bb80e34961430b13e60381c5
> Author: Jens Axboe <[email protected]>
> Date: Tue Apr 19 13:32:46 2011 +0200
>
> block: get rid of QUEUE_FLAG_REENTER
>
> We are currently using this flag to check whether it's safe
> to call into ->request_fn(). If it is set, we punt to kblockd.
> But we get a lot of false positives and excessive punts to
> kblockd, which hurts performance.
>
> The only real abuser of this infrastructure is SCSI. So export
> the async queue run and convert SCSI over to use that. There's
> room for improvement in that SCSI need not always use the async
> call, but this fixes our performance issue and they can fix that
> up in due time.
>
> Signed-off-by: Jens Axboe <[email protected]>
>
> my system panics and reboots in early userspace. It is slightly
> difficult to figure out where -- the reboot happens so fast -- but it is
> either triggered by
>
> /sbin/mdadm --assemble --scan --auto=md
>
> (with mdadm v2.6.9, yes, I know, it's quite old but it works)
>
> or by
>
> /sbin/lvm vgscan --ignorelockingfailure --mknodes
>
> (most probably the former, since I don't see any sign of lvm running in
> the text that blinks up right before the reboot, and the oops below
> mentions md1, not anything lvmish.
>
> netconsole reports this (ignore the fact that md1 is resyncing, that's
> because of previous instances of this bug!):
>
> [ 6.773532] md: md0 stopped.
> [ 6.976368] md: bind<sdb1>
> [ 6.978284] md: bind<sda1>
> [ 6.980162] bio: create slab <bio-1> at 1
> [ 6.981992] md/raid1:md0: active with 2 out of 2 mirrors
> [ 6.983745] md0: detected capacity change from 0 to 271319040
> [ 6.987345] md: md1 stopped.
> [ 6.989411] md0: unknown partition table
> [ 7.000464] md: bind<sdb3>
> [ 7.002247] md: bind<sda3>
> [ 7.003998] md/raid1:md1: not clean -- starting background reconstruction
> [ 7.005669] md/raid1:md1: active with 2 out of 2 mirrors
> [ 7.007330] md1: detected capacity change from 0 to 486936436736
> [ 7.008982] md: resync of RAID array md1
> [ 7.008984] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
> [ 7.008985] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync.
> [ 7.008988] md: using 128k window, over a total of 475523864 blocks.
> [ 7.008990] md: resuming resync of md1 from checkpoint.
> [ 7.176568] md1: unknown partition table
> [ 7.350823] general protection fault: 0000 [#1] PREEMPT SMP
> [ 7.353166] last sysfs file: /sys/devices/virtual/block/md1/dev
> [ 7.355496] CPU 1
> [ 7.355514] Modules linked in:
> [ 7.360073]
> [ 7.362310] Pid: 0, comm: kworker/0:0 Not tainted 2.6.39-rc4-00119-g584f790-dirty #11
> System manufacturer System Product Name /P6T
> [ 7.364629] RIP: 0010:[<ffffffff8122bb01>] [<ffffffff8122bb01>] kobject_put+0x11/0x4b
> [ 7.366921] RSP: 0018:ffff88033fc0e510 EFLAGS: 00010202
> [ 7.369178] RAX: 0000000400000008 RBX: 3d9e2838ffff8813 RCX: 0000000000000003
> [ 7.371417] RDX: ffff8803396feec8 RSI: ffff8803391ea800 RDI: 3d9e2838ffff8813
> [ 7.373621] RBP: ffff88033fc0e520 R08: ffff88033fc0e530 R09: 00000000000003e8
> [ 7.375827] R10: 0000000001887509 R11: 0000000200000000 R12: ffff8803391ea800
> [ 7.378040] R13: ffff8803396fee00 R14: ffff88033d9e2848 R15: 0000000000001055
> [ 7.380265] FS: 0000000000000000(0000) GS:ffff88033fc40000(0000) knlGS:0000000000000000
> [ 7.382514] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 7.384765] CR2: 00000000004051d0 CR3: 000000033a22c000 CR4: 00000000000006e0
> [ 7.387037] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 7.389325] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 7.391610] Process kworker/0:0 (pid: 0, threadinfo ffff88033e256000, task ffff88033e254300)
> [ 7.393914] Stack:
> [ 7.396196] ffff88033fc0e530 ffff88033d9e2800 ffff88033fc0e530 ffffffff81367f19
> [ 7.398544] ffff88033fc0e580 ffffffff81381614 ffff88033a2669c0 3d9e2838ffff8803
> [ 7.400876] 0000000000000053 ffff8803396fee00 0000000000000202 0000000000000246
> [ 7.403207] Call Trace:
> [ 7.405481] Code: 89 de 48 c7 c7 d8 ee 7d 81 31 c0 e8 c8 7b 33 00 e8
> 9d 79 33 00 5b 41 5c c9 c3 55 48 89 e5 53 48 89 fb 48 83 ec 08 48 85 ff
> 74 36 <f6> 47 3c 01 75 20 49 89 f8 48 8b 0f 48 c7 c2 ed ee 7d 81 be 53
>
> [ 7.411141] RIP [<ffffffff8122bb01>] kobject_put+0x11/0x4b
> [ 7.413725] RSP <ffff88033fc0e510>
> [ 7.416289] ---[ end trace 2a57282106bd5f52 ]---
> [ 7.418831] Kernel panic - not syncing: Fatal exception in interrupt
> [ 7.421364] Pid: 0, comm: kworker/0:0 Tainted: G D 2.6.39-rc4-00119-g584f790-dirty #11
> [ 7.423926] Call Trace:
>
> (There is no call trace, ever. I guess it doesn't have time to get over
> the network before the panic?)

That is unfortunate....

I'm having trouble seeing md implicated given the patch, but one never
knows...

I'd try reverting just the __blk_run_queue part of the patch.

i.e. leave __blk_run_queue behaving how it did before the patch, but keep the
blk_run_queue_async parts of the new code.
If that still crashes, then there must be someone going wrong when scsi calls
the new blk_run_queue_async.
If it doesn't crash, then the recursion-protection of QUEUE_FLAG_REENTER must
be protecting us from something else other than SCSI re-entering.

NeilBrown

2011-05-16 07:29:25

by Jens Axboe

[permalink] [raw]
Subject: Re: [BISECTED] 2.6.39rc: kobject-related reboot after RAID array initialization(?) post-QUEUE_FLAG_REENTER-removal

On 2011-05-16 01:21, NeilBrown wrote:
> On Sun, 15 May 2011 23:05:32 +0100 Nix <[email protected]> wrote:
>
>> After this change:
>>
>> commit c21e6beba8835d09bb80e34961430b13e60381c5
>> Author: Jens Axboe <[email protected]>
>> Date: Tue Apr 19 13:32:46 2011 +0200
>>
>> block: get rid of QUEUE_FLAG_REENTER
>>
>> We are currently using this flag to check whether it's safe
>> to call into ->request_fn(). If it is set, we punt to kblockd.
>> But we get a lot of false positives and excessive punts to
>> kblockd, which hurts performance.
>>
>> The only real abuser of this infrastructure is SCSI. So export
>> the async queue run and convert SCSI over to use that. There's
>> room for improvement in that SCSI need not always use the async
>> call, but this fixes our performance issue and they can fix that
>> up in due time.
>>
>> Signed-off-by: Jens Axboe <[email protected]>
>>
>> my system panics and reboots in early userspace. It is slightly
>> difficult to figure out where -- the reboot happens so fast -- but it is
>> either triggered by
>>
>> /sbin/mdadm --assemble --scan --auto=md
>>
>> (with mdadm v2.6.9, yes, I know, it's quite old but it works)
>>
>> or by
>>
>> /sbin/lvm vgscan --ignorelockingfailure --mknodes
>>
>> (most probably the former, since I don't see any sign of lvm running in
>> the text that blinks up right before the reboot, and the oops below
>> mentions md1, not anything lvmish.
>>
>> netconsole reports this (ignore the fact that md1 is resyncing, that's
>> because of previous instances of this bug!):
>>
>> [ 6.773532] md: md0 stopped.
>> [ 6.976368] md: bind<sdb1>
>> [ 6.978284] md: bind<sda1>
>> [ 6.980162] bio: create slab <bio-1> at 1
>> [ 6.981992] md/raid1:md0: active with 2 out of 2 mirrors
>> [ 6.983745] md0: detected capacity change from 0 to 271319040
>> [ 6.987345] md: md1 stopped.
>> [ 6.989411] md0: unknown partition table
>> [ 7.000464] md: bind<sdb3>
>> [ 7.002247] md: bind<sda3>
>> [ 7.003998] md/raid1:md1: not clean -- starting background reconstruction
>> [ 7.005669] md/raid1:md1: active with 2 out of 2 mirrors
>> [ 7.007330] md1: detected capacity change from 0 to 486936436736
>> [ 7.008982] md: resync of RAID array md1
>> [ 7.008984] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
>> [ 7.008985] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync.
>> [ 7.008988] md: using 128k window, over a total of 475523864 blocks.
>> [ 7.008990] md: resuming resync of md1 from checkpoint.
>> [ 7.176568] md1: unknown partition table
>> [ 7.350823] general protection fault: 0000 [#1] PREEMPT SMP
>> [ 7.353166] last sysfs file: /sys/devices/virtual/block/md1/dev
>> [ 7.355496] CPU 1
>> [ 7.355514] Modules linked in:
>> [ 7.360073]
>> [ 7.362310] Pid: 0, comm: kworker/0:0 Not tainted 2.6.39-rc4-00119-g584f790-dirty #11
>> System manufacturer System Product Name /P6T
>> [ 7.364629] RIP: 0010:[<ffffffff8122bb01>] [<ffffffff8122bb01>] kobject_put+0x11/0x4b
>> [ 7.366921] RSP: 0018:ffff88033fc0e510 EFLAGS: 00010202
>> [ 7.369178] RAX: 0000000400000008 RBX: 3d9e2838ffff8813 RCX: 0000000000000003
>> [ 7.371417] RDX: ffff8803396feec8 RSI: ffff8803391ea800 RDI: 3d9e2838ffff8813
>> [ 7.373621] RBP: ffff88033fc0e520 R08: ffff88033fc0e530 R09: 00000000000003e8
>> [ 7.375827] R10: 0000000001887509 R11: 0000000200000000 R12: ffff8803391ea800
>> [ 7.378040] R13: ffff8803396fee00 R14: ffff88033d9e2848 R15: 0000000000001055
>> [ 7.380265] FS: 0000000000000000(0000) GS:ffff88033fc40000(0000) knlGS:0000000000000000
>> [ 7.382514] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [ 7.384765] CR2: 00000000004051d0 CR3: 000000033a22c000 CR4: 00000000000006e0
>> [ 7.387037] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [ 7.389325] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> [ 7.391610] Process kworker/0:0 (pid: 0, threadinfo ffff88033e256000, task ffff88033e254300)
>> [ 7.393914] Stack:
>> [ 7.396196] ffff88033fc0e530 ffff88033d9e2800 ffff88033fc0e530 ffffffff81367f19
>> [ 7.398544] ffff88033fc0e580 ffffffff81381614 ffff88033a2669c0 3d9e2838ffff8803
>> [ 7.400876] 0000000000000053 ffff8803396fee00 0000000000000202 0000000000000246
>> [ 7.403207] Call Trace:
>> [ 7.405481] Code: 89 de 48 c7 c7 d8 ee 7d 81 31 c0 e8 c8 7b 33 00 e8
>> 9d 79 33 00 5b 41 5c c9 c3 55 48 89 e5 53 48 89 fb 48 83 ec 08 48 85 ff
>> 74 36 <f6> 47 3c 01 75 20 49 89 f8 48 8b 0f 48 c7 c2 ed ee 7d 81 be 53
>>
>> [ 7.411141] RIP [<ffffffff8122bb01>] kobject_put+0x11/0x4b
>> [ 7.413725] RSP <ffff88033fc0e510>
>> [ 7.416289] ---[ end trace 2a57282106bd5f52 ]---
>> [ 7.418831] Kernel panic - not syncing: Fatal exception in interrupt
>> [ 7.421364] Pid: 0, comm: kworker/0:0 Tainted: G D 2.6.39-rc4-00119-g584f790-dirty #11
>> [ 7.423926] Call Trace:
>>
>> (There is no call trace, ever. I guess it doesn't have time to get over
>> the network before the panic?)
>
> That is unfortunate....
>
> I'm having trouble seeing md implicated given the patch, but one never
> knows...
>
> I'd try reverting just the __blk_run_queue part of the patch.
>
> i.e. leave __blk_run_queue behaving how it did before the patch, but keep the
> blk_run_queue_async parts of the new code.
> If that still crashes, then there must be someone going wrong when scsi calls
> the new blk_run_queue_async.
> If it doesn't crash, then the recursion-protection of QUEUE_FLAG_REENTER must
> be protecting us from something else other than SCSI re-entering.

I don't think it's the re-enter part, it looks more like a missing
reference to the queue. The problem is most likely there before the
change as well, the difference is just that there's a bigger window
where the caller can now drop the reference before the queue is run and
thus accessed.

--
Jens Axboe

2011-05-16 10:05:35

by Nix

[permalink] [raw]
Subject: Re: [BISECTED] 2.6.39rc: kobject-related reboot after RAID array initialization(?) post-QUEUE_FLAG_REENTER-removal

On 16 May 2011, NeilBrown said:

> On Sun, 15 May 2011 23:05:32 +0100 Nix <[email protected]> wrote:
>
>> After this change:
>>
>> commit c21e6beba8835d09bb80e34961430b13e60381c5
>> Author: Jens Axboe <[email protected]>
>> Date: Tue Apr 19 13:32:46 2011 +0200
>>
>> block: get rid of QUEUE_FLAG_REENTER
>>
>> We are currently using this flag to check whether it's safe
>> to call into ->request_fn(). If it is set, we punt to kblockd.
>> But we get a lot of false positives and excessive punts to
>> kblockd, which hurts performance.
>>
>> The only real abuser of this infrastructure is SCSI. So export
>> the async queue run and convert SCSI over to use that. There's
>> room for improvement in that SCSI need not always use the async
>> call, but this fixes our performance issue and they can fix that
>> up in due time.
>>
>> Signed-off-by: Jens Axboe <[email protected]>
>>
>> my system panics and reboots in early userspace. It is slightly
>> difficult to figure out where -- the reboot happens so fast -- but it is
>> either triggered by
>>
>> /sbin/mdadm --assemble --scan --auto=md
>>
>> (with mdadm v2.6.9, yes, I know, it's quite old but it works)
>>
>> or by
>>
>> /sbin/lvm vgscan --ignorelockingfailure --mknodes

No it isn't. I'm sorry for misleading you. I ran the commands manually
one by one in an emergency boot shell until I got a panic, and md is
blameless. More below.

>> (most probably the former, since I don't see any sign of lvm running in
>> the text that blinks up right before the reboot, and the oops below
>> mentions md1, not anything lvmish.
>>
>> netconsole reports this (ignore the fact that md1 is resyncing, that's
>> because of previous instances of this bug!):
>>
>> [ 6.773532] md: md0 stopped.
>> [ 6.976368] md: bind<sdb1>
>> [ 6.978284] md: bind<sda1>
>> [ 6.980162] bio: create slab <bio-1> at 1
>> [ 6.981992] md/raid1:md0: active with 2 out of 2 mirrors
>> [ 6.983745] md0: detected capacity change from 0 to 271319040
>> [ 6.987345] md: md1 stopped.
>> [ 6.989411] md0: unknown partition table
>> [ 7.000464] md: bind<sdb3>
>> [ 7.002247] md: bind<sda3>
>> [ 7.003998] md/raid1:md1: not clean -- starting background reconstruction
>> [ 7.005669] md/raid1:md1: active with 2 out of 2 mirrors
>> [ 7.007330] md1: detected capacity change from 0 to 486936436736
>> [ 7.008982] md: resync of RAID array md1
>> [ 7.008984] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
>> [ 7.008985] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync.
>> [ 7.008988] md: using 128k window, over a total of 475523864 blocks.
>> [ 7.008990] md: resuming resync of md1 from checkpoint.
>> [ 7.176568] md1: unknown partition table
>> [ 7.350823] general protection fault: 0000 [#1] PREEMPT SMP
>> [ 7.353166] last sysfs file: /sys/devices/virtual/block/md1/dev
>> [ 7.355496] CPU 1
>> [ 7.355514] Modules linked in:
>> [ 7.360073]
>> [ 7.362310] Pid: 0, comm: kworker/0:0 Not tainted 2.6.39-rc4-00119-g584f790-dirty #11
>> System manufacturer System Product Name /P6T
>> [ 7.364629] RIP: 0010:[<ffffffff8122bb01>] [<ffffffff8122bb01>] kobject_put+0x11/0x4b
>> [ 7.366921] RSP: 0018:ffff88033fc0e510 EFLAGS: 00010202
>> [ 7.369178] RAX: 0000000400000008 RBX: 3d9e2838ffff8813 RCX: 0000000000000003
>> [ 7.371417] RDX: ffff8803396feec8 RSI: ffff8803391ea800 RDI: 3d9e2838ffff8813
>> [ 7.373621] RBP: ffff88033fc0e520 R08: ffff88033fc0e530 R09: 00000000000003e8
>> [ 7.375827] R10: 0000000001887509 R11: 0000000200000000 R12: ffff8803391ea800
>> [ 7.378040] R13: ffff8803396fee00 R14: ffff88033d9e2848 R15: 0000000000001055
>> [ 7.380265] FS: 0000000000000000(0000) GS:ffff88033fc40000(0000) knlGS:0000000000000000
>> [ 7.382514] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>> [ 7.384765] CR2: 00000000004051d0 CR3: 000000033a22c000 CR4: 00000000000006e0
>> [ 7.387037] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>> [ 7.389325] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>> [ 7.391610] Process kworker/0:0 (pid: 0, threadinfo ffff88033e256000, task ffff88033e254300)
>> [ 7.393914] Stack:
>> [ 7.396196] ffff88033fc0e530 ffff88033d9e2800 ffff88033fc0e530 ffffffff81367f19
>> [ 7.398544] ffff88033fc0e580 ffffffff81381614 ffff88033a2669c0 3d9e2838ffff8803
>> [ 7.400876] 0000000000000053 ffff8803396fee00 0000000000000202 0000000000000246
>> [ 7.403207] Call Trace:
>> [ 7.405481] Code: 89 de 48 c7 c7 d8 ee 7d 81 31 c0 e8 c8 7b 33 00 e8
>> 9d 79 33 00 5b 41 5c c9 c3 55 48 89 e5 53 48 89 fb 48 83 ec 08 48 85 ff
>> 74 36 <f6> 47 3c 01 75 20 49 89 f8 48 8b 0f 48 c7 c2 ed ee 7d 81 be 53
>>
>> [ 7.411141] RIP [<ffffffff8122bb01>] kobject_put+0x11/0x4b
>> [ 7.413725] RSP <ffff88033fc0e510>
>> [ 7.416289] ---[ end trace 2a57282106bd5f52 ]---
>> [ 7.418831] Kernel panic - not syncing: Fatal exception in interrupt
>> [ 7.421364] Pid: 0, comm: kworker/0:0 Tainted: G D 2.6.39-rc4-00119-g584f790-dirty #11
>> [ 7.423926] Call Trace:

This crash is caused by *fsck*, to be specific by this line in my
initramfs:

fsck -t $TYPE -a $ROOT

where $TYPE is "ext4" and $ROOT is "/dev/main/root", an filesystem atop
LVM atop md.

fsck kicks up, does a journal replay, and then we panic. Why we panic is
unclear: it's hard to save output from strace in an emergency boot shell
with nothing mounted, and I suspect that if fsck panics, mount will
panic too (but I haven't tried it yet).

--
NULL && (void)

2011-05-16 10:35:25

by Jens Axboe

[permalink] [raw]
Subject: Re: [BISECTED] 2.6.39rc: kobject-related reboot after RAID array initialization(?) post-QUEUE_FLAG_REENTER-removal

On 2011-05-16 12:05, Nix wrote:
> On 16 May 2011, NeilBrown said:
>
>> On Sun, 15 May 2011 23:05:32 +0100 Nix <[email protected]> wrote:
>>
>>> After this change:
>>>
>>> commit c21e6beba8835d09bb80e34961430b13e60381c5
>>> Author: Jens Axboe <[email protected]>
>>> Date: Tue Apr 19 13:32:46 2011 +0200
>>>
>>> block: get rid of QUEUE_FLAG_REENTER
>>>
>>> We are currently using this flag to check whether it's safe
>>> to call into ->request_fn(). If it is set, we punt to kblockd.
>>> But we get a lot of false positives and excessive punts to
>>> kblockd, which hurts performance.
>>>
>>> The only real abuser of this infrastructure is SCSI. So export
>>> the async queue run and convert SCSI over to use that. There's
>>> room for improvement in that SCSI need not always use the async
>>> call, but this fixes our performance issue and they can fix that
>>> up in due time.
>>>
>>> Signed-off-by: Jens Axboe <[email protected]>
>>>
>>> my system panics and reboots in early userspace. It is slightly
>>> difficult to figure out where -- the reboot happens so fast -- but it is
>>> either triggered by
>>>
>>> /sbin/mdadm --assemble --scan --auto=md
>>>
>>> (with mdadm v2.6.9, yes, I know, it's quite old but it works)
>>>
>>> or by
>>>
>>> /sbin/lvm vgscan --ignorelockingfailure --mknodes
>
> No it isn't. I'm sorry for misleading you. I ran the commands manually
> one by one in an emergency boot shell until I got a panic, and md is
> blameless. More below.
>
>>> (most probably the former, since I don't see any sign of lvm running in
>>> the text that blinks up right before the reboot, and the oops below
>>> mentions md1, not anything lvmish.
>>>
>>> netconsole reports this (ignore the fact that md1 is resyncing, that's
>>> because of previous instances of this bug!):
>>>
>>> [ 6.773532] md: md0 stopped.
>>> [ 6.976368] md: bind<sdb1>
>>> [ 6.978284] md: bind<sda1>
>>> [ 6.980162] bio: create slab <bio-1> at 1
>>> [ 6.981992] md/raid1:md0: active with 2 out of 2 mirrors
>>> [ 6.983745] md0: detected capacity change from 0 to 271319040
>>> [ 6.987345] md: md1 stopped.
>>> [ 6.989411] md0: unknown partition table
>>> [ 7.000464] md: bind<sdb3>
>>> [ 7.002247] md: bind<sda3>
>>> [ 7.003998] md/raid1:md1: not clean -- starting background reconstruction
>>> [ 7.005669] md/raid1:md1: active with 2 out of 2 mirrors
>>> [ 7.007330] md1: detected capacity change from 0 to 486936436736
>>> [ 7.008982] md: resync of RAID array md1
>>> [ 7.008984] md: minimum _guaranteed_ speed: 1000 KB/sec/disk.
>>> [ 7.008985] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync.
>>> [ 7.008988] md: using 128k window, over a total of 475523864 blocks.
>>> [ 7.008990] md: resuming resync of md1 from checkpoint.
>>> [ 7.176568] md1: unknown partition table
>>> [ 7.350823] general protection fault: 0000 [#1] PREEMPT SMP
>>> [ 7.353166] last sysfs file: /sys/devices/virtual/block/md1/dev
>>> [ 7.355496] CPU 1
>>> [ 7.355514] Modules linked in:
>>> [ 7.360073]
>>> [ 7.362310] Pid: 0, comm: kworker/0:0 Not tainted 2.6.39-rc4-00119-g584f790-dirty #11
>>> System manufacturer System Product Name /P6T
>>> [ 7.364629] RIP: 0010:[<ffffffff8122bb01>] [<ffffffff8122bb01>] kobject_put+0x11/0x4b
>>> [ 7.366921] RSP: 0018:ffff88033fc0e510 EFLAGS: 00010202
>>> [ 7.369178] RAX: 0000000400000008 RBX: 3d9e2838ffff8813 RCX: 0000000000000003
>>> [ 7.371417] RDX: ffff8803396feec8 RSI: ffff8803391ea800 RDI: 3d9e2838ffff8813
>>> [ 7.373621] RBP: ffff88033fc0e520 R08: ffff88033fc0e530 R09: 00000000000003e8
>>> [ 7.375827] R10: 0000000001887509 R11: 0000000200000000 R12: ffff8803391ea800
>>> [ 7.378040] R13: ffff8803396fee00 R14: ffff88033d9e2848 R15: 0000000000001055
>>> [ 7.380265] FS: 0000000000000000(0000) GS:ffff88033fc40000(0000) knlGS:0000000000000000
>>> [ 7.382514] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
>>> [ 7.384765] CR2: 00000000004051d0 CR3: 000000033a22c000 CR4: 00000000000006e0
>>> [ 7.387037] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
>>> [ 7.389325] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
>>> [ 7.391610] Process kworker/0:0 (pid: 0, threadinfo ffff88033e256000, task ffff88033e254300)
>>> [ 7.393914] Stack:
>>> [ 7.396196] ffff88033fc0e530 ffff88033d9e2800 ffff88033fc0e530 ffffffff81367f19
>>> [ 7.398544] ffff88033fc0e580 ffffffff81381614 ffff88033a2669c0 3d9e2838ffff8803
>>> [ 7.400876] 0000000000000053 ffff8803396fee00 0000000000000202 0000000000000246
>>> [ 7.403207] Call Trace:
>>> [ 7.405481] Code: 89 de 48 c7 c7 d8 ee 7d 81 31 c0 e8 c8 7b 33 00 e8
>>> 9d 79 33 00 5b 41 5c c9 c3 55 48 89 e5 53 48 89 fb 48 83 ec 08 48 85 ff
>>> 74 36 <f6> 47 3c 01 75 20 49 89 f8 48 8b 0f 48 c7 c2 ed ee 7d 81 be 53
>>>
>>> [ 7.411141] RIP [<ffffffff8122bb01>] kobject_put+0x11/0x4b
>>> [ 7.413725] RSP <ffff88033fc0e510>
>>> [ 7.416289] ---[ end trace 2a57282106bd5f52 ]---
>>> [ 7.418831] Kernel panic - not syncing: Fatal exception in interrupt
>>> [ 7.421364] Pid: 0, comm: kworker/0:0 Tainted: G D 2.6.39-rc4-00119-g584f790-dirty #11
>>> [ 7.423926] Call Trace:
>
> This crash is caused by *fsck*, to be specific by this line in my
> initramfs:
>
> fsck -t $TYPE -a $ROOT
>
> where $TYPE is "ext4" and $ROOT is "/dev/main/root", an filesystem atop
> LVM atop md.
>
> fsck kicks up, does a journal replay, and then we panic. Why we panic is
> unclear: it's hard to save output from strace in an emergency boot shell
> with nothing mounted, and I suspect that if fsck panics, mount will
> panic too (but I haven't tried it yet).

Out of curiousity, does this patch make a difference?

diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c
index 0bac91e..ec1803a 100644
--- a/drivers/scsi/scsi_lib.c
+++ b/drivers/scsi/scsi_lib.c
@@ -74,8 +74,6 @@ struct kmem_cache *scsi_sdb_cache;
*/
#define SCSI_QUEUE_DELAY 3

-static void scsi_run_queue(struct request_queue *q);
-
/*
* Function: scsi_unprep_request()
*
@@ -161,7 +159,7 @@ static int __scsi_queue_insert(struct scsi_cmnd *cmd, int reason, int unbusy)
blk_requeue_request(q, cmd->request);
spin_unlock_irqrestore(q->queue_lock, flags);

- scsi_run_queue(q);
+ kblockd_schedule_work(q, &device->requeue_work);

return 0;
}
@@ -438,7 +436,11 @@ static void scsi_run_queue(struct request_queue *q)
continue;
}

- blk_run_queue_async(sdev->request_queue);
+ spin_unlock(shost->host_lock);
+ spin_lock(sdev->request_queue->queue_lock);
+ __blk_run_queue(sdev->request_queue);
+ spin_unlock(sdev->request_queue->queue_lock);
+ spin_lock(shost->host_lock);
}
/* put any unprocessed entries back */
list_splice(&starved_list, &shost->starved_list);
@@ -447,6 +449,16 @@ static void scsi_run_queue(struct request_queue *q)
blk_run_queue(q);
}

+void scsi_requeue_run_queue(struct work_struct *work)
+{
+ struct scsi_device *sdev;
+ struct request_queue *q;
+
+ sdev = container_of(work, struct scsi_device, requeue_work);
+ q = sdev->request_queue;
+ scsi_run_queue(q);
+}
+
/*
* Function: scsi_requeue_command()
*
diff --git a/drivers/scsi/scsi_scan.c b/drivers/scsi/scsi_scan.c
index 087821f..58584dc 100644
--- a/drivers/scsi/scsi_scan.c
+++ b/drivers/scsi/scsi_scan.c
@@ -242,6 +242,7 @@ static struct scsi_device *scsi_alloc_sdev(struct scsi_target *starget,
int display_failure_msg = 1, ret;
struct Scsi_Host *shost = dev_to_shost(starget->dev.parent);
extern void scsi_evt_thread(struct work_struct *work);
+ extern void scsi_requeue_run_queue(struct work_struct *work);

sdev = kzalloc(sizeof(*sdev) + shost->transportt->device_size,
GFP_ATOMIC);
@@ -264,6 +265,7 @@ static struct scsi_device *scsi_alloc_sdev(struct scsi_target *starget,
INIT_LIST_HEAD(&sdev->event_list);
spin_lock_init(&sdev->list_lock);
INIT_WORK(&sdev->event_work, scsi_evt_thread);
+ INIT_WORK(&sdev->requeue_work, scsi_requeue_run_queue);

sdev->sdev_gendev.parent = get_device(&starget->dev);
sdev->sdev_target = starget;
diff --git a/include/scsi/scsi_device.h b/include/scsi/scsi_device.h
index 2d3ec50..dd82e02 100644
--- a/include/scsi/scsi_device.h
+++ b/include/scsi/scsi_device.h
@@ -169,6 +169,7 @@ struct scsi_device {
sdev_dev;

struct execute_work ew; /* used to get process context on put */
+ struct work_struct requeue_work;

struct scsi_dh_data *scsi_dh_data;
enum scsi_device_state sdev_state;

--
Jens Axboe

2011-05-16 21:17:17

by Nix

[permalink] [raw]
Subject: Re: [BISECTED] 2.6.39rc: kobject-related reboot after RAID array initialization(?) post-QUEUE_FLAG_REENTER-removal

On 16 May 2011, Jens Axboe verbalised:
> Out of curiousity, does this patch make a difference?

Yes. Crash cured, I can boot again.

Now I only have to worry about this ext4-quota-related panic on reboot,
completely uncaptured by netconsole (just as netconsole seems to fail to
capture almost every interesting piece of data these days).

I'll probably have to take a photo and transcribe it :(