2008-02-10 14:01:05

by Alexey Dobriyan

[permalink] [raw]
Subject: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

This happened during LTP. FWIW, modprobe/rmmod trivial empty module
together with cat /proc/*/wchan and cat /proc/modules were also running.

Box is E6400, much debugging is on, config below.


[ 4057.310000] BUG: unable to handle kernel paging request at ffff810101dbc008
[ 4057.310000] IP: [<ffffffff802596c8>] iov_iter_advance+0x38/0x70
[ 4057.310000] PGD 8063 PUD c063 PMD 153baa163 PTE 8000000101dbc160
[ 4057.310000] Oops: 0000 [1] SMP DEBUG_PAGEALLOC
[ 4057.310000] CPU 0
[ 4057.310000] Modules linked in: [last unloaded: foo]
[ 4057.310000] Pid: 7035, comm: ftest03 Not tainted 2.6.24-25f666300625d894ebe04bac2b4b3aadb907c861 #2
[ 4057.310000] RIP: 0010:[<ffffffff802596c8>] [<ffffffff802596c8>] iov_iter_advance+0x38/0x70
[ 4057.310000] RSP: 0018:ffff810110329b20 EFLAGS: 00010246
[ 4057.310000] RAX: 0000000000000000 RBX: 0000000000000800 RCX: 0000000000000000
[ 4057.310000] RDX: 0000000000000000 RSI: 0000000000000800 RDI: ffff810110329ba8
[ 4057.310000] RBP: 0000000000000800 R08: 0000000000000000 R09: ffff810101dbc000
[ 4057.310000] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000026000
[ 4057.310000] R13: ffff81010d765c98 R14: 0000000000001000 R15: 0000000000000000
[ 4057.310000] FS: 00007fee589146d0(0000) GS:ffffffff80501000(0000) knlGS:0000000000000000
[ 4057.310000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 4057.310000] CR2: ffff810101dbc008 CR3: 00000001103da000 CR4: 00000000000006e0
[ 4057.310000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 4057.310000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 4057.310000] Process ftest03 (pid: 7035, threadinfo ffff810110328000, task ffff810160b00000)
[ 4057.310000] Stack: ffffffff8025b413 ffff81010d765ab0 ffffffff804e6fd8 00000000001201d2
[ 4057.310000] ffff810110329db8 0000000000026000 ffff810110329d38 ffff81017b9fb500
[ 4057.310000] ffff81010d765c98 ffffffff804175e0 ffff81010d765ab0 0000000000000000
[ 4057.310000] Call Trace:
[ 4057.310000] [<ffffffff8025b413>] ? generic_file_buffered_write+0x1e3/0x6f0
[ 4057.310000] [<ffffffff8022f4ae>] ? current_fs_time+0x1e/0x30
[ 4057.310000] [<ffffffff8025bbaf>] ? __generic_file_aio_write_nolock+0x28f/0x440
[ 4057.310000] [<ffffffff8025bdc3>] ? generic_file_aio_write+0x63/0xd0
[ 4057.310000] [<ffffffff802bfbc3>] ? ext3_file_write+0x23/0xc0
[ 4057.310000] [<ffffffff802bfba0>] ? ext3_file_write+0x0/0xc0
[ 4057.310000] [<ffffffff8027ebeb>] ? do_sync_readv_writev+0xcb/0x110
[ 4057.310000] [<ffffffff8023f2b0>] ? autoremove_wake_function+0x0/0x30
[ 4057.310000] [<ffffffff8024be8d>] ? debug_check_no_locks_freed+0x7d/0x130
[ 4057.310000] [<ffffffff8024bd8f>] ? trace_hardirqs_on+0xcf/0x150
[ 4057.310000] [<ffffffff8027c265>] ? __kmalloc+0x15/0xc0
[ 4057.310000] [<ffffffff8027ea4d>] ? rw_copy_check_uvector+0x9d/0x130
[ 4057.310000] [<ffffffff8027f330>] ? do_readv_writev+0xe0/0x170
[ 4057.310000] [<ffffffff803fb5e7>] ? mutex_lock_nested+0x1a7/0x280
[ 4057.310000] [<ffffffff8024bd8f>] ? trace_hardirqs_on+0xcf/0x150
[ 4057.310000] [<ffffffff803fb389>] ? __mutex_unlock_slowpath+0xc9/0x170
[ 4057.310000] [<ffffffff8024bd8f>] ? trace_hardirqs_on+0xcf/0x150
[ 4057.310000] [<ffffffff803fc6ba>] ? trace_hardirqs_on_thunk+0x35/0x3a
[ 4057.310000] [<ffffffff8027f7c3>] ? sys_writev+0x53/0x90
[ 4057.310000] [<ffffffff8020b39b>] ? system_call_after_swapgs+0x7b/0x80
[ 4057.310000]
[ 4057.310000]
[ 4057.310000] Code: 48 01 77 10 48 29 77 18 c3 0f 0b eb fe 66 66 90 66 66 90 4c 8b 0f 48 8b 4f 10 49 89 f0 eb 07 66 66 66 90 49 29 c0 4d 85 c0 75 07 <49> 83 79 08 00 75 23 49 8b 51 08 48 89 d0 48 29 c8 49 39 c0 49
[ 4057.310000] RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70
[ 4057.310000] RSP <ffff810110329b20>
[ 4057.310000] CR2: ffff810101dbc008
[ 4057.310000] Kernel panic - not syncing: Fatal exception


#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.24-25f666300625d894ebe04bac2b4b3aadb907c861
# Sun Feb 10 14:28:45 2008
#
CONFIG_64BIT=y
# CONFIG_X86_32 is not set
CONFIG_X86_64=y
CONFIG_X86=y
# CONFIG_GENERIC_LOCKBREAK is not set
CONFIG_GENERIC_TIME=y
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_SEMAPHORE_SLEEPERS=y
CONFIG_FAST_CMPXCHG_LOCAL=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
# CONFIG_QUICKLIST is not set
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_HWEIGHT=y
# CONFIG_GENERIC_GPIO is not set
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_DMI=y
CONFIG_RWSEM_GENERIC_SPINLOCK=y
# CONFIG_RWSEM_XCHGADD_ALGORITHM is not set
# CONFIG_ARCH_HAS_ILOG2_U32 is not set
# CONFIG_ARCH_HAS_ILOG2_U64 is not set
CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_ZONE_DMA32=y
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_AUDIT_ARCH=y
CONFIG_ARCH_SUPPORTS_AOUT=y
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_X86_SMP=y
CONFIG_X86_64_SMP=y
CONFIG_X86_HT=y
CONFIG_X86_TRAMPOLINE=y
# CONFIG_KTIME_SCALAR is not set
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"

#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_LOCALVERSION=""
# CONFIG_LOCALVERSION_AUTO is not set
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
# CONFIG_POSIX_MQUEUE is not set
# CONFIG_BSD_PROCESS_ACCT is not set
# CONFIG_TASKSTATS is not set
# CONFIG_AUDIT is not set
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=15
# CONFIG_CGROUPS is not set
# CONFIG_FAIR_GROUP_SCHED is not set
# CONFIG_SYSFS_DEPRECATED is not set
# CONFIG_RELAY is not set
# CONFIG_NAMESPACES is not set
# CONFIG_BLK_DEV_INITRD is not set
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SYSCTL=y
CONFIG_EMBEDDED=y
# CONFIG_SYSCTL_SYSCALL is not set
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
# CONFIG_COMPAT_BRK is not set
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
# CONFIG_EPOLL is not set
# CONFIG_SIGNALFD is not set
# CONFIG_TIMERFD is not set
# CONFIG_EVENTFD is not set
CONFIG_SHMEM=y
# CONFIG_VM_EVENT_COUNTERS is not set
# CONFIG_SLUB_DEBUG is not set
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLOB is not set
# CONFIG_PROFILING is not set
# CONFIG_MARKERS is not set
CONFIG_HAVE_OPROFILE=y
# CONFIG_KPROBES is not set
CONFIG_HAVE_KPROBES=y
# CONFIG_PROC_PAGE_MONITOR is not set
CONFIG_SLABINFO=y
CONFIG_RT_MUTEXES=y
# CONFIG_TINY_SHMEM is not set
CONFIG_BASE_SMALL=0
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
# CONFIG_MODULE_FORCE_UNLOAD is not set
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
# CONFIG_BLK_DEV_IO_TRACE is not set
# CONFIG_BLK_DEV_BSG is not set

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
# CONFIG_IOSCHED_AS is not set
# CONFIG_IOSCHED_DEADLINE is not set
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_AS is not set
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"
CONFIG_CLASSIC_RCU=y
# CONFIG_PREEMPT_RCU is not set
CONFIG_FOO=m

#
# Processor type and features
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_SMP=y
CONFIG_X86_PC=y
# CONFIG_X86_ELAN is not set
# CONFIG_X86_VOYAGER is not set
# CONFIG_X86_NUMAQ is not set
# CONFIG_X86_SUMMIT is not set
# CONFIG_X86_BIGSMP is not set
# CONFIG_X86_VISWS is not set
# CONFIG_X86_GENERICARCH is not set
# CONFIG_X86_ES7000 is not set
# CONFIG_X86_RDC321X is not set
# CONFIG_X86_VSMP is not set
# CONFIG_PARAVIRT_GUEST is not set
# CONFIG_M386 is not set
# CONFIG_M486 is not set
# CONFIG_M586 is not set
# CONFIG_M586TSC is not set
# CONFIG_M586MMX is not set
# CONFIG_M686 is not set
# CONFIG_MPENTIUMII is not set
# CONFIG_MPENTIUMIII is not set
# CONFIG_MPENTIUMM is not set
# CONFIG_MPENTIUM4 is not set
# CONFIG_MK6 is not set
# CONFIG_MK7 is not set
# CONFIG_MK8 is not set
# CONFIG_MCRUSOE is not set
# CONFIG_MEFFICEON is not set
# CONFIG_MWINCHIPC6 is not set
# CONFIG_MWINCHIP2 is not set
# CONFIG_MWINCHIP3D is not set
# CONFIG_MGEODEGX1 is not set
# CONFIG_MGEODE_LX is not set
# CONFIG_MCYRIXIII is not set
# CONFIG_MVIAC3_2 is not set
# CONFIG_MVIAC7 is not set
# CONFIG_MPSC is not set
CONFIG_MCORE2=y
# CONFIG_GENERIC_CPU is not set
CONFIG_X86_L1_CACHE_BYTES=64
CONFIG_X86_INTERNODE_CACHE_BYTES=64
CONFIG_X86_CMPXCHG=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_GOOD_APIC=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_TSC=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_GART_IOMMU=y
# CONFIG_CALGARY_IOMMU is not set
CONFIG_IOMMU_HELPER=y
CONFIG_SWIOTLB=y
CONFIG_NR_CPUS=2
# CONFIG_SCHED_SMT is not set
# CONFIG_SCHED_MC is not set
CONFIG_PREEMPT_NONE=y
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT is not set
# CONFIG_RCU_TRACE is not set
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_MCE=y
CONFIG_X86_MCE_INTEL=y
# CONFIG_X86_MCE_AMD is not set
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set
# CONFIG_NUMA is not set
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_SELECT_MEMORY_MODEL=y
# CONFIG_FLATMEM_MANUAL is not set
# CONFIG_DISCONTIGMEM_MANUAL is not set
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_HAVE_MEMORY_PRESENT=y
# CONFIG_SPARSEMEM_STATIC is not set
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSEMEM_VMEMMAP=y
# CONFIG_MEMORY_HOTPLUG is not set
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_RESOURCES_64BIT=y
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_MTRR=y
# CONFIG_EFI is not set
# CONFIG_SECCOMP is not set
# CONFIG_CC_STACKPROTECTOR is not set
CONFIG_HZ_100=y
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=100
CONFIG_SCHED_HRTICK=y
# CONFIG_KEXEC is not set
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x200000
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_ALIGN=0x200000
# CONFIG_HOTPLUG_CPU is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y

#
# Power management options
#
CONFIG_PM=y
# CONFIG_PM_LEGACY is not set
# CONFIG_PM_DEBUG is not set
# CONFIG_SUSPEND is not set
# CONFIG_HIBERNATION is not set
CONFIG_ACPI=y
# CONFIG_ACPI_PROCFS is not set
# CONFIG_ACPI_PROCFS_POWER is not set
# CONFIG_ACPI_SYSFS_POWER is not set
# CONFIG_ACPI_PROC_EVENT is not set
# CONFIG_ACPI_AC is not set
# CONFIG_ACPI_BATTERY is not set
# CONFIG_ACPI_BUTTON is not set
# CONFIG_ACPI_FAN is not set
# CONFIG_ACPI_DOCK is not set
# CONFIG_ACPI_PROCESSOR is not set
# CONFIG_ACPI_WMI is not set
# CONFIG_ACPI_ASUS is not set
# CONFIG_ACPI_TOSHIBA is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_SYSTEM=y
# CONFIG_X86_PM_TIMER is not set
# CONFIG_ACPI_CONTAINER is not set
# CONFIG_ACPI_SBS is not set

#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set
# CONFIG_CPU_IDLE is not set

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
# CONFIG_PCI_MMCONFIG is not set
CONFIG_PCI_DOMAINS=y
# CONFIG_DMAR is not set
# CONFIG_PCIEPORTBUS is not set
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
# CONFIG_PCI_LEGACY is not set
# CONFIG_PCI_DEBUG is not set
# CONFIG_HT_IRQ is not set
CONFIG_ISA_DMA_API=y
CONFIG_K8_NB=y
# CONFIG_PCCARD is not set
# CONFIG_HOTPLUG_PCI is not set

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
# CONFIG_BINFMT_MISC is not set
# CONFIG_IA32_EMULATION is not set
# CONFIG_COMPAT_FOR_U64_ALIGNMENT is not set

#
# Networking
#
CONFIG_NET=y

#
# Networking options
#
# CONFIG_PACKET is not set
CONFIG_UNIX=y
# CONFIG_NET_KEY is not set
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
CONFIG_IP_FIB_HASH=y
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_ARPD is not set
# CONFIG_SYN_COOKIES is not set
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
# CONFIG_INET_XFRM_TUNNEL is not set
# CONFIG_INET_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_INET_LRO is not set
# CONFIG_INET_DIAG is not set
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
# CONFIG_IPV6 is not set
# CONFIG_INET6_XFRM_TUNNEL is not set
# CONFIG_INET6_TUNNEL is not set
# CONFIG_NETWORK_SECMARK is not set
# CONFIG_NETFILTER is not set
# CONFIG_IP_DCCP is not set
# CONFIG_IP_SCTP is not set
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_SCHED is not set

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not set

#
# Wireless
#
# CONFIG_CFG80211 is not set
# CONFIG_WIRELESS_EXT is not set
# CONFIG_MAC80211 is not set
# CONFIG_IEEE80211 is not set
# CONFIG_RFKILL is not set
# CONFIG_NET_9P is not set

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
# CONFIG_FW_LOADER is not set
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_SYS_HYPERVISOR is not set
# CONFIG_CONNECTOR is not set
# CONFIG_MTD is not set
# CONFIG_PARPORT is not set
CONFIG_PNP=y
# CONFIG_PNP_DEBUG is not set

#
# Protocols
#
CONFIG_PNPACPI=y
# CONFIG_BLK_DEV is not set
# CONFIG_MISC_DEVICES is not set
CONFIG_HAVE_IDE=y
# CONFIG_IDE is not set

#
# SCSI device support
#
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
# CONFIG_SCSI_TGT is not set
# CONFIG_SCSI_NETLINK is not set
# CONFIG_SCSI_PROC_FS is not set

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=y
# CONFIG_BLK_DEV_SR_VENDOR is not set
# CONFIG_CHR_DEV_SG is not set
# CONFIG_CHR_DEV_SCH is not set

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set
# CONFIG_SCSI_SCAN_ASYNC is not set
CONFIG_SCSI_WAIT_SCAN=m

#
# SCSI Transports
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_LIBSAS is not set
# CONFIG_SCSI_SRP_ATTRS is not set
# CONFIG_SCSI_LOWLEVEL is not set
CONFIG_ATA=y
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_ACPI=y
CONFIG_SATA_AHCI=y
# CONFIG_SATA_SVW is not set
CONFIG_ATA_PIIX=y
# CONFIG_SATA_MV is not set
# CONFIG_SATA_NV is not set
# CONFIG_PDC_ADMA is not set
# CONFIG_SATA_QSTOR is not set
# CONFIG_SATA_PROMISE is not set
# CONFIG_SATA_SX4 is not set
# CONFIG_SATA_SIL is not set
# CONFIG_SATA_SIL24 is not set
# CONFIG_SATA_SIS is not set
# CONFIG_SATA_ULI is not set
# CONFIG_SATA_VIA is not set
# CONFIG_SATA_VITESSE is not set
# CONFIG_SATA_INIC162X is not set
# CONFIG_PATA_ACPI is not set
# CONFIG_PATA_ALI is not set
# CONFIG_PATA_AMD is not set
# CONFIG_PATA_ARTOP is not set
# CONFIG_PATA_ATIIXP is not set
# CONFIG_PATA_CMD640_PCI is not set
# CONFIG_PATA_CMD64X is not set
# CONFIG_PATA_CS5520 is not set
# CONFIG_PATA_CS5530 is not set
# CONFIG_PATA_CYPRESS is not set
# CONFIG_PATA_EFAR is not set
# CONFIG_ATA_GENERIC is not set
# CONFIG_PATA_HPT366 is not set
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
# CONFIG_PATA_HPT3X3 is not set
# CONFIG_PATA_IT821X is not set
# CONFIG_PATA_IT8213 is not set
CONFIG_PATA_JMICRON=y
# CONFIG_PATA_TRIFLEX is not set
# CONFIG_PATA_MARVELL is not set
# CONFIG_PATA_MPIIX is not set
# CONFIG_PATA_OLDPIIX is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NINJA32 is not set
# CONFIG_PATA_NS87410 is not set
# CONFIG_PATA_NS87415 is not set
# CONFIG_PATA_OPTI is not set
# CONFIG_PATA_OPTIDMA is not set
# CONFIG_PATA_PDC_OLD is not set
# CONFIG_PATA_RADISYS is not set
# CONFIG_PATA_RZ1000 is not set
# CONFIG_PATA_SC1200 is not set
# CONFIG_PATA_SERVERWORKS is not set
# CONFIG_PATA_PDC2027X is not set
# CONFIG_PATA_SIL680 is not set
# CONFIG_PATA_SIS is not set
# CONFIG_PATA_VIA is not set
# CONFIG_PATA_WINBOND is not set
# CONFIG_PATA_PLATFORM is not set
# CONFIG_MD is not set
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_FIREWIRE is not set
# CONFIG_IEEE1394 is not set
# CONFIG_I2O is not set
# CONFIG_MACINTOSH_DRIVERS is not set
CONFIG_NETDEVICES=y
# CONFIG_NETDEVICES_MULTIQUEUE is not set
# CONFIG_DUMMY is not set
# CONFIG_BONDING is not set
# CONFIG_MACVLAN is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set
# CONFIG_VETH is not set
# CONFIG_NET_SB1000 is not set
# CONFIG_ARCNET is not set
# CONFIG_NET_ETHERNET is not set
CONFIG_MII=y
CONFIG_NETDEV_1000=y
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_E1000E is not set
# CONFIG_E1000E_ENABLED is not set
# CONFIG_IP1000 is not set
# CONFIG_IGB is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SIS190 is not set
# CONFIG_SKGE is not set
# CONFIG_SKY2 is not set
# CONFIG_SK98LIN is not set
# CONFIG_VIA_VELOCITY is not set
# CONFIG_TIGON3 is not set
# CONFIG_BNX2 is not set
# CONFIG_QLA3XXX is not set
CONFIG_ATL1=y
# CONFIG_NETDEV_10000 is not set
# CONFIG_TR is not set

#
# Wireless LAN
#
# CONFIG_WLAN_PRE80211 is not set
# CONFIG_WLAN_80211 is not set
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
CONFIG_NETCONSOLE=y
# CONFIG_NETCONSOLE_DYNAMIC is not set
CONFIG_NETPOLL=y
# CONFIG_NETPOLL_TRAP is not set
CONFIG_NET_POLL_CONTROLLER=y
# CONFIG_ISDN is not set
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y
# CONFIG_INPUT_FF_MEMLESS is not set
# CONFIG_INPUT_POLLDEV is not set

#
# Userland interfaces
#
# CONFIG_INPUT_MOUSEDEV is not set
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_STOWAWAY is not set
# CONFIG_INPUT_MOUSE is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
# CONFIG_GAMEPORT is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_VT_HW_CONSOLE_BINDING is not set
# CONFIG_SERIAL_NONSTANDARD is not set
# CONFIG_NOZOMI is not set

#
# Serial drivers
#
# CONFIG_SERIAL_8250 is not set
CONFIG_FIX_EARLYCON_MEM=y

#
# Non-8250 serial port support
#
# CONFIG_SERIAL_JSM is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_IPMI_HANDLER is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_MWAVE is not set
# CONFIG_PC8736x_GPIO is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HPET is not set
# CONFIG_HANGCHECK_TIMER is not set
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
CONFIG_DEVPORT=y
# CONFIG_I2C is not set

#
# SPI support
#
# CONFIG_SPI is not set
# CONFIG_SPI_MASTER is not set
# CONFIG_W1 is not set
# CONFIG_POWER_SUPPLY is not set
# CONFIG_HWMON is not set
# CONFIG_THERMAL is not set
# CONFIG_WATCHDOG is not set

#
# Sonics Silicon Backplane
#
CONFIG_SSB_POSSIBLE=y
# CONFIG_SSB is not set

#
# Multifunction device drivers
#
# CONFIG_MFD_SM501 is not set

#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set
# CONFIG_DVB_CORE is not set
# CONFIG_DAB is not set

#
# Graphics support
#
CONFIG_AGP=y
CONFIG_AGP_AMD64=y
# CONFIG_AGP_INTEL is not set
# CONFIG_AGP_SIS is not set
# CONFIG_AGP_VIA is not set
# CONFIG_DRM is not set
# CONFIG_VGASTATE is not set
# CONFIG_VIDEO_OUTPUT_CONTROL is not set
# CONFIG_FB is not set
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set

#
# Display device support
#
# CONFIG_DISPLAY_SUPPORT is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_VGACON_SOFT_SCROLLBACK is not set
# CONFIG_VIDEO_SELECT is not set
CONFIG_DUMMY_CONSOLE=y

#
# Sound
#
# CONFIG_SOUND is not set
# CONFIG_HID_SUPPORT is not set
# CONFIG_USB_SUPPORT is not set
# CONFIG_MMC is not set
# CONFIG_MEMSTICK is not set
# CONFIG_NEW_LEDS is not set
# CONFIG_INFINIBAND is not set
# CONFIG_EDAC is not set
# CONFIG_RTC_CLASS is not set
# CONFIG_DMADEVICES is not set

#
# Userspace I/O
#
# CONFIG_UIO is not set

#
# Firmware Drivers
#
# CONFIG_EDD is not set
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
# CONFIG_DMIID is not set

#
# File systems
#
CONFIG_EXT2_FS=y
# CONFIG_EXT2_FS_XATTR is not set
# CONFIG_EXT2_FS_XIP is not set
CONFIG_EXT3_FS=y
# CONFIG_EXT3_FS_XATTR is not set
# CONFIG_EXT4DEV_FS is not set
CONFIG_JBD=y
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
# CONFIG_FS_POSIX_ACL is not set
# CONFIG_XFS_FS is not set
# CONFIG_GFS2_FS is not set
# CONFIG_OCFS2_FS is not set
# CONFIG_DNOTIFY is not set
# CONFIG_INOTIFY is not set
# CONFIG_QUOTA is not set
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set
# CONFIG_FUSE_FS is not set

#
# CD-ROM/DVD Filesystems
#
# CONFIG_ISO9660_FS is not set
# CONFIG_UDF_FS is not set

#
# DOS/FAT/NT Filesystems
#
# CONFIG_MSDOS_FS is not set
# CONFIG_VFAT_FS is not set
# CONFIG_NTFS_FS is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
# CONFIG_PROC_KCORE is not set
CONFIG_PROC_SYSCTL=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_POSIX_ACL is not set
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
# CONFIG_CONFIGFS_FS is not set

#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_ROMFS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
# CONFIG_NETWORK_FILESYSTEMS is not set

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y
# CONFIG_NLS is not set
# CONFIG_DLM is not set

#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
CONFIG_PRINTK_TIME=y
# CONFIG_ENABLE_WARN_DEPRECATED is not set
# CONFIG_ENABLE_MUST_CHECK is not set
CONFIG_MAGIC_SYSRQ=y
# CONFIG_UNUSED_SYMBOLS is not set
# CONFIG_DEBUG_FS is not set
# CONFIG_HEADERS_CHECK is not set
CONFIG_DEBUG_KERNEL=y
CONFIG_DEBUG_SHIRQ=y
CONFIG_DETECT_SOFTLOCKUP=y
# CONFIG_SCHED_DEBUG is not set
# CONFIG_SCHEDSTATS is not set
# CONFIG_TIMER_STATS is not set
# CONFIG_SLUB_STATS is not set
CONFIG_DEBUG_RT_MUTEXES=y
CONFIG_DEBUG_PI_LIST=y
# CONFIG_RT_MUTEX_TESTER is not set
CONFIG_DEBUG_SPINLOCK=y
CONFIG_DEBUG_MUTEXES=y
CONFIG_DEBUG_LOCK_ALLOC=y
CONFIG_PROVE_LOCKING=y
CONFIG_LOCKDEP=y
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_LOCKDEP is not set
CONFIG_TRACE_IRQFLAGS=y
CONFIG_DEBUG_SPINLOCK_SLEEP=y
CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
CONFIG_STACKTRACE=y
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_INFO is not set
CONFIG_DEBUG_VM=y
CONFIG_DEBUG_LIST=y
CONFIG_DEBUG_SG=y
# CONFIG_FRAME_POINTER is not set
# CONFIG_BOOT_PRINTK_DELAY is not set
# CONFIG_RCU_TORTURE_TEST is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_FAULT_INJECTION is not set
# CONFIG_LATENCYTOP is not set
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
# CONFIG_SAMPLES is not set
# CONFIG_EARLY_PRINTK is not set
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set
CONFIG_DEBUG_PAGEALLOC=y
CONFIG_DEBUG_PER_CPU_MAPS=y
CONFIG_DEBUG_RODATA=y
CONFIG_DEBUG_RODATA_TEST=y
# CONFIG_DEBUG_NX_TEST is not set
CONFIG_X86_MPPARSE=y
# CONFIG_IOMMU_DEBUG is not set
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_DEFAULT_IO_DELAY_TYPE=0
# CONFIG_CPA_DEBUG is not set

#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY is not set
# CONFIG_SECURITY_FILE_CAPABILITIES is not set
# CONFIG_CRYPTO is not set
CONFIG_HAVE_KVM=y
# CONFIG_VIRTUALIZATION is not set

#
# Library routines
#
CONFIG_BITREVERSE=y
# CONFIG_CRC_CCITT is not set
# CONFIG_CRC16 is not set
# CONFIG_CRC_ITU_T is not set
CONFIG_CRC32=y
# CONFIG_CRC7 is not set
# CONFIG_LIBCRC32C is not set
CONFIG_PLIST=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y


2008-02-10 17:46:21

by Alexey Dobriyan

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Sun, Feb 10, 2008 at 05:00:31PM +0300, wrote:
> This happened during LTP. FWIW, modprobe/rmmod trivial empty module
> together with cat /proc/*/wchan and cat /proc/modules were also running.
>
> Box is E6400, much debugging is on, config below.
>
>
> [ 4057.310000] BUG: unable to handle kernel paging request at ffff810101dbc008
> [ 4057.310000] IP: [<ffffffff802596c8>] iov_iter_advance+0x38/0x70
> [ 4057.310000] PGD 8063 PUD c063 PMD 153baa163 PTE 8000000101dbc160
> [ 4057.310000] Oops: 0000 [1] SMP DEBUG_PAGEALLOC
> [ 4057.310000] CPU 0
> [ 4057.310000] Modules linked in: [last unloaded: foo]
> [ 4057.310000] Pid: 7035, comm: ftest03 Not tainted 2.6.24-25f666300625d894ebe04bac2b4b3aadb907c861 #2
> [ 4057.310000] RIP: 0010:[<ffffffff802596c8>] [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

Reproducible, BTW. As in, leave LTP for a while, go to sauna, return,
watch the corpse.

2008-02-12 22:04:45

by Andrew Morton

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Sun, 10 Feb 2008 17:00:31 +0300
Alexey Dobriyan <[email protected]> wrote:

> This happened during LTP. FWIW, modprobe/rmmod trivial empty module
> together with cat /proc/*/wchan and cat /proc/modules were also running.
>
> Box is E6400, much debugging is on, config below.
>
>
> [ 4057.310000] BUG: unable to handle kernel paging request at ffff810101dbc008
> [ 4057.310000] IP: [<ffffffff802596c8>] iov_iter_advance+0x38/0x70
> [ 4057.310000] PGD 8063 PUD c063 PMD 153baa163 PTE 8000000101dbc160
> [ 4057.310000] Oops: 0000 [1] SMP DEBUG_PAGEALLOC
> [ 4057.310000] CPU 0
> [ 4057.310000] Modules linked in: [last unloaded: foo]

what is this foo.ko of which you speak, and did it wreck your kernel?

> [ 4057.310000] Pid: 7035, comm: ftest03 Not tainted 2.6.24-25f666300625d894ebe04bac2b4b3aadb907c861 #2
> [ 4057.310000] RIP: 0010:[<ffffffff802596c8>] [<ffffffff802596c8>] iov_iter_advance+0x38/0x70
> [ 4057.310000] RSP: 0018:ffff810110329b20 EFLAGS: 00010246
> [ 4057.310000] RAX: 0000000000000000 RBX: 0000000000000800 RCX: 0000000000000000
> [ 4057.310000] RDX: 0000000000000000 RSI: 0000000000000800 RDI: ffff810110329ba8
> [ 4057.310000] RBP: 0000000000000800 R08: 0000000000000000 R09: ffff810101dbc000
> [ 4057.310000] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000026000
> [ 4057.310000] R13: ffff81010d765c98 R14: 0000000000001000 R15: 0000000000000000
> [ 4057.310000] FS: 00007fee589146d0(0000) GS:ffffffff80501000(0000) knlGS:0000000000000000
> [ 4057.310000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 4057.310000] CR2: ffff810101dbc008 CR3: 00000001103da000 CR4: 00000000000006e0
> [ 4057.310000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 4057.310000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 4057.310000] Process ftest03 (pid: 7035, threadinfo ffff810110328000, task ffff810160b00000)
> [ 4057.310000] Stack: ffffffff8025b413 ffff81010d765ab0 ffffffff804e6fd8 00000000001201d2
> [ 4057.310000] ffff810110329db8 0000000000026000 ffff810110329d38 ffff81017b9fb500
> [ 4057.310000] ffff81010d765c98 ffffffff804175e0 ffff81010d765ab0 0000000000000000
> [ 4057.310000] Call Trace:
> [ 4057.310000] [<ffffffff8025b413>] ? generic_file_buffered_write+0x1e3/0x6f0
> [ 4057.310000] [<ffffffff8022f4ae>] ? current_fs_time+0x1e/0x30
> [ 4057.310000] [<ffffffff8025bbaf>] ? __generic_file_aio_write_nolock+0x28f/0x440
> [ 4057.310000] [<ffffffff8025bdc3>] ? generic_file_aio_write+0x63/0xd0
> [ 4057.310000] [<ffffffff802bfbc3>] ? ext3_file_write+0x23/0xc0
> [ 4057.310000] [<ffffffff802bfba0>] ? ext3_file_write+0x0/0xc0
> [ 4057.310000] [<ffffffff8027ebeb>] ? do_sync_readv_writev+0xcb/0x110
> [ 4057.310000] [<ffffffff8023f2b0>] ? autoremove_wake_function+0x0/0x30
> [ 4057.310000] [<ffffffff8024be8d>] ? debug_check_no_locks_freed+0x7d/0x130
> [ 4057.310000] [<ffffffff8024bd8f>] ? trace_hardirqs_on+0xcf/0x150
> [ 4057.310000] [<ffffffff8027c265>] ? __kmalloc+0x15/0xc0
> [ 4057.310000] [<ffffffff8027ea4d>] ? rw_copy_check_uvector+0x9d/0x130
> [ 4057.310000] [<ffffffff8027f330>] ? do_readv_writev+0xe0/0x170
> [ 4057.310000] [<ffffffff803fb5e7>] ? mutex_lock_nested+0x1a7/0x280
> [ 4057.310000] [<ffffffff8024bd8f>] ? trace_hardirqs_on+0xcf/0x150
> [ 4057.310000] [<ffffffff803fb389>] ? __mutex_unlock_slowpath+0xc9/0x170
> [ 4057.310000] [<ffffffff8024bd8f>] ? trace_hardirqs_on+0xcf/0x150
> [ 4057.310000] [<ffffffff803fc6ba>] ? trace_hardirqs_on_thunk+0x35/0x3a
> [ 4057.310000] [<ffffffff8027f7c3>] ? sys_writev+0x53/0x90
> [ 4057.310000] [<ffffffff8020b39b>] ? system_call_after_swapgs+0x7b/0x80
> [ 4057.310000]
> [ 4057.310000]
> [ 4057.310000] Code: 48 01 77 10 48 29 77 18 c3 0f 0b eb fe 66 66 90 66 66 90 4c 8b 0f 48 8b 4f 10 49 89 f0 eb 07 66 66 66 90 49 29 c0 4d 85 c0 75 07 <49> 83 79 08 00 75 23 49 8b 51 08 48 89 d0 48 29 c8 49 39 c0 49
> [ 4057.310000] RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70
> [ 4057.310000] RSP <ffff810110329b20>
> [ 4057.310000] CR2: ffff810101dbc008
> [ 4057.310000] Kernel panic - not syncing: Fatal exception

because this has me stumped. It might need bisecting, please.

Nick, can you think of anything around there which we changed?

2008-02-12 22:27:22

by Alexey Dobriyan

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Tue, Feb 12, 2008 at 02:04:30PM -0800, Andrew Morton wrote:
> On Sun, 10 Feb 2008 17:00:31 +0300
> Alexey Dobriyan <[email protected]> wrote:
>
> > This happened during LTP. FWIW, modprobe/rmmod trivial empty module
> > together with cat /proc/*/wchan and cat /proc/modules were also running.
> >
> > Box is E6400, much debugging is on, config below.
> >
> >
> > [ 4057.310000] BUG: unable to handle kernel paging request at ffff810101dbc008
> > [ 4057.310000] IP: [<ffffffff802596c8>] iov_iter_advance+0x38/0x70
> > [ 4057.310000] PGD 8063 PUD c063 PMD 153baa163 PTE 8000000101dbc160
> > [ 4057.310000] Oops: 0000 [1] SMP DEBUG_PAGEALLOC
> > [ 4057.310000] CPU 0
> > [ 4057.310000] Modules linked in: [last unloaded: foo]
>
> what is this foo.ko of which you speak, and did it wreck your kernel?

It's a trivial dumb module which does nothing but loads and unloads.
I redid ftest03 later without any suspicious activity and it oopsed the
same way. Don't worry about it.

> > [ 4057.310000] Pid: 7035, comm: ftest03 Not tainted 2.6.24-25f666300625d894ebe04bac2b4b3aadb907c861 #2
> > [ 4057.310000] RIP: 0010:[<ffffffff802596c8>] [<ffffffff802596c8>] iov_iter_advance+0x38/0x70
> > [ 4057.310000] RSP: 0018:ffff810110329b20 EFLAGS: 00010246
> > [ 4057.310000] RAX: 0000000000000000 RBX: 0000000000000800 RCX: 0000000000000000
> > [ 4057.310000] RDX: 0000000000000000 RSI: 0000000000000800 RDI: ffff810110329ba8
> > [ 4057.310000] RBP: 0000000000000800 R08: 0000000000000000 R09: ffff810101dbc000
> > [ 4057.310000] R10: 0000000000000004 R11: 0000000000000000 R12: 0000000000026000
> > [ 4057.310000] R13: ffff81010d765c98 R14: 0000000000001000 R15: 0000000000000000
> > [ 4057.310000] FS: 00007fee589146d0(0000) GS:ffffffff80501000(0000) knlGS:0000000000000000
> > [ 4057.310000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> > [ 4057.310000] CR2: ffff810101dbc008 CR3: 00000001103da000 CR4: 00000000000006e0
> > [ 4057.310000] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> > [ 4057.310000] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> > [ 4057.310000] Process ftest03 (pid: 7035, threadinfo ffff810110328000, task ffff810160b00000)
> > [ 4057.310000] Stack: ffffffff8025b413 ffff81010d765ab0 ffffffff804e6fd8 00000000001201d2
> > [ 4057.310000] ffff810110329db8 0000000000026000 ffff810110329d38 ffff81017b9fb500
> > [ 4057.310000] ffff81010d765c98 ffffffff804175e0 ffff81010d765ab0 0000000000000000
> > [ 4057.310000] Call Trace:
> > [ 4057.310000] [<ffffffff8025b413>] ? generic_file_buffered_write+0x1e3/0x6f0
> > [ 4057.310000] [<ffffffff8022f4ae>] ? current_fs_time+0x1e/0x30
> > [ 4057.310000] [<ffffffff8025bbaf>] ? __generic_file_aio_write_nolock+0x28f/0x440
> > [ 4057.310000] [<ffffffff8025bdc3>] ? generic_file_aio_write+0x63/0xd0
> > [ 4057.310000] [<ffffffff802bfbc3>] ? ext3_file_write+0x23/0xc0
> > [ 4057.310000] [<ffffffff802bfba0>] ? ext3_file_write+0x0/0xc0
> > [ 4057.310000] [<ffffffff8027ebeb>] ? do_sync_readv_writev+0xcb/0x110
> > [ 4057.310000] [<ffffffff8023f2b0>] ? autoremove_wake_function+0x0/0x30
> > [ 4057.310000] [<ffffffff8024be8d>] ? debug_check_no_locks_freed+0x7d/0x130
> > [ 4057.310000] [<ffffffff8024bd8f>] ? trace_hardirqs_on+0xcf/0x150
> > [ 4057.310000] [<ffffffff8027c265>] ? __kmalloc+0x15/0xc0
> > [ 4057.310000] [<ffffffff8027ea4d>] ? rw_copy_check_uvector+0x9d/0x130
> > [ 4057.310000] [<ffffffff8027f330>] ? do_readv_writev+0xe0/0x170
> > [ 4057.310000] [<ffffffff803fb5e7>] ? mutex_lock_nested+0x1a7/0x280
> > [ 4057.310000] [<ffffffff8024bd8f>] ? trace_hardirqs_on+0xcf/0x150
> > [ 4057.310000] [<ffffffff803fb389>] ? __mutex_unlock_slowpath+0xc9/0x170
> > [ 4057.310000] [<ffffffff8024bd8f>] ? trace_hardirqs_on+0xcf/0x150
> > [ 4057.310000] [<ffffffff803fc6ba>] ? trace_hardirqs_on_thunk+0x35/0x3a
> > [ 4057.310000] [<ffffffff8027f7c3>] ? sys_writev+0x53/0x90
> > [ 4057.310000] [<ffffffff8020b39b>] ? system_call_after_swapgs+0x7b/0x80
> > [ 4057.310000]
> > [ 4057.310000]
> > [ 4057.310000] Code: 48 01 77 10 48 29 77 18 c3 0f 0b eb fe 66 66 90 66 66 90 4c 8b 0f 48 8b 4f 10 49 89 f0 eb 07 66 66 66 90 49 29 c0 4d 85 c0 75 07 <49> 83 79 08 00 75 23 49 8b 51 08 48 89 d0 48 29 c8 49 39 c0 49
> > [ 4057.310000] RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70
> > [ 4057.310000] RSP <ffff810110329b20>
> > [ 4057.310000] CR2: ffff810101dbc008
> > [ 4057.310000] Kernel panic - not syncing: Fatal exception
>
> because this has me stumped. It might need bisecting, please.

OK.

> Nick, can you think of anything around there which we changed?

2008-02-13 00:18:18

by Nick Piggin

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Wednesday 13 February 2008 09:27, Alexey Dobriyan wrote:
> On Tue, Feb 12, 2008 at 02:04:30PM -0800, Andrew Morton wrote:
> > On Sun, 10 Feb 2008 17:00:31 +0300
> >
> > Alexey Dobriyan <[email protected]> wrote:
> > > This happened during LTP. FWIW, modprobe/rmmod trivial empty module
> > > together with cat /proc/*/wchan and cat /proc/modules were also
> > > running.
> > >
> > > Box is E6400, much debugging is on, config below.
> > >
> > >
> > > [ 4057.310000] BUG: unable to handle kernel paging request at
> > > ffff810101dbc008 [ 4057.310000] IP: [<ffffffff802596c8>]
> > > iov_iter_advance+0x38/0x70 [ 4057.310000] PGD 8063 PUD c063 PMD
> > > 153baa163 PTE 8000000101dbc160 [ 4057.310000] Oops: 0000 [1] SMP
> > > DEBUG_PAGEALLOC
> > > [ 4057.310000] CPU 0
> > > [ 4057.310000] Modules linked in: [last unloaded: foo]
> >
> > what is this foo.ko of which you speak, and did it wreck your kernel?
>
> It's a trivial dumb module which does nothing but loads and unloads.
> I redid ftest03 later without any suspicious activity and it oopsed the
> same way.

Ah crap. Hmm, maybe I didn't consider all cases with my last patch to
that code... is there an easy way to get the ftest03 source and run
it?

2008-02-13 04:42:25

by Nick Piggin

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Wednesday 13 February 2008 11:17, Nick Piggin wrote:
> On Wednesday 13 February 2008 09:27, Alexey Dobriyan wrote:

> > It's a trivial dumb module which does nothing but loads and unloads.
> > I redid ftest03 later without any suspicious activity and it oopsed the
> > same way.
>
> Ah crap. Hmm, maybe I didn't consider all cases with my last patch to
> that code... is there an easy way to get the ftest03 source and run
> it?

OK I didn't realise it is a test from ltp.

But I can't reproduce it for the life of me with the latest git kernel
and latest ltp tarball.

Is it easy to reproduce? Are you reproducing it simply by running the
ftest03 binary directly from the shell? How many times between oopses?
It is multi-process but no threads, so races should be minimal down
this path -- can you get an strace of the failing process?

Thanks,
Nick

2008-02-19 04:07:36

by Nick Piggin

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Wednesday 13 February 2008 09:27, Alexey Dobriyan wrote:
> On Tue, Feb 12, 2008 at 02:04:30PM -0800, Andrew Morton wrote:

> > > [ 4057.310000] Pid: 7035, comm: ftest03 Not tainted
> > > 2.6.24-25f666300625d894ebe04bac2b4b3aadb907c861 #2 [ 4057.310000] RIP:
> > > 0010:[<ffffffff802596c8>] [<ffffffff802596c8>]
> > > iov_iter_advance+0x38/0x70 [ 4057.310000] RSP: 0018:ffff810110329b20
> > > EFLAGS: 00010246
> > > [ 4057.310000] RAX: 0000000000000000 RBX: 0000000000000800 RCX:
> > > 0000000000000000 [ 4057.310000] RDX: 0000000000000000 RSI:
> > > 0000000000000800 RDI: ffff810110329ba8 [ 4057.310000] RBP:
> > > 0000000000000800 R08: 0000000000000000 R09: ffff810101dbc000 [
> > > 4057.310000] R10: 0000000000000004 R11: 0000000000000000 R12:
> > > 0000000000026000 [ 4057.310000] R13: ffff81010d765c98 R14:
> > > 0000000000001000 R15: 0000000000000000 [ 4057.310000] FS:
> > > 00007fee589146d0(0000) GS:ffffffff80501000(0000) knlGS:0000000000000000
> > > [ 4057.310000] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b [
> > > 4057.310000] CR2: ffff810101dbc008 CR3: 00000001103da000 CR4:
> > > 00000000000006e0 [ 4057.310000] DR0: 0000000000000000 DR1:
> > > 0000000000000000 DR2: 0000000000000000 [ 4057.310000] DR3:
> > > 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 [
> > > 4057.310000] Process ftest03 (pid: 7035, threadinfo ffff810110328000,
> > > task ffff810160b00000) [ 4057.310000] Stack: ffffffff8025b413
> > > ffff81010d765ab0 ffffffff804e6fd8 00000000001201d2 [ 4057.310000]
> > > ffff810110329db8 0000000000026000 ffff810110329d38 ffff81017b9fb500 [
> > > 4057.310000] ffff81010d765c98 ffffffff804175e0 ffff81010d765ab0
> > > 0000000000000000 [ 4057.310000] Call Trace:
> > > [ 4057.310000] [<ffffffff8025b413>] ?
> > > generic_file_buffered_write+0x1e3/0x6f0 [ 4057.310000]
> > > [<ffffffff8022f4ae>] ? current_fs_time+0x1e/0x30 [ 4057.310000]
> > > [<ffffffff8025bbaf>] ? __generic_file_aio_write_nolock+0x28f/0x440 [
> > > 4057.310000] [<ffffffff8025bdc3>] ? generic_file_aio_write+0x63/0xd0 [
> > > 4057.310000] [<ffffffff802bfbc3>] ? ext3_file_write+0x23/0xc0 [
> > > 4057.310000] [<ffffffff802bfba0>] ? ext3_file_write+0x0/0xc0 [
> > > 4057.310000] [<ffffffff8027ebeb>] ? do_sync_readv_writev+0xcb/0x110 [
> > > 4057.310000] [<ffffffff8023f2b0>] ? autoremove_wake_function+0x0/0x30
> > > [ 4057.310000] [<ffffffff8024be8d>] ?
> > > debug_check_no_locks_freed+0x7d/0x130 [ 4057.310000]
> > > [<ffffffff8024bd8f>] ? trace_hardirqs_on+0xcf/0x150 [ 4057.310000]
> > > [<ffffffff8027c265>] ? __kmalloc+0x15/0xc0
> > > [ 4057.310000] [<ffffffff8027ea4d>] ? rw_copy_check_uvector+0x9d/0x130
> > > [ 4057.310000] [<ffffffff8027f330>] ? do_readv_writev+0xe0/0x170
> > > [ 4057.310000] [<ffffffff803fb5e7>] ? mutex_lock_nested+0x1a7/0x280
> > > [ 4057.310000] [<ffffffff8024bd8f>] ? trace_hardirqs_on+0xcf/0x150
> > > [ 4057.310000] [<ffffffff803fb389>] ?
> > > __mutex_unlock_slowpath+0xc9/0x170 [ 4057.310000] [<ffffffff8024bd8f>]
> > > ? trace_hardirqs_on+0xcf/0x150 [ 4057.310000] [<ffffffff803fc6ba>] ?
> > > trace_hardirqs_on_thunk+0x35/0x3a [ 4057.310000] [<ffffffff8027f7c3>]
> > > ? sys_writev+0x53/0x90
> > > [ 4057.310000] [<ffffffff8020b39b>] ?
> > > system_call_after_swapgs+0x7b/0x80 [ 4057.310000]
> > > [ 4057.310000]
> > > [ 4057.310000] Code: 48 01 77 10 48 29 77 18 c3 0f 0b eb fe 66 66 90 66
> > > 66 90 4c 8b 0f 48 8b 4f 10 49 89 f0 eb 07 66 66 66 90 49 29 c0 4d 85 c0
> > > 75 07 <49> 83 79 08 00 75 23 49 8b 51 08 48 89 d0 48 29 c8 49 39 c0 49
> > > [ 4057.310000] RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70 [
> > > 4057.310000] RSP <ffff810110329b20>
> > > [ 4057.310000] CR2: ffff810101dbc008
> > > [ 4057.310000] Kernel panic - not syncing: Fatal exception

Can you try this patch please?


Attachments:
(No filename) (3.70 kB)
mm-iov-fix.patch (608.00 B)
Download all attachments

2008-02-19 19:22:44

by Kevin Coffman

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Feb 18, 2008 8:20 PM, Nick Piggin <[email protected]> wrote:
> On Wednesday 13 February 2008 09:27, Alexey Dobriyan wrote:
> > On Tue, Feb 12, 2008 at 02:04:30PM -0800, Andrew Morton wrote:
>
> Can you try this patch please?

Sorry, I still see (only with DEBUG_PAGEALLOC enabled):

kernel: BUG: unable to handle kernel paging request at ffff81006a509008
kernel: IP: [<ffffffff80268313>] iov_iter_advance+0x4f/0x67
kernel: PGD 8063 PUD a063 PMD 6a400163 PTE 800000006a509160
kernel: Oops: 0000 [1] SMP DEBUG_PAGEALLOC
kernel: CPU 1
kernel: Modules linked in: i915 drm nfsd exportfs autofs4 hidp l2cap
bluetooth cbc crypto_blkcipher nfs lockd nfs_acl rpcsec_gss_krb5
auth_rpcgss des_generic sunrpc ipv6 nf_conntrack_netbios_ns ipt_REJECT
nf_conntrack_ipv4 xt_state nf_conntrack xt_tcpudp iptable_filter
ip_tables x_tables cpufreq_ondemand acpi_cpufreq dm_mirror
dm_multipath dm_mod video output sbs sbshc dock battery ac
snd_hda_intel snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq
snd_seq_device snd_pcm_oss snd_mixer_oss sr_mod snd_pcm snd_timer
cdrom snd serio_raw button ohci1394 ieee1394 pcspkr i2c_i801 i2c_core
soundcore snd_page_alloc r8169 iTCO_wdt iTCO_vendor_support sg ahci
ata_piix libata sd_mod scsi_mod ext3 jbd ehci_hcd ohci_hcd uhci_hcd
[last unloaded: scsi_wait_scan]
kernel: Pid: 2299, comm: nfsd Not tainted 2.6.25-rc1 #46
kernel: RIP: 0010:[<ffffffff80268313>] [<ffffffff80268313>]
iov_iter_advance+0x4f/0x67
kernel: RSP: 0018:ffff810073c4d8a8 EFLAGS: 00010246
kernel: RAX: 0000000000000000 RBX: 0000000000001000 RCX: 0000000000000000
kernel: RDX: 0000000000000000 RSI: 0000000000001000 RDI: ffff810073c4d940
kernel: RBP: 0000000000001000 R08: 0000000000000000 R09: ffff81006a509000
kernel: R10: 0000000000001000 R11: 6a6a6a6a6a6a6a6a R12: 0000000000ef7000
kernel: R13: 0000000000001000 R14: ffff810073c4d940 R15: 0000000000007000
kernel: FS: 00007f1898ea86f0(0000) GS:ffff81007e3b99c0(0000)
knlGS:0000000000000000
kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
kernel: CR2: ffff81006a509008 CR3: 000000007b1e2000 CR4: 00000000000006e0
kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
kernel: Process nfsd (pid: 2299, threadinfo ffff810073c4c000, task
ffff81007c62f500)
kernel: Stack: ffffffff8026999b ffff8100698a9500 ffff810073c4d970
ffff81006a5b65b0
kernel: ffff8100698a9658 0000000100000014 ffff810073c4db50 0000000000ef0000
kernel: ffff810073c4dad0 ffff81007d84fe80 ffff8100698a9658 ffffffff88042bc0
kernel: Call Trace:
kernel: [<ffffffff8026999b>] ? generic_file_buffered_write+0x1dc/0x631
kernel: [<ffffffff80237fcb>] ? current_fs_time+0x1e/0x24
kernel: [<ffffffff80222ec7>] ? __change_page_attr_set_clr+0x57/0xaf
kernel: [<ffffffff8026a12f>] ? __generic_file_aio_write_nolock+0x33f/0x3a9
kernel: [<ffffffff8026a1fa>] ? generic_file_aio_write+0x61/0xc1
kernel: [<ffffffff880322fa>] ? :ext3:ext3_file_write+0x16/0x94
kernel: [<ffffffff880322e4>] ? :ext3:ext3_file_write+0x0/0x94
kernel: [<ffffffff80290443>] ? do_sync_readv_writev+0xc0/0x107
kernel: [<ffffffff802454b6>] ? autoremove_wake_function+0x0/0x2e
kernel: [<ffffffff802902d9>] ? rw_copy_check_uvector+0x6c/0xdc
kernel: [<ffffffff80290af6>] ? do_readv_writev+0xb2/0x18b
kernel: [<ffffffff8841048a>] ? :exportfs:find_acceptable_alias+0x1a/0xe2
kernel: [<ffffffff884105fb>] ? :exportfs:exportfs_decode_fh+0xa9/0x1de
kernel: [<ffffffff88417c2d>] ? :nfsd:nfsd_acceptable+0x0/0xd3
kernel: [<ffffffff88418bf6>] ? :nfsd:nfsd_vfs_write+0xe9/0x2dd
kernel: [<ffffffff8023fad7>] ? set_current_groups+0x156/0x161
kernel: [<ffffffff8841dfc1>] ? :nfsd:nfsd_setuser+0x116/0x201
kernel: [<ffffffff88419483>] ? :nfsd:nfsd_write+0x79/0xe2
kernel: [<ffffffff8842329a>] ? :nfsd:nfsd4_write+0xd0/0xfc
kernel: [<ffffffff88423833>] ? :nfsd:nfsd4_proc_compound+0x1ad/0x2b9
kernel: [<ffffffff88415245>] ? :nfsd:nfsd_dispatch+0xde/0x1b6
kernel: [<ffffffff8832b25e>] ? :sunrpc:svc_process+0x3f0/0x6e2
kernel: [<ffffffff8841567c>] ? :nfsd:nfsd+0x0/0x2c0
kernel: [<ffffffff8841581f>] ? :nfsd:nfsd+0x1a3/0x2c0
kernel: [<ffffffff8022f1d1>] ? schedule_tail+0x28/0x5d
kernel: [<ffffffff8020cc18>] ? child_rip+0xa/0x12
kernel: [<ffffffff8841567c>] ? :nfsd:nfsd+0x0/0x2c0
last message repeated 2 times
kernel: [<ffffffff8020cc0e>] ? child_rip+0x0/0x12
kernel:
kernel:
kernel: Code: eb 24 49 8b 51 08 48 89 d0 48 29 c8 4c 39 c0 49 0f 47 c0
48 98 48 01 c1 48 39 ca 75 06 49 83 c1 10 31 c9 49 29 c0 4d 85 c0 75
d7 <49> 83 79 08 00 75 05 4d 85 d2 75 cb 4c 89 0f 48 89 4f 10 48 29
kernel: RIP [<ffffffff80268313>] iov_iter_advance+0x4f/0x67
kernel: RSP <ffff810073c4d8a8>
kernel: CR2: ffff81006a509008
kernel: ---[ end trace 0ce02d00f5455fd8 ]---

2008-02-19 20:47:37

by Alexey Dobriyan

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Wed, Feb 13, 2008 at 03:42:02PM +1100, Nick Piggin wrote:
> On Wednesday 13 February 2008 11:17, Nick Piggin wrote:
> > On Wednesday 13 February 2008 09:27, Alexey Dobriyan wrote:
>
> > > It's a trivial dumb module which does nothing but loads and unloads.
> > > I redid ftest03 later without any suspicious activity and it oopsed the
> > > same way.
> >
> > Ah crap. Hmm, maybe I didn't consider all cases with my last patch to
> > that code... is there an easy way to get the ftest03 source and run
> > it?
>
> OK I didn't realise it is a test from ltp.
>
> But I can't reproduce it for the life of me with the latest git kernel
> and latest ltp tarball.
>
> Is it easy to reproduce?

Well, yes. SMP, non-preemptible kernel, (and maxcpus=1 really helps!)

while true; do
./ftest03
done

This alone seems stable, but starting whole LTP in parallel downs the box
very quickly.

> Are you reproducing it simply by running the
> ftest03 binary directly from the shell? How many times between oopses?
> It is multi-process but no threads, so races should be minimal down
> this path -- can you get an strace of the failing process?

so far it generates too many output, trying to reduce testcase.

Patch doesn't help here, too.

2008-02-19 22:01:30

by Alexey Dobriyan

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Tue, Feb 19, 2008 at 11:47:11PM +0300, wrote:
> On Wed, Feb 13, 2008 at 03:42:02PM +1100, Nick Piggin wrote:
> > On Wednesday 13 February 2008 11:17, Nick Piggin wrote:
> > > On Wednesday 13 February 2008 09:27, Alexey Dobriyan wrote:
> >
> > > > It's a trivial dumb module which does nothing but loads and unloads.
> > > > I redid ftest03 later without any suspicious activity and it oopsed the
> > > > same way.
> > >
> > > Ah crap. Hmm, maybe I didn't consider all cases with my last patch to
> > > that code... is there an easy way to get the ftest03 source and run
> > > it?
> >
> > OK I didn't realise it is a test from ltp.
> >
> > But I can't reproduce it for the life of me with the latest git kernel
> > and latest ltp tarball.
> >
> > Is it easy to reproduce?
>
> Well, yes. SMP, non-preemptible kernel, (and maxcpus=1 really helps!)
>
> while true; do
> ./ftest03
> done
>
> This alone seems stable, but starting whole LTP in parallel downs the box
> very quickly.
>
> > Are you reproducing it simply by running the
> > ftest03 binary directly from the shell? How many times between oopses?
> > It is multi-process but no threads, so races should be minimal down
> > this path -- can you get an strace of the failing process?

Speaking of multi-proceseness, changing MAXCHILD to 1, nchild to 1,
AFAICS, generates one child which oopses the very same way (in parallel
with generic LTP) But, lowering MAXIOVCNT to 8 generates no oops.

2008-02-26 03:47:20

by Nick Piggin

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Wednesday 20 February 2008 09:01, Alexey Dobriyan wrote:
> On Tue, Feb 19, 2008 at 11:47:11PM +0300, wrote:

> > > Are you reproducing it simply by running the
> > > ftest03 binary directly from the shell? How many times between oopses?
> > > It is multi-process but no threads, so races should be minimal down
> > > this path -- can you get an strace of the failing process?
>
> Speaking of multi-proceseness, changing MAXCHILD to 1, nchild to 1,
> AFAICS, generates one child which oopses the very same way (in parallel
> with generic LTP) But, lowering MAXIOVCNT to 8 generates no oops.

Thanks, I was able to reproduce quite easily with these settings.
I think I have the correct patch now (at least it isn't triggerable
any more here).

Thanks,
Nick


Attachments:
(No filename) (759.00 B)
iov-iter-fix.patch (780.00 B)
Download all attachments

2008-02-27 17:42:21

by Kevin Coffman

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Mon, Feb 25, 2008 at 10:46 PM, Nick Piggin <[email protected]> wrote:
> On Wednesday 20 February 2008 09:01, Alexey Dobriyan wrote:
>
> > On Tue, Feb 19, 2008 at 11:47:11PM +0300, wrote:
>
> > > > Are you reproducing it simply by running the
> > > > ftest03 binary directly from the shell? How many times between oopses?
> > > > It is multi-process but no threads, so races should be minimal down
> > > > this path -- can you get an strace of the failing process?
> >
> > Speaking of multi-proceseness, changing MAXCHILD to 1, nchild to 1,
> > AFAICS, generates one child which oopses the very same way (in parallel
> > with generic LTP) But, lowering MAXIOVCNT to 8 generates no oops.
>
> Thanks, I was able to reproduce quite easily with these settings.
> I think I have the correct patch now (at least it isn't triggerable
> any more here).
>
> Thanks,
> Nick

Hi Nick,
With this patch, I'm getting "bad data" errors while running the
connectathon tests from a Solaris NFS client to my Linux server. (The
failing test writes a "big" file (1MB) and reads it back, verifying
the data read back is correct.) Let me know if I can provide more
details.

K.C.

2008-03-03 04:45:44

by Nick Piggin

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Thursday 28 February 2008 04:41, Kevin Coffman wrote:
> On Mon, Feb 25, 2008 at 10:46 PM, Nick Piggin <[email protected]>
wrote:
> > On Wednesday 20 February 2008 09:01, Alexey Dobriyan wrote:
> > > On Tue, Feb 19, 2008 at 11:47:11PM +0300, wrote:
> > > > > Are you reproducing it simply by running the
> > > > > ftest03 binary directly from the shell? How many times between
> > > > > oopses? It is multi-process but no threads, so races should be
> > > > > minimal down this path -- can you get an strace of the failing
> > > > > process?
> > >
> > > Speaking of multi-proceseness, changing MAXCHILD to 1, nchild to 1,
> > > AFAICS, generates one child which oopses the very same way (in
> > > parallel with generic LTP) But, lowering MAXIOVCNT to 8 generates no
> > > oops.
> >
> > Thanks, I was able to reproduce quite easily with these settings.
> > I think I have the correct patch now (at least it isn't triggerable
> > any more here).
> >
> > Thanks,
> > Nick
>
> Hi Nick,
> With this patch, I'm getting "bad data" errors while running the
> connectathon tests from a Solaris NFS client to my Linux server. (The
> failing test writes a "big" file (1MB) and reads it back, verifying
> the data read back is correct.) Let me know if I can provide more
> details.

Hmm, thanks for testing... Could it be caused by other bugs introduced
in the kernel you are testing? Is the failing test using writev to
write the file? Do you have any indication of the pattern of corruption?

Thanks,
Nick

2008-03-03 13:31:11

by Kevin Coffman

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Sun, Mar 2, 2008 at 11:38 PM, Nick Piggin <[email protected]> wrote:
>
> On Thursday 28 February 2008 04:41, Kevin Coffman wrote:
> > On Mon, Feb 25, 2008 at 10:46 PM, Nick Piggin <[email protected]>
> wrote:
> > > On Wednesday 20 February 2008 09:01, Alexey Dobriyan wrote:
> > > > On Tue, Feb 19, 2008 at 11:47:11PM +0300, wrote:
> > > > > > Are you reproducing it simply by running the
> > > > > > ftest03 binary directly from the shell? How many times between
> > > > > > oopses? It is multi-process but no threads, so races should be
> > > > > > minimal down this path -- can you get an strace of the failing
> > > > > > process?
> > > >
> > > > Speaking of multi-proceseness, changing MAXCHILD to 1, nchild to 1,
> > > > AFAICS, generates one child which oopses the very same way (in
> > > > parallel with generic LTP) But, lowering MAXIOVCNT to 8 generates no
> > > > oops.
> > >
> > > Thanks, I was able to reproduce quite easily with these settings.
> > > I think I have the correct patch now (at least it isn't triggerable
> > > any more here).
> > >
> > > Thanks,
> > > Nick
> >
> > Hi Nick,
> > With this patch, I'm getting "bad data" errors while running the
> > connectathon tests from a Solaris NFS client to my Linux server. (The
> > failing test writes a "big" file (1MB) and reads it back, verifying
> > the data read back is correct.) Let me know if I can provide more
> > details.
>
> Hmm, thanks for testing... Could it be caused by other bugs introduced
> in the kernel you are testing? Is the failing test using writev to
> write the file? Do you have any indication of the pattern of corruption?
>

Well, all I did was apply your patch and re-run the same tests I've
been running successfully. (I'm still sitting at 2.6.25-rc1 right
now). Sorry, I don't have details about the corruption. I'm home
sick today, but I'll try to get more details when I get back in the
office.

K.C.

2008-03-03 16:41:50

by Nick Piggin

[permalink] [raw]
Subject: Re: 2.6.24-sha1: RIP [<ffffffff802596c8>] iov_iter_advance+0x38/0x70

On Tuesday 04 March 2008 00:30, Kevin Coffman wrote:
> On Sun, Mar 2, 2008 at 11:38 PM, Nick Piggin <[email protected]>
> > Hmm, thanks for testing... Could it be caused by other bugs introduced
> > in the kernel you are testing? Is the failing test using writev to
> > write the file? Do you have any indication of the pattern of corruption?
>
> Well, all I did was apply your patch and re-run the same tests I've
> been running successfully. (I'm still sitting at 2.6.25-rc1 right
> now). Sorry, I don't have details about the corruption. I'm home
> sick today, but I'll try to get more details when I get back in the
> office.

OK my last patch had a silly thinko in it, sorry. I'll improve it
and retest it before asking you to do any more. Thanks so far..