2011-03-24 13:43:45

by Jens Axboe

[permalink] [raw]
Subject: [GIT PULL] Core block IO bits for 2.6.39

Hi Linus,

This is the main pull request for the block IO layer and friends for
2.6.39.

There are two major things in this tree:

- The removal of the per-device plugging state for disks. On fast
devices, it ended up hammering the queue lock quite hard. The new
scheme puts the plugging state on the stack and allows an IO submitter
to finish his batch of IO before pushing it to the queue. Once that
push starts, we'll insert/merge with the existing queue.

A pointer to this plugging context is stored in the task structure. If
a task ends up blocking before it has submitted it's IO (usual cause
would be memory allocation of some sort), the plugged list is
auto-submitted before the task goes to sleep.

While reducing the queue lock frequency, this patch also provides the
nice benefit of getting rid of the aops->sync_page() callback. We used
to use this for auto-unplugging the below device if we needed to wait
on page IO. This is also the reason the diffstat looks so tasty, we
end up removing a lot more lines of code than we add.

Another nice benefit is that the API is now explicit. You call
blk_start_plug() before starting an IO sequence, and blk_finish_plug()
when that sequence is done and you want to flush it out. No more 'hey
I'll plug behind his back, hope he remembers to unplug' games need to
be played.

I did not go overboard with adding plugging calls, so it may very well
be that there are cases where we need to add this during the 2.6.39-rc
cycle. I'd encourage everyone to test their favorite workload and keep
an eye out for regressions.

- Final conversion of drivers to the new ->check_events() interface. So
this work is now complete.

Other notable features/changes:

- Various fixes and improvements to the cfq-ioscheduled.

- Merging of FLUSH/FUA requests to speed up workloads that are intensive
on durable writes.

- Updates and fixes to the block IO throttler.


Note that you'll have to do a trivial merge when pulling this in. I left
that as an exercise for you, since you've expressed interest in seeing
and doing those kinds of merges.

Please pull!

git://git.kernel.dk/linux-2.6-block.git for-2.6.39/core




Dan Carpenter (1):
block: NULL dereference on error path in __blkdev_get()

Gui Jianfeng (1):
cfq-iosched: Fix update_vdisktime logic

Jens Axboe (20):
Merge commit 'v2.6.38-rc6' into for-2.6.39/core
cfq-iosched: fix race in cfq_set_request()
Merge branch 'block-for-2.6.39-core' of ssh://master.kernel.org/.../tj/misc into for-2.6.39/core
block: add API for delaying work/request_fn a little bit
ide-cd: convert to blk_delay_queue() for a short pause
scsi: convert to blk_delay_queue()
block: initial patch for on-stack per-task plugging
block: remove per-queue plugging
fs: make generic file read/write functions plug
read-ahead: use plugging
fs: make mpage read/write_pages() plug
aio: remove request submission batching
block: kill off REQ_UNPLUG
Merge branch 'for-2.6.39/stack-plug' into for-2.6.39/core
block: fixup plugging stubs for !CONFIG_BLOCK
fs: make fsync_buffers_list() plug
jbd: finish conversion from WRITE_SYNC_PLUG to WRITE_SYNC and explicit plugging
jbd2: finish conversion from WRITE_SYNC_PLUG to WRITE_SYNC and explicit plugging
fs: assign sb->s_bdi to default_backing_dev_info if the bdi is going away
block: attempt to merge with existing requests on plug flush

Justin TerAvest (7):
cfq-iosched: Always provide group isolation.
blk-cgroup: Lower minimum weight from 100 to 10.
blk-cgroup: Add unaccounted time to timeslice_used.
cfq-iosched: Don't update group weights when on service tree
cfq-iosched: Don't set active queue in preempt
blk-cgroup: Only give unaccounted_time under debug
cfq-iosched: Don't clear queue stats when preempt.

Li, Shaohua (1):
cfq-iosched: removing unnecessary think time checking

Liu Yuan (1):
block/genhd: Change some numerals into macros

Martin K. Petersen (2):
block: biovec_slab vs. CONFIG_BLK_DEV_INTEGRITY
block: Require subsystems to explicitly allocate bio_set integrity mempool

Mike Snitzer (2):
block: skip elevator data initialization for flush requests
block: share request flush fields with elevator_private

Randy Dunlap (1):
Documentation/iostats.txt: bit-size reference etc.

Shaohua Li (4):
cfq-iosched: give busy sync queue no dispatch limit
fs: make aio plug
mm: make generic_writepages() use plugging
block: fix non-atomic access to genhd inflight structures

Tao Ma (2):
blktrace: Use rq->cmd_flags directly in blk_add_trace_rq.
block: remove obsolete comments for blkdev_issue_zeroout.

Tejun Heo (20):
block: add REQ_FLUSH_SEQ
block: improve flush bio completion
block: reimplement FLUSH/FUA to support merge
Merge branch 'for-linus' of ../linux-2.6-block into block-for-2.6.39/core
block: Don't implicitly trigger event check on disk_unblock_events()
block: Don't check events on close unless it was blocked
block: Don't check events while open is in progress
ide: Convert to bdops->check_events()
floppy,{ami|ata}flop: Convert to bdops->check_events()
gdrom,viocd: Convert to bdops->check_events()
paride: Convert to bdops->check_events()
dac960: Convert to bdops->check_events()
swim[3]: Convert to bdops->check_events()
ub: Convert to bdops->check_events()
xsysace: Convert to bdops->check_events()
i2o_block: Convert to bdops->check_events()
s390/tape_block: Convert to bdops->check_events()
umem: Drop dummy ->media_changed()
pktcdvd: Convert to bdops->check_events()
staging: Convert to bdops->check_events()

Vivek Goyal (7):
block: Initialize ->queue_lock to internal lock at queue allocation time
loop: No need to initialize ->queue_lock explicitly before calling blk_cleanup_queue()
block: Move blk_throtl_exit() call to blk_cleanup_queue()
blk-throttle: process limit change only through one function
blk-throttle: Some cleanups and race fixes in limit update code
blk-throttle: Use blk_plug in throttle dispatch
blk-throttle: Reset group slice when limits are changed

Documentation/block/biodoc.txt | 5 -
Documentation/cgroups/blkio-controller.txt | 30 +-
Documentation/iostats.txt | 17 +-
block/blk-cgroup.c | 16 +-
block/blk-cgroup.h | 14 +-
block/blk-core.c | 646 ++++++++++++--------
block/blk-exec.c | 4 +-
block/blk-flush.c | 439 +++++++++----
block/blk-lib.c | 2 -
block/blk-merge.c | 6 +
block/blk-settings.c | 15 -
block/blk-sysfs.c | 2 -
block/blk-throttle.c | 139 +++--
block/blk.h | 16 +-
block/cfq-iosched.c | 163 +++---
block/cfq.h | 6 +-
block/deadline-iosched.c | 9 -
block/elevator.c | 108 ++--
block/genhd.c | 18 +-
block/noop-iosched.c | 8 -
drivers/block/DAC960.c | 8 +-
drivers/block/amiflop.c | 9 +-
drivers/block/ataflop.c | 14 +-
drivers/block/cciss.c | 6 -
drivers/block/cpqarray.c | 3 -
drivers/block/drbd/drbd_actlog.c | 4 +-
drivers/block/drbd/drbd_bitmap.c | 1 -
drivers/block/drbd/drbd_int.h | 16 +-
drivers/block/drbd/drbd_main.c | 36 +-
drivers/block/drbd/drbd_receiver.c | 29 +-
drivers/block/drbd/drbd_req.c | 4 -
drivers/block/drbd/drbd_worker.c | 1 -
drivers/block/drbd/drbd_wrappers.h | 18 -
drivers/block/floppy.c | 11 +-
drivers/block/loop.c | 16 -
drivers/block/paride/pcd.c | 18 +-
drivers/block/paride/pd.c | 7 +-
drivers/block/paride/pf.c | 10 +-
drivers/block/pktcdvd.c | 15 +-
drivers/block/swim.c | 8 +-
drivers/block/swim3.c | 11 +-
drivers/block/ub.c | 10 +-
drivers/block/umem.c | 26 +-
drivers/block/xsysace.c | 9 +-
drivers/cdrom/gdrom.c | 16 +-
drivers/cdrom/viocd.c | 17 +-
drivers/ide/ide-atapi.c | 3 +-
drivers/ide/ide-cd.c | 23 +-
drivers/ide/ide-cd.h | 3 +-
drivers/ide/ide-cd_ioctl.c | 8 +-
drivers/ide/ide-gd.c | 14 +-
drivers/ide/ide-io.c | 4 -
drivers/ide/ide-park.c | 2 +-
drivers/md/bitmap.c | 5 +-
drivers/md/dm-crypt.c | 9 +-
drivers/md/dm-io.c | 2 +-
drivers/md/dm-kcopyd.c | 55 +--
drivers/md/dm-raid.c | 2 +-
drivers/md/dm-raid1.c | 2 -
drivers/md/dm-table.c | 31 +-
drivers/md/dm.c | 52 +-
drivers/md/dm.h | 2 +-
drivers/md/linear.c | 20 +-
drivers/md/md.c | 20 +-
drivers/md/multipath.c | 38 +-
drivers/md/raid0.c | 19 +-
drivers/md/raid1.c | 91 +---
drivers/md/raid10.c | 97 +---
drivers/md/raid5.c | 63 +--
drivers/md/raid5.h | 2 +-
drivers/message/i2o/i2o_block.c | 17 +-
drivers/mmc/card/queue.c | 3 +-
drivers/s390/block/dasd.c | 2 +-
drivers/s390/char/tape_block.c | 12 +-
drivers/scsi/scsi_lib.c | 44 +-
drivers/scsi/scsi_transport_fc.c | 2 +-
drivers/scsi/scsi_transport_sas.c | 6 +-
drivers/staging/hv/blkvsc_drv.c | 11 +-
.../westbridge/astoria/block/cyasblkdev_block.c | 11 +-
drivers/target/target_core_iblock.c | 7 +-
fs/adfs/inode.c | 1 -
fs/affs/file.c | 2 -
fs/aio.c | 77 +---
fs/befs/linuxvfs.c | 1 -
fs/bfs/file.c | 1 -
fs/bio-integrity.c | 3 +
fs/bio.c | 10 +-
fs/block_dev.c | 27 +-
fs/btrfs/disk-io.c | 79 ---
fs/btrfs/extent_io.c | 2 +-
fs/btrfs/inode.c | 1 -
fs/btrfs/volumes.c | 91 +---
fs/buffer.c | 51 +--
fs/cifs/file.c | 30 -
fs/direct-io.c | 7 +-
fs/efs/inode.c | 1 -
fs/exofs/inode.c | 1 -
fs/ext2/inode.c | 2 -
fs/ext3/inode.c | 3 -
fs/ext4/inode.c | 4 -
fs/ext4/page-io.c | 3 +-
fs/fat/inode.c | 1 -
fs/freevxfs/vxfs_subr.c | 1 -
fs/fuse/inode.c | 1 -
fs/gfs2/aops.c | 3 -
fs/gfs2/log.c | 4 +-
fs/gfs2/lops.c | 12 +-
fs/gfs2/meta_io.c | 3 +-
fs/hfs/inode.c | 2 -
fs/hfsplus/inode.c | 2 -
fs/hpfs/file.c | 1 -
fs/isofs/inode.c | 1 -
fs/jbd/commit.c | 22 +-
fs/jbd2/commit.c | 22 +-
fs/jfs/inode.c | 1 -
fs/jfs/jfs_metapage.c | 1 -
fs/logfs/dev_bdev.c | 2 -
fs/minix/inode.c | 1 -
fs/mpage.c | 8 +
fs/nilfs2/btnode.c | 7 +-
fs/nilfs2/gcinode.c | 1 -
fs/nilfs2/inode.c | 1 -
fs/nilfs2/mdt.c | 9 +-
fs/nilfs2/page.c | 5 +-
fs/nilfs2/page.h | 3 +-
fs/nilfs2/segbuf.c | 2 +-
fs/ntfs/aops.c | 4 -
fs/ntfs/compress.c | 3 +-
fs/ocfs2/aops.c | 1 -
fs/ocfs2/cluster/heartbeat.c | 4 -
fs/omfs/file.c | 1 -
fs/partitions/check.c | 3 +-
fs/qnx4/inode.c | 1 -
fs/reiserfs/inode.c | 1 -
fs/super.c | 2 +
fs/sync.c | 4 +-
fs/sysv/itree.c | 1 -
fs/ubifs/super.c | 1 -
fs/udf/file.c | 1 -
fs/udf/inode.c | 1 -
fs/ufs/inode.c | 1 -
fs/ufs/truncate.c | 2 +-
fs/xfs/linux-2.6/xfs_aops.c | 4 +-
fs/xfs/linux-2.6/xfs_buf.c | 13 +-
include/linux/backing-dev.h | 16 -
include/linux/bio.h | 1 -
include/linux/blk_types.h | 6 +-
include/linux/blkdev.h | 101 +++-
include/linux/buffer_head.h | 1 -
include/linux/device-mapper.h | 5 -
include/linux/elevator.h | 10 +-
include/linux/fs.h | 29 +-
include/linux/genhd.h | 12 +-
include/linux/pagemap.h | 12 -
include/linux/sched.h | 6 +
include/linux/swap.h | 2 -
kernel/exit.c | 1 +
kernel/fork.c | 3 +
kernel/power/block_io.c | 2 +-
kernel/sched.c | 12 +
kernel/trace/blktrace.c | 15 +-
mm/backing-dev.c | 8 +-
mm/filemap.c | 74 +--
mm/memory-failure.c | 8 +-
mm/nommu.c | 4 -
mm/page-writeback.c | 10 +-
mm/page_io.c | 2 +-
mm/readahead.c | 18 +-
mm/shmem.c | 1 -
mm/swap_state.c | 5 +-
mm/swapfile.c | 37 --
mm/vmscan.c | 2 +-
172 files changed, 1520 insertions(+), 2112 deletions(-)

--
Jens Axboe


2011-03-24 18:30:28

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
>
> This is the main pull request for the block IO layer and friends for
> 2.6.39.

This merge results in an early oops on my system (amd64, xfs).
See the attached photo.

--
Markus


Attachments:
(No filename) (232.00 B)
shot.jpg (202.09 kB)
Download all attachments

2011-03-24 18:36:42

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-24 19:30, Markus Trippelsdorf wrote:
> On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
>>
>> This is the main pull request for the block IO layer and friends for
>> 2.6.39.
>
> This merge results in an early oops on my system (amd64, xfs).
> See the attached photo.
>

Auch. Can you ensure that you have CONFIG_DEBUGINFO=y in your .config
and then do:

$ gdb vmlinux
...
l *cfq_insert_request+0x32

and send that output?

--
Jens Axboe

2011-03-24 18:47:54

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.24 at 19:36 +0100, Jens Axboe wrote:
> On 2011-03-24 19:30, Markus Trippelsdorf wrote:
> > On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
> >>
> >> This is the main pull request for the block IO layer and friends for
> >> 2.6.39.
> >
> > This merge results in an early oops on my system (amd64, xfs).
> > See the attached photo.
> >
>
> Auch. Can you ensure that you have CONFIG_DEBUGINFO=y in your .config
> and then do:
>
> $ gdb vmlinux
> ...
> l *cfq_insert_request+0x32
>
> and send that output?

(gdb) l *cfq_insert_request+0x32
0xffffffff81265ad2 is in cfq_insert_request (block/cfq-iosched.c:3396).
3391 {
3392 struct cfq_data *cfqd = q->elevator->elevator_data;
3393 struct cfq_queue *cfqq = RQ_CFQQ(rq);
3394
3395 cfq_log_cfqq(cfqd, cfqq, "insert_request");
3396 cfq_init_prio_data(cfqq, RQ_CIC(rq)->ioc);
3397
3398 rq_set_fifo_time(rq, jiffies + cfqd->cfq_fifo_expire[rq_is_sync(rq)]);
3399 list_add_tail(&rq->queuelist, &cfqq->fifo);
3400 cfq_add_rq_rb(rq);
(gdb)

--
Markus

2011-03-24 18:51:35

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-24 19:36, Jens Axboe wrote:
> On 2011-03-24 19:30, Markus Trippelsdorf wrote:
>> On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
>>>
>>> This is the main pull request for the block IO layer and friends for
>>> 2.6.39.
>>
>> This merge results in an early oops on my system (amd64, xfs).
>> See the attached photo.
>>
>
> Auch. Can you ensure that you have CONFIG_DEBUGINFO=y in your .config
> and then do:
>
> $ gdb vmlinux
> ...
> l *cfq_insert_request+0x32
>
> and send that output?

I took a closer look at the oops, and it most likely looks like q ==
NULL (offset 0x18 == q->elevator). You left out the Code part, so I
can't verify that for certain. Which makes very little sense. I take it
this is 100% reproducible? When you send the gdb output, please also
attach your .config.


--
Jens Axboe

2011-03-24 18:54:50

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.24 at 19:51 +0100, Jens Axboe wrote:
> On 2011-03-24 19:36, Jens Axboe wrote:
> > On 2011-03-24 19:30, Markus Trippelsdorf wrote:
> >> On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
> >>>
> >>> This is the main pull request for the block IO layer and friends for
> >>> 2.6.39.
> >>
> >> This merge results in an early oops on my system (amd64, xfs).
> >> See the attached photo.
> >>
> >
> > Auch. Can you ensure that you have CONFIG_DEBUGINFO=y in your .config
> > and then do:
> >
> > $ gdb vmlinux
> > ...
> > l *cfq_insert_request+0x32
> >
> > and send that output?
>
> I took a closer look at the oops, and it most likely looks like q ==
> NULL (offset 0x18 == q->elevator). You left out the Code part, so I
> can't verify that for certain. Which makes very little sense. I take it
> this is 100% reproducible? When you send the gdb output, please also
> attach your .config.

Yes, it's 100% reproducible here. My .config follows:

#
# Automatically generated make config: don't edit
# Linux/x86_64 2.6.38 Kernel Configuration
# Thu Mar 24 19:41:43 2011
#
CONFIG_64BIT=y
# CONFIG_X86_32 is not set
CONFIG_X86_64=y
CONFIG_X86=y
CONFIG_INSTRUCTION_DECODER=y
CONFIG_OUTPUT_FORMAT="elf64-x86-64"
CONFIG_ARCH_DEFCONFIG="arch/x86/configs/x86_64_defconfig"
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_HAVE_LATENCYTOP_SUPPORT=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_NEED_DMA_MAP_STATE=y
CONFIG_NEED_SG_DMA_LENGTH=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_BUG=y
CONFIG_GENERIC_BUG_RELATIVE_POINTERS=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
# CONFIG_RWSEM_GENERIC_SPINLOCK is not set
CONFIG_RWSEM_XCHGADD_ALGORITHM=y
CONFIG_ARCH_HAS_CPU_IDLE_WAIT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_ARCH_HAS_CPU_RELAX=y
CONFIG_ARCH_HAS_DEFAULT_IDLE=y
CONFIG_ARCH_HAS_CACHE_LINE_SIZE=y
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_NEED_PER_CPU_EMBED_FIRST_CHUNK=y
CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK=y
CONFIG_HAVE_CPUMASK_OF_CPU_MAP=y
CONFIG_ARCH_HIBERNATION_POSSIBLE=y
CONFIG_ARCH_SUSPEND_POSSIBLE=y
CONFIG_ZONE_DMA32=y
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_AUDIT_ARCH=y
CONFIG_ARCH_SUPPORTS_OPTIMIZED_INLINING=y
CONFIG_ARCH_SUPPORTS_DEBUG_PAGEALLOC=y
CONFIG_X86_64_SMP=y
CONFIG_X86_HT=y
CONFIG_ARCH_HWEIGHT_CFLAGS="-fcall-saved-rdi -fcall-saved-rsi -fcall-saved-rdx -fcall-saved-rcx -fcall-saved-r8 -fcall-saved-r9 -fcall-saved-r10 -fcall-saved-r11"
# CONFIG_KTIME_SCALAR is not set
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"
CONFIG_CONSTRUCTORS=y
CONFIG_HAVE_IRQ_WORK=y
CONFIG_IRQ_WORK=y

#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_CROSS_COMPILE=""
CONFIG_LOCALVERSION=""
CONFIG_LOCALVERSION_AUTO=y
CONFIG_HAVE_KERNEL_GZIP=y
CONFIG_HAVE_KERNEL_BZIP2=y
CONFIG_HAVE_KERNEL_LZMA=y
CONFIG_HAVE_KERNEL_XZ=y
CONFIG_HAVE_KERNEL_LZO=y
# CONFIG_KERNEL_GZIP is not set
# CONFIG_KERNEL_BZIP2 is not set
# CONFIG_KERNEL_LZMA is not set
# CONFIG_KERNEL_XZ is not set
CONFIG_KERNEL_LZO=y
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y
CONFIG_BSD_PROCESS_ACCT=y
# CONFIG_BSD_PROCESS_ACCT_V3 is not set
# CONFIG_FHANDLE is not set
# CONFIG_TASKSTATS is not set
# CONFIG_AUDIT is not set
CONFIG_HAVE_GENERIC_HARDIRQS=y

#
# IRQ subsystem
#
CONFIG_GENERIC_HARDIRQS=y
CONFIG_HAVE_SPARSE_IRQ=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_GENERIC_IRQ_SHOW=y
CONFIG_GENERIC_PENDING_IRQ=y
CONFIG_IRQ_FORCED_THREADING=y
# CONFIG_SPARSE_IRQ is not set

#
# RCU Subsystem
#
CONFIG_TREE_PREEMPT_RCU=y
CONFIG_PREEMPT_RCU=y
# CONFIG_RCU_TRACE is not set
CONFIG_RCU_FANOUT=64
# CONFIG_RCU_FANOUT_EXACT is not set
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=16
CONFIG_HAVE_UNSTABLE_SCHED_CLOCK=y
CONFIG_CGROUPS=y
# CONFIG_CGROUP_DEBUG is not set
# CONFIG_CGROUP_NS is not set
# CONFIG_CGROUP_FREEZER is not set
# CONFIG_CGROUP_DEVICE is not set
# CONFIG_CPUSETS is not set
# CONFIG_CGROUP_CPUACCT is not set
# CONFIG_RESOURCE_COUNTERS is not set
# CONFIG_CGROUP_PERF is not set
CONFIG_CGROUP_SCHED=y
CONFIG_FAIR_GROUP_SCHED=y
# CONFIG_RT_GROUP_SCHED is not set
# CONFIG_BLK_CGROUP is not set
# CONFIG_NAMESPACES is not set
# CONFIG_SCHED_AUTOGROUP is not set
# CONFIG_SYSFS_DEPRECATED is not set
# CONFIG_RELAY is not set
# CONFIG_BLK_DEV_INITRD is not set
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SYSCTL=y
CONFIG_ANON_INODES=y
CONFIG_EXPERT=y
CONFIG_EMBEDDED=y
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
# CONFIG_ELF_CORE is not set
# CONFIG_PCSPKR_PLATFORM is not set
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SIGNALFD=y
CONFIG_TIMERFD=y
CONFIG_EVENTFD=y
CONFIG_SHMEM=y
CONFIG_AIO=y
CONFIG_HAVE_PERF_EVENTS=y

#
# Kernel Performance Events And Counters
#
CONFIG_PERF_EVENTS=y
# CONFIG_PERF_COUNTERS is not set
# CONFIG_DEBUG_PERF_USE_VMALLOC is not set
CONFIG_VM_EVENT_COUNTERS=y
CONFIG_PCI_QUIRKS=y
# CONFIG_SLUB_DEBUG is not set
# CONFIG_COMPAT_BRK is not set
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLOB is not set
# CONFIG_PROFILING is not set
CONFIG_TRACEPOINTS=y
CONFIG_HAVE_OPROFILE=y
CONFIG_JUMP_LABEL=y
CONFIG_HAVE_EFFICIENT_UNALIGNED_ACCESS=y
CONFIG_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_IOREMAP_PROT=y
CONFIG_HAVE_KPROBES=y
CONFIG_HAVE_KRETPROBES=y
CONFIG_HAVE_OPTPROBES=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
CONFIG_HAVE_DMA_ATTRS=y
CONFIG_USE_GENERIC_SMP_HELPERS=y
CONFIG_HAVE_REGS_AND_STACK_ACCESS_API=y
CONFIG_HAVE_DMA_API_DEBUG=y
CONFIG_HAVE_HW_BREAKPOINT=y
CONFIG_HAVE_MIXED_BREAKPOINTS_REGS=y
CONFIG_HAVE_USER_RETURN_NOTIFIER=y
CONFIG_HAVE_PERF_EVENTS_NMI=y
CONFIG_HAVE_ARCH_JUMP_LABEL=y

#
# GCOV-based kernel profiling
#
# CONFIG_GCOV_KERNEL is not set
# CONFIG_HAVE_GENERIC_DMA_COHERENT is not set
CONFIG_RT_MUTEXES=y
CONFIG_BASE_SMALL=0
# CONFIG_MODULES is not set
CONFIG_STOP_MACHINE=y
CONFIG_BLOCK=y
CONFIG_BLK_DEV_BSG=y
# CONFIG_BLK_DEV_INTEGRITY is not set
CONFIG_BLOCK_COMPAT=y

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
# CONFIG_IOSCHED_DEADLINE is not set
CONFIG_IOSCHED_CFQ=y
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"
CONFIG_PREEMPT_NOTIFIERS=y
# CONFIG_INLINE_SPIN_TRYLOCK is not set
# CONFIG_INLINE_SPIN_TRYLOCK_BH is not set
# CONFIG_INLINE_SPIN_LOCK is not set
# CONFIG_INLINE_SPIN_LOCK_BH is not set
# CONFIG_INLINE_SPIN_LOCK_IRQ is not set
# CONFIG_INLINE_SPIN_LOCK_IRQSAVE is not set
# CONFIG_INLINE_SPIN_UNLOCK is not set
# CONFIG_INLINE_SPIN_UNLOCK_BH is not set
# CONFIG_INLINE_SPIN_UNLOCK_IRQ is not set
# CONFIG_INLINE_SPIN_UNLOCK_IRQRESTORE is not set
# CONFIG_INLINE_READ_TRYLOCK is not set
# CONFIG_INLINE_READ_LOCK is not set
# CONFIG_INLINE_READ_LOCK_BH is not set
# CONFIG_INLINE_READ_LOCK_IRQ is not set
# CONFIG_INLINE_READ_LOCK_IRQSAVE is not set
# CONFIG_INLINE_READ_UNLOCK is not set
# CONFIG_INLINE_READ_UNLOCK_BH is not set
# CONFIG_INLINE_READ_UNLOCK_IRQ is not set
# CONFIG_INLINE_READ_UNLOCK_IRQRESTORE is not set
# CONFIG_INLINE_WRITE_TRYLOCK is not set
# CONFIG_INLINE_WRITE_LOCK is not set
# CONFIG_INLINE_WRITE_LOCK_BH is not set
# CONFIG_INLINE_WRITE_LOCK_IRQ is not set
# CONFIG_INLINE_WRITE_LOCK_IRQSAVE is not set
# CONFIG_INLINE_WRITE_UNLOCK is not set
# CONFIG_INLINE_WRITE_UNLOCK_BH is not set
# CONFIG_INLINE_WRITE_UNLOCK_IRQ is not set
# CONFIG_INLINE_WRITE_UNLOCK_IRQRESTORE is not set
CONFIG_MUTEX_SPIN_ON_OWNER=y
# CONFIG_FREEZER is not set

#
# Processor type and features
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_SMP=y
# CONFIG_X86_MPPARSE is not set
# CONFIG_X86_EXTENDED_PLATFORM is not set
CONFIG_X86_SUPPORTS_MEMORY_FAILURE=y
CONFIG_SCHED_OMIT_FRAME_POINTER=y
# CONFIG_PARAVIRT_GUEST is not set
CONFIG_NO_BOOTMEM=y
# CONFIG_MEMTEST is not set
CONFIG_MK8=y
# CONFIG_MPSC is not set
# CONFIG_MCORE2 is not set
# CONFIG_MATOM is not set
# CONFIG_GENERIC_CPU is not set
CONFIG_X86_INTERNODE_CACHE_SHIFT=6
CONFIG_X86_CMPXCHG=y
CONFIG_CMPXCHG_LOCAL=y
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_XADD=y
CONFIG_X86_WP_WORKS_OK=y
CONFIG_X86_INTEL_USERCOPY=y
CONFIG_X86_USE_PPRO_CHECKSUM=y
CONFIG_X86_TSC=y
CONFIG_X86_CMPXCHG64=y
CONFIG_X86_CMOV=y
CONFIG_X86_MINIMUM_CPU_FAMILY=64
CONFIG_X86_DEBUGCTLMSR=y
CONFIG_PROCESSOR_SELECT=y
# CONFIG_CPU_SUP_INTEL is not set
CONFIG_CPU_SUP_AMD=y
# CONFIG_CPU_SUP_CENTAUR is not set
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
CONFIG_DMI=y
# CONFIG_GART_IOMMU is not set
# CONFIG_CALGARY_IOMMU is not set
# CONFIG_AMD_IOMMU is not set
CONFIG_SWIOTLB=y
CONFIG_IOMMU_HELPER=y
# CONFIG_IOMMU_API is not set
# CONFIG_MAXSMP is not set
CONFIG_NR_CPUS=4
# CONFIG_SCHED_SMT is not set
CONFIG_SCHED_MC=y
# CONFIG_IRQ_TIME_ACCOUNTING is not set
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
CONFIG_PREEMPT=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y
# CONFIG_X86_REROUTE_FOR_BROKEN_BOOT_IRQS is not set
CONFIG_X86_MCE=y
# CONFIG_X86_MCE_INTEL is not set
CONFIG_X86_MCE_AMD=y
CONFIG_X86_MCE_THRESHOLD=y
# CONFIG_X86_MCE_INJECT is not set
# CONFIG_I8K is not set
# CONFIG_MICROCODE is not set
CONFIG_X86_MSR=y
CONFIG_X86_CPUID=y
CONFIG_ARCH_PHYS_ADDR_T_64BIT=y
CONFIG_ARCH_DMA_ADDR_T_64BIT=y
CONFIG_DIRECT_GBPAGES=y
# CONFIG_NUMA is not set
CONFIG_ARCH_PROC_KCORE_TEXT=y
CONFIG_ARCH_SPARSEMEM_DEFAULT=y
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_SELECT_MEMORY_MODEL=y
CONFIG_ILLEGAL_POINTER_VALUE=0xdead000000000000
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_SPARSEMEM_MANUAL=y
CONFIG_SPARSEMEM=y
CONFIG_HAVE_MEMORY_PRESENT=y
CONFIG_SPARSEMEM_EXTREME=y
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPARSEMEM_ALLOC_MEM_MAP_TOGETHER=y
CONFIG_SPARSEMEM_VMEMMAP=y
CONFIG_HAVE_MEMBLOCK=y
# CONFIG_MEMORY_HOTPLUG is not set
CONFIG_PAGEFLAGS_EXTENDED=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_COMPACTION=y
CONFIG_MIGRATION=y
CONFIG_PHYS_ADDR_T_64BIT=y
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_MMU_NOTIFIER=y
# CONFIG_KSM is not set
CONFIG_DEFAULT_MMAP_MIN_ADDR=4096
CONFIG_ARCH_SUPPORTS_MEMORY_FAILURE=y
# CONFIG_MEMORY_FAILURE is not set
CONFIG_TRANSPARENT_HUGEPAGE=y
CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS=y
# CONFIG_TRANSPARENT_HUGEPAGE_MADVISE is not set
# CONFIG_X86_CHECK_BIOS_CORRUPTION is not set
CONFIG_X86_RESERVE_LOW=64
CONFIG_MTRR=y
CONFIG_MTRR_SANITIZER=y
CONFIG_MTRR_SANITIZER_ENABLE_DEFAULT=1
CONFIG_MTRR_SANITIZER_SPARE_REG_NR_DEFAULT=2
CONFIG_X86_PAT=y
CONFIG_ARCH_USES_PG_UNCACHED=y
# CONFIG_EFI is not set
CONFIG_SECCOMP=y
# CONFIG_CC_STACKPROTECTOR is not set
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_SCHED_HRTICK=y
CONFIG_KEXEC=y
# CONFIG_CRASH_DUMP is not set
CONFIG_PHYSICAL_START=0x200000
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_ALIGN=0x1000000
# CONFIG_HOTPLUG_CPU is not set
CONFIG_COMPAT_VDSO=y
# CONFIG_CMDLINE_BOOL is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y

#
# Power management and ACPI options
#
# CONFIG_SUSPEND is not set
# CONFIG_HIBERNATION is not set
# CONFIG_PM_RUNTIME is not set
CONFIG_ACPI=y
# CONFIG_ACPI_PROCFS is not set
# CONFIG_ACPI_PROCFS_POWER is not set
# CONFIG_ACPI_POWER_METER is not set
# CONFIG_ACPI_EC_DEBUGFS is not set
# CONFIG_ACPI_PROC_EVENT is not set
# CONFIG_ACPI_AC is not set
# CONFIG_ACPI_BATTERY is not set
CONFIG_ACPI_BUTTON=y
# CONFIG_ACPI_VIDEO is not set
# CONFIG_ACPI_FAN is not set
# CONFIG_ACPI_DOCK is not set
CONFIG_ACPI_PROCESSOR=y
# CONFIG_ACPI_PROCESSOR_AGGREGATOR is not set
CONFIG_ACPI_THERMAL=y
# CONFIG_ACPI_CUSTOM_DSDT is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
# CONFIG_ACPI_PCI_SLOT is not set
CONFIG_X86_PM_TIMER=y
# CONFIG_ACPI_CONTAINER is not set
# CONFIG_ACPI_SBS is not set
# CONFIG_ACPI_HED is not set
# CONFIG_ACPI_APEI is not set
# CONFIG_SFI is not set

#
# CPU Frequency scaling
#
CONFIG_CPU_FREQ=y
CONFIG_CPU_FREQ_TABLE=y
# CONFIG_CPU_FREQ_DEBUG is not set
CONFIG_CPU_FREQ_STAT=y
# CONFIG_CPU_FREQ_STAT_DETAILS is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_POWERSAVE is not set
# CONFIG_CPU_FREQ_DEFAULT_GOV_USERSPACE is not set
CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_CONSERVATIVE is not set
CONFIG_CPU_FREQ_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_GOV_POWERSAVE is not set
# CONFIG_CPU_FREQ_GOV_USERSPACE is not set
CONFIG_CPU_FREQ_GOV_ONDEMAND=y
# CONFIG_CPU_FREQ_GOV_CONSERVATIVE is not set

#
# CPUFreq processor drivers
#
# CONFIG_X86_PCC_CPUFREQ is not set
# CONFIG_X86_ACPI_CPUFREQ is not set
CONFIG_X86_POWERNOW_K8=y
# CONFIG_X86_SPEEDSTEP_CENTRINO is not set
# CONFIG_X86_P4_CLOCKMOD is not set

#
# shared options
#
# CONFIG_X86_SPEEDSTEP_LIB is not set
CONFIG_CPU_IDLE=y
CONFIG_CPU_IDLE_GOV_LADDER=y
CONFIG_CPU_IDLE_GOV_MENU=y

#
# Memory power savings
#
# CONFIG_I7300_IDLE is not set

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
# CONFIG_PCI_MMCONFIG is not set
CONFIG_PCI_DOMAINS=y
# CONFIG_PCI_CNB20LE_QUIRK is not set
# CONFIG_DMAR is not set
# CONFIG_INTR_REMAP is not set
# CONFIG_PCIEPORTBUS is not set
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
# CONFIG_PCI_DEBUG is not set
# CONFIG_PCI_STUB is not set
CONFIG_HT_IRQ=y
# CONFIG_PCI_IOV is not set
CONFIG_PCI_IOAPIC=y
CONFIG_ISA_DMA_API=y
CONFIG_AMD_NB=y
# CONFIG_PCCARD is not set
# CONFIG_HOTPLUG_PCI is not set
# CONFIG_RAPIDIO is not set

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
CONFIG_COMPAT_BINFMT_ELF=y
# CONFIG_HAVE_AOUT is not set
# CONFIG_BINFMT_MISC is not set
CONFIG_IA32_EMULATION=y
# CONFIG_IA32_AOUT is not set
CONFIG_X86_X32_ABI=y
CONFIG_COMPAT=y
CONFIG_COMPAT_FOR_U64_ALIGNMENT=y
CONFIG_SYSVIPC_COMPAT=y
CONFIG_HAVE_TEXT_POKE_SMP=y
CONFIG_NET=y

#
# Networking options
#
CONFIG_PACKET=y
CONFIG_UNIX=y
# CONFIG_NET_KEY is not set
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE_DEMUX is not set
# CONFIG_ARPD is not set
# CONFIG_SYN_COOKIES is not set
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
# CONFIG_INET_XFRM_TUNNEL is not set
# CONFIG_INET_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
CONFIG_INET_LRO=y
CONFIG_INET_DIAG=y
CONFIG_INET_TCP_DIAG=y
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
# CONFIG_IPV6 is not set
# CONFIG_NETWORK_SECMARK is not set
# CONFIG_NETWORK_PHY_TIMESTAMPING is not set
CONFIG_NETFILTER=y
# CONFIG_NETFILTER_DEBUG is not set
# CONFIG_NETFILTER_ADVANCED is not set

#
# Core Netfilter Configuration
#
CONFIG_NETFILTER_NETLINK=y
CONFIG_NETFILTER_NETLINK_LOG=y
CONFIG_NF_CONNTRACK=y
CONFIG_NF_CONNTRACK_FTP=y
CONFIG_NF_CONNTRACK_IRC=y
CONFIG_NF_CONNTRACK_SIP=y
CONFIG_NF_CT_NETLINK=y
CONFIG_NETFILTER_XTABLES=y

#
# Xtables combined modules
#
CONFIG_NETFILTER_XT_MARK=y

#
# Xtables targets
#
CONFIG_NETFILTER_XT_TARGET_NFLOG=y
CONFIG_NETFILTER_XT_TARGET_TCPMSS=y

#
# Xtables matches
#
CONFIG_NETFILTER_XT_MATCH_CONNTRACK=y
CONFIG_NETFILTER_XT_MATCH_STATE=y
# CONFIG_IP_SET is not set
# CONFIG_IP_VS is not set

#
# IP: Netfilter Configuration
#
CONFIG_NF_DEFRAG_IPV4=y
CONFIG_NF_CONNTRACK_IPV4=y
CONFIG_NF_CONNTRACK_PROC_COMPAT=y
CONFIG_IP_NF_IPTABLES=y
CONFIG_IP_NF_FILTER=y
CONFIG_IP_NF_TARGET_REJECT=y
CONFIG_IP_NF_TARGET_LOG=y
CONFIG_IP_NF_TARGET_ULOG=y
CONFIG_NF_NAT=y
CONFIG_NF_NAT_NEEDED=y
CONFIG_IP_NF_TARGET_MASQUERADE=y
CONFIG_NF_NAT_FTP=y
CONFIG_NF_NAT_IRC=y
# CONFIG_NF_NAT_TFTP is not set
# CONFIG_NF_NAT_AMANDA is not set
# CONFIG_NF_NAT_PPTP is not set
# CONFIG_NF_NAT_H323 is not set
CONFIG_NF_NAT_SIP=y
CONFIG_IP_NF_MANGLE=y
# CONFIG_IP_DCCP is not set
# CONFIG_IP_SCTP is not set
# CONFIG_RDS is not set
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_L2TP is not set
CONFIG_STP=y
CONFIG_BRIDGE=y
CONFIG_BRIDGE_IGMP_SNOOPING=y
# CONFIG_NET_DSA is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
CONFIG_LLC=y
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_PHONET is not set
# CONFIG_IEEE802154 is not set
# CONFIG_NET_SCHED is not set
# CONFIG_DCB is not set
# CONFIG_BATMAN_ADV is not set
CONFIG_RPS=y
CONFIG_RFS_ACCEL=y
CONFIG_XPS=y

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_NET_DROP_MONITOR is not set
# CONFIG_HAMRADIO is not set
# CONFIG_CAN is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not set
# CONFIG_WIRELESS is not set
# CONFIG_WIMAX is not set
# CONFIG_RFKILL is not set
# CONFIG_NET_9P is not set
# CONFIG_CAIF is not set
# CONFIG_CEPH_LIB is not set

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_STANDALONE=y
# CONFIG_PREVENT_FIRMWARE_BUILD is not set
CONFIG_FW_LOADER=y
CONFIG_FIRMWARE_IN_KERNEL=y
CONFIG_EXTRA_FIRMWARE="radeon/R600_rlc.bin"
CONFIG_EXTRA_FIRMWARE_DIR="/lib/firmware"
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_SYS_HYPERVISOR is not set
# CONFIG_CONNECTOR is not set
# CONFIG_MTD is not set
# CONFIG_PARPORT is not set
CONFIG_PNP=y
CONFIG_PNP_DEBUG_MESSAGES=y

#
# Protocols
#
CONFIG_PNPACPI=y
CONFIG_BLK_DEV=y
# CONFIG_BLK_DEV_FD is not set
# CONFIG_BLK_CPQ_DA is not set
# CONFIG_BLK_CPQ_CISS_DA is not set
# CONFIG_BLK_DEV_DAC960 is not set
# CONFIG_BLK_DEV_UMEM is not set
# CONFIG_BLK_DEV_COW_COMMON is not set
CONFIG_BLK_DEV_LOOP=y
# CONFIG_BLK_DEV_CRYPTOLOOP is not set

#
# DRBD disabled because PROC_FS, INET or CONNECTOR not selected
#
# CONFIG_BLK_DEV_NBD is not set
# CONFIG_BLK_DEV_SX8 is not set
# CONFIG_BLK_DEV_UB is not set
# CONFIG_BLK_DEV_RAM is not set
CONFIG_CDROM_PKTCDVD=y
CONFIG_CDROM_PKTCDVD_BUFFERS=8
# CONFIG_CDROM_PKTCDVD_WCACHE is not set
# CONFIG_ATA_OVER_ETH is not set
# CONFIG_VIRTIO_BLK is not set
# CONFIG_BLK_DEV_HD is not set
# CONFIG_BLK_DEV_RBD is not set
# CONFIG_SENSORS_LIS3LV02D is not set
# CONFIG_MISC_DEVICES is not set
CONFIG_HAVE_IDE=y
# CONFIG_IDE is not set

#
# SCSI device support
#
CONFIG_SCSI_MOD=y
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
# CONFIG_SCSI_TGT is not set
# CONFIG_SCSI_NETLINK is not set
# CONFIG_SCSI_PROC_FS is not set

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=y
# CONFIG_BLK_DEV_SR_VENDOR is not set
CONFIG_CHR_DEV_SG=y
# CONFIG_CHR_DEV_SCH is not set
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set
# CONFIG_SCSI_SCAN_ASYNC is not set

#
# SCSI Transports
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_ATTRS is not set
# CONFIG_SCSI_SAS_LIBSAS is not set
# CONFIG_SCSI_SRP_ATTRS is not set
# CONFIG_SCSI_LOWLEVEL is not set
# CONFIG_SCSI_DH is not set
# CONFIG_SCSI_OSD_INITIATOR is not set
CONFIG_ATA=y
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_VERBOSE_ERROR=y
# CONFIG_ATA_ACPI is not set
# CONFIG_SATA_PMP is not set

#
# Controllers with non-SFF native interface
#
CONFIG_SATA_AHCI=y
# CONFIG_SATA_AHCI_PLATFORM is not set
# CONFIG_SATA_INIC162X is not set
# CONFIG_SATA_ACARD_AHCI is not set
# CONFIG_SATA_SIL24 is not set
CONFIG_ATA_SFF=y

#
# SFF controllers with custom DMA interface
#
# CONFIG_PDC_ADMA is not set
# CONFIG_SATA_QSTOR is not set
# CONFIG_SATA_SX4 is not set
CONFIG_ATA_BMDMA=y

#
# SATA SFF controllers with BMDMA
#
# CONFIG_ATA_PIIX is not set
# CONFIG_SATA_MV is not set
# CONFIG_SATA_NV is not set
# CONFIG_SATA_PROMISE is not set
# CONFIG_SATA_SIL is not set
# CONFIG_SATA_SIS is not set
# CONFIG_SATA_SVW is not set
# CONFIG_SATA_ULI is not set
# CONFIG_SATA_VIA is not set
# CONFIG_SATA_VITESSE is not set

#
# PATA SFF controllers with BMDMA
#
# CONFIG_PATA_ALI is not set
# CONFIG_PATA_AMD is not set
# CONFIG_PATA_ARASAN_CF is not set
# CONFIG_PATA_ARTOP is not set
CONFIG_PATA_ATIIXP=y
# CONFIG_PATA_ATP867X is not set
# CONFIG_PATA_CMD64X is not set
# CONFIG_PATA_CS5520 is not set
# CONFIG_PATA_CS5530 is not set
# CONFIG_PATA_CS5536 is not set
# CONFIG_PATA_CYPRESS is not set
# CONFIG_PATA_EFAR is not set
# CONFIG_PATA_HPT366 is not set
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
# CONFIG_PATA_HPT3X3 is not set
# CONFIG_PATA_IT8213 is not set
# CONFIG_PATA_IT821X is not set
# CONFIG_PATA_JMICRON is not set
# CONFIG_PATA_MARVELL is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NINJA32 is not set
# CONFIG_PATA_NS87415 is not set
# CONFIG_PATA_OLDPIIX is not set
# CONFIG_PATA_OPTIDMA is not set
# CONFIG_PATA_PDC2027X is not set
# CONFIG_PATA_PDC_OLD is not set
# CONFIG_PATA_RADISYS is not set
# CONFIG_PATA_RDC is not set
# CONFIG_PATA_SC1200 is not set
# CONFIG_PATA_SCH is not set
# CONFIG_PATA_SERVERWORKS is not set
# CONFIG_PATA_SIL680 is not set
# CONFIG_PATA_SIS is not set
# CONFIG_PATA_TOSHIBA is not set
# CONFIG_PATA_TRIFLEX is not set
# CONFIG_PATA_VIA is not set
# CONFIG_PATA_WINBOND is not set

#
# PIO-only SFF controllers
#
# CONFIG_PATA_CMD640_PCI is not set
# CONFIG_PATA_MPIIX is not set
# CONFIG_PATA_NS87410 is not set
# CONFIG_PATA_OPTI is not set
# CONFIG_PATA_PLATFORM is not set
# CONFIG_PATA_RZ1000 is not set

#
# Generic fallback / legacy drivers
#
# CONFIG_ATA_GENERIC is not set
# CONFIG_PATA_LEGACY is not set
# CONFIG_MD is not set
# CONFIG_TARGET_CORE is not set
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_FIREWIRE is not set
# CONFIG_FIREWIRE_NOSY is not set
# CONFIG_I2O is not set
# CONFIG_MACINTOSH_DRIVERS is not set
CONFIG_NETDEVICES=y
CONFIG_DUMMY=y
# CONFIG_BONDING is not set
# CONFIG_MACVLAN is not set
# CONFIG_EQUALIZER is not set
CONFIG_TUN=y
# CONFIG_VETH is not set
# CONFIG_NET_SB1000 is not set
# CONFIG_ARCNET is not set
CONFIG_MII=y
# CONFIG_PHYLIB is not set
# CONFIG_NET_ETHERNET is not set
CONFIG_NETDEV_1000=y
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_E1000E is not set
# CONFIG_IP1000 is not set
# CONFIG_IGB is not set
# CONFIG_IGBVF is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SIS190 is not set
# CONFIG_SKGE is not set
# CONFIG_SKY2 is not set
# CONFIG_VIA_VELOCITY is not set
# CONFIG_TIGON3 is not set
# CONFIG_BNX2 is not set
# CONFIG_CNIC is not set
# CONFIG_QLA3XXX is not set
# CONFIG_ATL1 is not set
CONFIG_ATL1E=y
# CONFIG_ATL1C is not set
# CONFIG_JME is not set
# CONFIG_STMMAC_ETH is not set
# CONFIG_PCH_GBE is not set
# CONFIG_NETDEV_10000 is not set
# CONFIG_TR is not set
# CONFIG_WLAN is not set

#
# Enable WiMAX (Networking options) to see the WiMAX drivers
#

#
# USB Network Adapters
#
# CONFIG_USB_CATC is not set
# CONFIG_USB_KAWETH is not set
# CONFIG_USB_PEGASUS is not set
# CONFIG_USB_RTL8150 is not set
# CONFIG_USB_USBNET is not set
# CONFIG_USB_IPHETH is not set
# CONFIG_WAN is not set

#
# CAIF transport drivers
#
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
CONFIG_PPP=y
# CONFIG_PPP_MULTILINK is not set
CONFIG_PPP_FILTER=y
CONFIG_PPP_ASYNC=y
CONFIG_PPP_SYNC_TTY=y
# CONFIG_PPP_DEFLATE is not set
# CONFIG_PPP_BSDCOMP is not set
# CONFIG_PPP_MPPE is not set
# CONFIG_PPPOE is not set
# CONFIG_SLIP is not set
CONFIG_SLHC=y
# CONFIG_NET_FC is not set
# CONFIG_NETCONSOLE is not set
# CONFIG_NETPOLL is not set
# CONFIG_NET_POLL_CONTROLLER is not set
# CONFIG_VIRTIO_NET is not set
# CONFIG_VMXNET3 is not set
# CONFIG_ISDN is not set
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y
# CONFIG_INPUT_FF_MEMLESS is not set
# CONFIG_INPUT_POLLDEV is not set
# CONFIG_INPUT_SPARSEKMAP is not set

#
# Userland interfaces
#
CONFIG_INPUT_MOUSEDEV=y
# CONFIG_INPUT_MOUSEDEV_PSAUX is not set
CONFIG_INPUT_MOUSEDEV_SCREEN_X=1024
CONFIG_INPUT_MOUSEDEV_SCREEN_Y=768
# CONFIG_INPUT_JOYDEV is not set
CONFIG_INPUT_EVDEV=y
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
# CONFIG_KEYBOARD_ADP5588 is not set
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_QT1070 is not set
# CONFIG_KEYBOARD_QT2160 is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_TCA6416 is not set
# CONFIG_KEYBOARD_MAX7359 is not set
# CONFIG_KEYBOARD_MCS is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_OPENCORES is not set
# CONFIG_KEYBOARD_STOWAWAY is not set
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
CONFIG_INPUT_MOUSE=y
# CONFIG_MOUSE_PS2 is not set
# CONFIG_MOUSE_SERIAL is not set
# CONFIG_MOUSE_APPLETOUCH is not set
# CONFIG_MOUSE_BCM5974 is not set
# CONFIG_MOUSE_VSXXXAA is not set
# CONFIG_MOUSE_SYNAPTICS_I2C is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
# CONFIG_SERIO_ALTERA_PS2 is not set
# CONFIG_SERIO_PS2MULT is not set
# CONFIG_GAMEPORT is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_CONSOLE_TRANSLATIONS=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_VT_HW_CONSOLE_BINDING is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_DEVPTS_MULTIPLE_INSTANCES is not set
# CONFIG_LEGACY_PTYS is not set
# CONFIG_SERIAL_NONSTANDARD is not set
# CONFIG_NOZOMI is not set
# CONFIG_N_GSM is not set
# CONFIG_DEVKMEM is not set

#
# Serial drivers
#
# CONFIG_SERIAL_8250 is not set
CONFIG_FIX_EARLYCON_MEM=y

#
# Non-8250 serial port support
#
# CONFIG_SERIAL_MFD_HSU is not set
# CONFIG_SERIAL_JSM is not set
# CONFIG_SERIAL_TIMBERDALE is not set
# CONFIG_SERIAL_ALTERA_JTAGUART is not set
# CONFIG_SERIAL_ALTERA_UART is not set
# CONFIG_SERIAL_PCH_UART is not set
# CONFIG_TTY_PRINTK is not set
# CONFIG_VIRTIO_CONSOLE is not set
# CONFIG_IPMI_HANDLER is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_NVRAM is not set
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_MWAVE is not set
# CONFIG_RAW_DRIVER is not set
CONFIG_HPET=y
CONFIG_HPET_MMAP=y
# CONFIG_HANGCHECK_TIMER is not set
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
CONFIG_DEVPORT=y
# CONFIG_RAMOOPS is not set
CONFIG_I2C=y
CONFIG_I2C_BOARDINFO=y
CONFIG_I2C_COMPAT=y
CONFIG_I2C_CHARDEV=y
# CONFIG_I2C_MUX is not set
CONFIG_I2C_HELPER_AUTO=y
CONFIG_I2C_ALGOBIT=y

#
# I2C Hardware Bus support
#

#
# PC SMBus host controller drivers
#
# CONFIG_I2C_ALI1535 is not set
# CONFIG_I2C_ALI1563 is not set
# CONFIG_I2C_ALI15X3 is not set
# CONFIG_I2C_AMD756 is not set
# CONFIG_I2C_AMD8111 is not set
# CONFIG_I2C_I801 is not set
# CONFIG_I2C_ISCH is not set
# CONFIG_I2C_PIIX4 is not set
# CONFIG_I2C_NFORCE2 is not set
# CONFIG_I2C_SIS5595 is not set
# CONFIG_I2C_SIS630 is not set
# CONFIG_I2C_SIS96X is not set
# CONFIG_I2C_VIA is not set
# CONFIG_I2C_VIAPRO is not set

#
# ACPI drivers
#
# CONFIG_I2C_SCMI is not set

#
# I2C system bus drivers (mostly embedded / system-on-chip)
#
# CONFIG_I2C_INTEL_MID is not set
# CONFIG_I2C_OCORES is not set
# CONFIG_I2C_PCA_PLATFORM is not set
# CONFIG_I2C_PXA_PCI is not set
# CONFIG_I2C_SIMTEC is not set
# CONFIG_I2C_XILINX is not set
# CONFIG_I2C_EG20T is not set

#
# External I2C/SMBus adapter drivers
#
# CONFIG_I2C_DIOLAN_U2C is not set
# CONFIG_I2C_PARPORT_LIGHT is not set
# CONFIG_I2C_TAOS_EVM is not set
# CONFIG_I2C_TINY_USB is not set

#
# Other I2C/SMBus bus drivers
#
# CONFIG_I2C_DEBUG_CORE is not set
# CONFIG_I2C_DEBUG_ALGO is not set
# CONFIG_I2C_DEBUG_BUS is not set
# CONFIG_SPI is not set

#
# PPS support
#
# CONFIG_PPS is not set

#
# PPS generators support
#
CONFIG_ARCH_WANT_OPTIONAL_GPIOLIB=y
# CONFIG_GPIOLIB is not set
# CONFIG_W1 is not set
CONFIG_POWER_SUPPLY=y
# CONFIG_POWER_SUPPLY_DEBUG is not set
# CONFIG_PDA_POWER is not set
# CONFIG_TEST_POWER is not set
# CONFIG_BATTERY_DS2782 is not set
# CONFIG_BATTERY_BQ20Z75 is not set
# CONFIG_BATTERY_BQ27x00 is not set
# CONFIG_BATTERY_MAX17040 is not set
# CONFIG_BATTERY_MAX17042 is not set
CONFIG_HWMON=y
# CONFIG_HWMON_VID is not set
# CONFIG_HWMON_DEBUG_CHIP is not set

#
# Native drivers
#
# CONFIG_SENSORS_ABITUGURU is not set
# CONFIG_SENSORS_ABITUGURU3 is not set
# CONFIG_SENSORS_AD7414 is not set
# CONFIG_SENSORS_AD7418 is not set
# CONFIG_SENSORS_ADM1021 is not set
# CONFIG_SENSORS_ADM1025 is not set
# CONFIG_SENSORS_ADM1026 is not set
# CONFIG_SENSORS_ADM1029 is not set
# CONFIG_SENSORS_ADM1031 is not set
# CONFIG_SENSORS_ADM9240 is not set
# CONFIG_SENSORS_ADT7411 is not set
# CONFIG_SENSORS_ADT7462 is not set
# CONFIG_SENSORS_ADT7470 is not set
# CONFIG_SENSORS_ADT7475 is not set
# CONFIG_SENSORS_ASC7621 is not set
# CONFIG_SENSORS_K8TEMP is not set
CONFIG_SENSORS_K10TEMP=y
# CONFIG_SENSORS_ASB100 is not set
# CONFIG_SENSORS_ATXP1 is not set
# CONFIG_SENSORS_DS620 is not set
# CONFIG_SENSORS_DS1621 is not set
# CONFIG_SENSORS_I5K_AMB is not set
# CONFIG_SENSORS_F71805F is not set
# CONFIG_SENSORS_F71882FG is not set
# CONFIG_SENSORS_F75375S is not set
# CONFIG_SENSORS_FSCHMD is not set
# CONFIG_SENSORS_G760A is not set
# CONFIG_SENSORS_GL518SM is not set
# CONFIG_SENSORS_GL520SM is not set
# CONFIG_SENSORS_CORETEMP is not set
# CONFIG_SENSORS_PKGTEMP is not set
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_JC42 is not set
# CONFIG_SENSORS_LINEAGE is not set
# CONFIG_SENSORS_LM63 is not set
# CONFIG_SENSORS_LM73 is not set
# CONFIG_SENSORS_LM75 is not set
# CONFIG_SENSORS_LM77 is not set
# CONFIG_SENSORS_LM78 is not set
# CONFIG_SENSORS_LM80 is not set
# CONFIG_SENSORS_LM83 is not set
# CONFIG_SENSORS_LM85 is not set
# CONFIG_SENSORS_LM87 is not set
# CONFIG_SENSORS_LM90 is not set
# CONFIG_SENSORS_LM92 is not set
# CONFIG_SENSORS_LM93 is not set
# CONFIG_SENSORS_LTC4151 is not set
# CONFIG_SENSORS_LTC4215 is not set
# CONFIG_SENSORS_LTC4245 is not set
# CONFIG_SENSORS_LTC4261 is not set
# CONFIG_SENSORS_LM95241 is not set
# CONFIG_SENSORS_MAX1619 is not set
# CONFIG_SENSORS_MAX6639 is not set
# CONFIG_SENSORS_MAX6650 is not set
# CONFIG_SENSORS_PC87360 is not set
# CONFIG_SENSORS_PC87427 is not set
# CONFIG_SENSORS_PCF8591 is not set
# CONFIG_PMBUS is not set
# CONFIG_SENSORS_SHT21 is not set
# CONFIG_SENSORS_SIS5595 is not set
# CONFIG_SENSORS_SMM665 is not set
# CONFIG_SENSORS_DME1737 is not set
# CONFIG_SENSORS_EMC1403 is not set
# CONFIG_SENSORS_EMC2103 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_SMSC47M192 is not set
# CONFIG_SENSORS_SMSC47B397 is not set
# CONFIG_SENSORS_SCH5627 is not set
# CONFIG_SENSORS_ADS1015 is not set
# CONFIG_SENSORS_ADS7828 is not set
# CONFIG_SENSORS_AMC6821 is not set
# CONFIG_SENSORS_THMC50 is not set
# CONFIG_SENSORS_TMP102 is not set
# CONFIG_SENSORS_TMP401 is not set
# CONFIG_SENSORS_TMP421 is not set
# CONFIG_SENSORS_VIA_CPUTEMP is not set
# CONFIG_SENSORS_VIA686A is not set
# CONFIG_SENSORS_VT1211 is not set
# CONFIG_SENSORS_VT8231 is not set
# CONFIG_SENSORS_W83781D is not set
# CONFIG_SENSORS_W83791D is not set
# CONFIG_SENSORS_W83792D is not set
# CONFIG_SENSORS_W83793 is not set
# CONFIG_SENSORS_W83795 is not set
# CONFIG_SENSORS_W83L785TS is not set
# CONFIG_SENSORS_W83L786NG is not set
# CONFIG_SENSORS_W83627HF is not set
# CONFIG_SENSORS_W83627EHF is not set
# CONFIG_SENSORS_APPLESMC is not set

#
# ACPI drivers
#
CONFIG_SENSORS_ATK0110=y
CONFIG_THERMAL=y
CONFIG_THERMAL_HWMON=y
# CONFIG_WATCHDOG is not set
CONFIG_SSB_POSSIBLE=y

#
# Sonics Silicon Backplane
#
# CONFIG_SSB is not set
# CONFIG_MFD_SUPPORT is not set
# CONFIG_REGULATOR is not set
# CONFIG_MEDIA_SUPPORT is not set

#
# Graphics support
#
# CONFIG_AGP is not set
# CONFIG_VGA_ARB is not set
# CONFIG_VGA_SWITCHEROO is not set
CONFIG_DRM=y
CONFIG_DRM_KMS_HELPER=y
CONFIG_DRM_TTM=y
# CONFIG_DRM_TDFX is not set
# CONFIG_DRM_R128 is not set
CONFIG_DRM_RADEON=y
CONFIG_DRM_RADEON_KMS=y
# CONFIG_DRM_MGA is not set
# CONFIG_DRM_VIA is not set
# CONFIG_DRM_SAVAGE is not set
# CONFIG_STUB_POULSBO is not set
# CONFIG_VGASTATE is not set
CONFIG_VIDEO_OUTPUT_CONTROL=y
CONFIG_FB=y
# CONFIG_FIRMWARE_EDID is not set
# CONFIG_FB_DDC is not set
# CONFIG_FB_BOOT_VESA_SUPPORT is not set
CONFIG_FB_CFB_FILLRECT=y
CONFIG_FB_CFB_COPYAREA=y
CONFIG_FB_CFB_IMAGEBLIT=y
# CONFIG_FB_CFB_REV_PIXELS_IN_BYTE is not set
# CONFIG_FB_SYS_FILLRECT is not set
# CONFIG_FB_SYS_COPYAREA is not set
# CONFIG_FB_SYS_IMAGEBLIT is not set
# CONFIG_FB_FOREIGN_ENDIAN is not set
# CONFIG_FB_SYS_FOPS is not set
# CONFIG_FB_WMT_GE_ROPS is not set
# CONFIG_FB_SVGALIB is not set
# CONFIG_FB_MACMODES is not set
# CONFIG_FB_BACKLIGHT is not set
CONFIG_FB_MODE_HELPERS=y
# CONFIG_FB_TILEBLITTING is not set

#
# Frame buffer hardware drivers
#
# CONFIG_FB_CIRRUS is not set
# CONFIG_FB_PM2 is not set
# CONFIG_FB_CYBER2000 is not set
# CONFIG_FB_ARC is not set
# CONFIG_FB_ASILIANT is not set
# CONFIG_FB_IMSTT is not set
# CONFIG_FB_VGA16 is not set
# CONFIG_FB_VESA is not set
# CONFIG_FB_N411 is not set
# CONFIG_FB_HGA is not set
# CONFIG_FB_S1D13XXX is not set
# CONFIG_FB_NVIDIA is not set
# CONFIG_FB_RIVA is not set
# CONFIG_FB_LE80578 is not set
# CONFIG_FB_MATROX is not set
# CONFIG_FB_RADEON is not set
# CONFIG_FB_ATY128 is not set
# CONFIG_FB_ATY is not set
# CONFIG_FB_S3 is not set
# CONFIG_FB_SAVAGE is not set
# CONFIG_FB_SIS is not set
# CONFIG_FB_VIA is not set
# CONFIG_FB_NEOMAGIC is not set
# CONFIG_FB_KYRO is not set
# CONFIG_FB_3DFX is not set
# CONFIG_FB_VOODOO1 is not set
# CONFIG_FB_VT8623 is not set
# CONFIG_FB_TRIDENT is not set
# CONFIG_FB_ARK is not set
# CONFIG_FB_PM3 is not set
# CONFIG_FB_CARMINE is not set
# CONFIG_FB_GEODE is not set
# CONFIG_FB_UDL is not set
# CONFIG_FB_VIRTUAL is not set
# CONFIG_FB_METRONOME is not set
# CONFIG_FB_MB862XX is not set
# CONFIG_FB_BROADSHEET is not set
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set
CONFIG_BACKLIGHT_CLASS_DEVICE=y

#
# Display device support
#
# CONFIG_DISPLAY_SUPPORT is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
CONFIG_VGACON_SOFT_SCROLLBACK=y
CONFIG_VGACON_SOFT_SCROLLBACK_SIZE=64
CONFIG_DUMMY_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE=y
# CONFIG_FRAMEBUFFER_CONSOLE_DETECT_PRIMARY is not set
CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y
# CONFIG_FONTS is not set
CONFIG_FONT_8x8=y
CONFIG_FONT_8x16=y
# CONFIG_LOGO is not set
CONFIG_SOUND=y
CONFIG_SOUND_OSS_CORE=y
CONFIG_SOUND_OSS_CORE_PRECLAIM=y
CONFIG_SND=y
CONFIG_SND_TIMER=y
CONFIG_SND_PCM=y
CONFIG_SND_HWDEP=y
CONFIG_SND_RAWMIDI=y
CONFIG_SND_SEQUENCER=y
CONFIG_SND_SEQ_DUMMY=y
CONFIG_SND_OSSEMUL=y
CONFIG_SND_MIXER_OSS=y
CONFIG_SND_PCM_OSS=y
CONFIG_SND_PCM_OSS_PLUGINS=y
CONFIG_SND_SEQUENCER_OSS=y
CONFIG_SND_HRTIMER=y
CONFIG_SND_SEQ_HRTIMER_DEFAULT=y
CONFIG_SND_DYNAMIC_MINORS=y
CONFIG_SND_SUPPORT_OLD_API=y
# CONFIG_SND_VERBOSE_PROCFS is not set
# CONFIG_SND_VERBOSE_PRINTK is not set
# CONFIG_SND_DEBUG is not set
CONFIG_SND_DMA_SGBUF=y
CONFIG_SND_RAWMIDI_SEQ=y
# CONFIG_SND_OPL3_LIB_SEQ is not set
# CONFIG_SND_OPL4_LIB_SEQ is not set
# CONFIG_SND_SBAWE_SEQ is not set
# CONFIG_SND_EMU10K1_SEQ is not set
# CONFIG_SND_DRIVERS is not set
# CONFIG_SND_PCI is not set
CONFIG_SND_USB=y
CONFIG_SND_USB_AUDIO=y
# CONFIG_SND_USB_UA101 is not set
# CONFIG_SND_USB_USX2Y is not set
# CONFIG_SND_USB_CAIAQ is not set
# CONFIG_SND_USB_US122L is not set
# CONFIG_SND_USB_6FIRE is not set
# CONFIG_SND_SOC is not set
# CONFIG_SOUND_PRIME is not set
CONFIG_HID_SUPPORT=y
CONFIG_HID=y
CONFIG_HIDRAW=y

#
# USB Input Devices
#
CONFIG_USB_HID=y
# CONFIG_HID_PID is not set
CONFIG_USB_HIDDEV=y

#
# Special HID drivers
#
# CONFIG_HID_3M_PCT is not set
# CONFIG_HID_A4TECH is not set
# CONFIG_HID_ACRUX is not set
# CONFIG_HID_APPLE is not set
# CONFIG_HID_BELKIN is not set
# CONFIG_HID_CANDO is not set
# CONFIG_HID_CHERRY is not set
# CONFIG_HID_CHICONY is not set
# CONFIG_HID_PRODIKEYS is not set
# CONFIG_HID_CYPRESS is not set
# CONFIG_HID_DRAGONRISE is not set
# CONFIG_HID_EMS_FF is not set
# CONFIG_HID_EZKEY is not set
# CONFIG_HID_KEYTOUCH is not set
# CONFIG_HID_KYE is not set
# CONFIG_HID_UCLOGIC is not set
# CONFIG_HID_WALTOP is not set
# CONFIG_HID_GYRATION is not set
# CONFIG_HID_TWINHAN is not set
# CONFIG_HID_KENSINGTON is not set
# CONFIG_HID_LCPOWER is not set
# CONFIG_HID_LOGITECH is not set
# CONFIG_HID_MICROSOFT is not set
# CONFIG_HID_MOSART is not set
# CONFIG_HID_MONTEREY is not set
# CONFIG_HID_MULTITOUCH is not set
# CONFIG_HID_NTRIG is not set
# CONFIG_HID_ORTEK is not set
# CONFIG_HID_PANTHERLORD is not set
# CONFIG_HID_PETALYNX is not set
# CONFIG_HID_PICOLCD is not set
# CONFIG_HID_QUANTA is not set
# CONFIG_HID_ROCCAT is not set
# CONFIG_HID_ROCCAT_ARVO is not set
# CONFIG_HID_ROCCAT_KONE is not set
# CONFIG_HID_ROCCAT_KONEPLUS is not set
# CONFIG_HID_ROCCAT_KOVAPLUS is not set
# CONFIG_HID_ROCCAT_PYRA is not set
# CONFIG_HID_SAMSUNG is not set
# CONFIG_HID_SONY is not set
# CONFIG_HID_STANTUM is not set
# CONFIG_HID_SUNPLUS is not set
# CONFIG_HID_GREENASIA is not set
# CONFIG_HID_SMARTJOYPLUS is not set
# CONFIG_HID_TOPSEED is not set
# CONFIG_HID_THRUSTMASTER is not set
# CONFIG_HID_ZEROPLUS is not set
# CONFIG_HID_ZYDACRON is not set
CONFIG_USB_SUPPORT=y
CONFIG_USB_ARCH_HAS_HCD=y
CONFIG_USB_ARCH_HAS_OHCI=y
CONFIG_USB_ARCH_HAS_EHCI=y
CONFIG_USB=y
# CONFIG_USB_DEBUG is not set
# CONFIG_USB_ANNOUNCE_NEW_DEVICES is not set

#
# Miscellaneous USB options
#
# CONFIG_USB_DEVICEFS is not set
# CONFIG_USB_DEVICE_CLASS is not set
# CONFIG_USB_DYNAMIC_MINORS is not set
# CONFIG_USB_OTG_WHITELIST is not set
# CONFIG_USB_OTG_BLACKLIST_HUB is not set
CONFIG_USB_MON=y
# CONFIG_USB_WUSB is not set
# CONFIG_USB_WUSB_CBAF is not set

#
# USB Host Controller Drivers
#
# CONFIG_USB_C67X00_HCD is not set
# CONFIG_USB_XHCI_HCD is not set
CONFIG_USB_EHCI_HCD=y
# CONFIG_USB_EHCI_ROOT_HUB_TT is not set
# CONFIG_USB_EHCI_TT_NEWSCHED is not set
# CONFIG_USB_OXU210HP_HCD is not set
# CONFIG_USB_ISP116X_HCD is not set
# CONFIG_USB_ISP1760_HCD is not set
# CONFIG_USB_ISP1362_HCD is not set
CONFIG_USB_OHCI_HCD=y
# CONFIG_USB_OHCI_BIG_ENDIAN_DESC is not set
# CONFIG_USB_OHCI_BIG_ENDIAN_MMIO is not set
CONFIG_USB_OHCI_LITTLE_ENDIAN=y
# CONFIG_USB_UHCI_HCD is not set
# CONFIG_USB_SL811_HCD is not set
# CONFIG_USB_R8A66597_HCD is not set
# CONFIG_USB_WHCI_HCD is not set
# CONFIG_USB_HWA_HCD is not set

#
# USB Device Class drivers
#
# CONFIG_USB_ACM is not set
# CONFIG_USB_PRINTER is not set
# CONFIG_USB_WDM is not set
# CONFIG_USB_TMC is not set

#
# NOTE: USB_STORAGE depends on SCSI but BLK_DEV_SD may
#

#
# also be needed; see USB_STORAGE Help for more info
#
CONFIG_USB_STORAGE=y
# CONFIG_USB_STORAGE_DEBUG is not set
# CONFIG_USB_STORAGE_REALTEK is not set
# CONFIG_USB_STORAGE_DATAFAB is not set
# CONFIG_USB_STORAGE_FREECOM is not set
# CONFIG_USB_STORAGE_ISD200 is not set
# CONFIG_USB_STORAGE_USBAT is not set
# CONFIG_USB_STORAGE_SDDR09 is not set
# CONFIG_USB_STORAGE_SDDR55 is not set
# CONFIG_USB_STORAGE_JUMPSHOT is not set
# CONFIG_USB_STORAGE_ALAUDA is not set
# CONFIG_USB_STORAGE_ONETOUCH is not set
# CONFIG_USB_STORAGE_KARMA is not set
# CONFIG_USB_STORAGE_CYPRESS_ATACB is not set
# CONFIG_USB_STORAGE_ENE_UB6250 is not set
# CONFIG_USB_UAS is not set
# CONFIG_USB_LIBUSUAL is not set

#
# USB Imaging devices
#
# CONFIG_USB_MDC800 is not set
# CONFIG_USB_MICROTEK is not set

#
# USB port drivers
#
CONFIG_USB_SERIAL=y
# CONFIG_USB_SERIAL_CONSOLE is not set
# CONFIG_USB_EZUSB is not set
CONFIG_USB_SERIAL_GENERIC=y
# CONFIG_USB_SERIAL_AIRCABLE is not set
# CONFIG_USB_SERIAL_ARK3116 is not set
# CONFIG_USB_SERIAL_BELKIN is not set
# CONFIG_USB_SERIAL_CH341 is not set
# CONFIG_USB_SERIAL_WHITEHEAT is not set
# CONFIG_USB_SERIAL_DIGI_ACCELEPORT is not set
# CONFIG_USB_SERIAL_CP210X is not set
# CONFIG_USB_SERIAL_CYPRESS_M8 is not set
# CONFIG_USB_SERIAL_EMPEG is not set
# CONFIG_USB_SERIAL_FTDI_SIO is not set
# CONFIG_USB_SERIAL_FUNSOFT is not set
# CONFIG_USB_SERIAL_VISOR is not set
# CONFIG_USB_SERIAL_IPAQ is not set
# CONFIG_USB_SERIAL_IR is not set
# CONFIG_USB_SERIAL_EDGEPORT is not set
# CONFIG_USB_SERIAL_EDGEPORT_TI is not set
# CONFIG_USB_SERIAL_GARMIN is not set
# CONFIG_USB_SERIAL_IPW is not set
# CONFIG_USB_SERIAL_IUU is not set
# CONFIG_USB_SERIAL_KEYSPAN_PDA is not set
# CONFIG_USB_SERIAL_KEYSPAN is not set
# CONFIG_USB_SERIAL_KLSI is not set
# CONFIG_USB_SERIAL_KOBIL_SCT is not set
# CONFIG_USB_SERIAL_MCT_U232 is not set
# CONFIG_USB_SERIAL_MOS7720 is not set
# CONFIG_USB_SERIAL_MOS7840 is not set
# CONFIG_USB_SERIAL_MOTOROLA is not set
# CONFIG_USB_SERIAL_NAVMAN is not set
# CONFIG_USB_SERIAL_PL2303 is not set
# CONFIG_USB_SERIAL_OTI6858 is not set
# CONFIG_USB_SERIAL_QCAUX is not set
# CONFIG_USB_SERIAL_QUALCOMM is not set
# CONFIG_USB_SERIAL_SPCP8X5 is not set
# CONFIG_USB_SERIAL_HP4X is not set
# CONFIG_USB_SERIAL_SAFE is not set
# CONFIG_USB_SERIAL_SAMBA is not set
# CONFIG_USB_SERIAL_SIEMENS_MPI is not set
# CONFIG_USB_SERIAL_SIERRAWIRELESS is not set
# CONFIG_USB_SERIAL_SYMBOL is not set
# CONFIG_USB_SERIAL_TI is not set
# CONFIG_USB_SERIAL_CYBERJACK is not set
# CONFIG_USB_SERIAL_XIRCOM is not set
CONFIG_USB_SERIAL_WWAN=y
CONFIG_USB_SERIAL_OPTION=y
# CONFIG_USB_SERIAL_OMNINET is not set
# CONFIG_USB_SERIAL_OPTICON is not set
# CONFIG_USB_SERIAL_VIVOPAY_SERIAL is not set
# CONFIG_USB_SERIAL_ZIO is not set
# CONFIG_USB_SERIAL_SSU100 is not set
# CONFIG_USB_SERIAL_DEBUG is not set

#
# USB Miscellaneous drivers
#
# CONFIG_USB_EMI62 is not set
# CONFIG_USB_EMI26 is not set
# CONFIG_USB_ADUTUX is not set
# CONFIG_USB_SEVSEG is not set
# CONFIG_USB_RIO500 is not set
# CONFIG_USB_LEGOTOWER is not set
# CONFIG_USB_LCD is not set
# CONFIG_USB_LED is not set
# CONFIG_USB_CYPRESS_CY7C63 is not set
# CONFIG_USB_CYTHERM is not set
# CONFIG_USB_IDMOUSE is not set
# CONFIG_USB_FTDI_ELAN is not set
# CONFIG_USB_APPLEDISPLAY is not set
# CONFIG_USB_SISUSBVGA is not set
# CONFIG_USB_LD is not set
# CONFIG_USB_TRANCEVIBRATOR is not set
# CONFIG_USB_IOWARRIOR is not set
# CONFIG_USB_TEST is not set
# CONFIG_USB_ISIGHTFW is not set
# CONFIG_USB_YUREX is not set
# CONFIG_USB_GADGET is not set

#
# OTG and related infrastructure
#
# CONFIG_NOP_USB_XCEIV is not set
# CONFIG_UWB is not set
# CONFIG_MMC is not set
# CONFIG_MEMSTICK is not set
# CONFIG_NEW_LEDS is not set
# CONFIG_NFC_DEVICES is not set
# CONFIG_ACCESSIBILITY is not set
# CONFIG_INFINIBAND is not set
CONFIG_EDAC=y

#
# Reporting subsystems
#
# CONFIG_EDAC_DEBUG is not set
CONFIG_EDAC_DECODE_MCE=y
# CONFIG_EDAC_MCE_INJ is not set
CONFIG_EDAC_MM_EDAC=y
CONFIG_EDAC_AMD64=y
# CONFIG_EDAC_AMD64_ERROR_INJECTION is not set
# CONFIG_EDAC_E752X is not set
# CONFIG_EDAC_I82975X is not set
# CONFIG_EDAC_I3000 is not set
# CONFIG_EDAC_I3200 is not set
# CONFIG_EDAC_X38 is not set
# CONFIG_EDAC_I5400 is not set
# CONFIG_EDAC_I7CORE is not set
# CONFIG_EDAC_I5000 is not set
# CONFIG_EDAC_I5100 is not set
# CONFIG_EDAC_I7300 is not set
CONFIG_RTC_LIB=y
CONFIG_RTC_CLASS=y
CONFIG_RTC_HCTOSYS=y
CONFIG_RTC_HCTOSYS_DEVICE="rtc0"
# CONFIG_RTC_DEBUG is not set

#
# RTC interfaces
#
CONFIG_RTC_INTF_SYSFS=y
CONFIG_RTC_INTF_PROC=y
CONFIG_RTC_INTF_DEV=y
# CONFIG_RTC_INTF_DEV_UIE_EMUL is not set
# CONFIG_RTC_DRV_TEST is not set

#
# I2C RTC drivers
#
# CONFIG_RTC_DRV_DS1307 is not set
# CONFIG_RTC_DRV_DS1374 is not set
# CONFIG_RTC_DRV_DS1672 is not set
# CONFIG_RTC_DRV_DS3232 is not set
# CONFIG_RTC_DRV_MAX6900 is not set
# CONFIG_RTC_DRV_RS5C372 is not set
# CONFIG_RTC_DRV_ISL1208 is not set
# CONFIG_RTC_DRV_ISL12022 is not set
# CONFIG_RTC_DRV_X1205 is not set
# CONFIG_RTC_DRV_PCF8563 is not set
# CONFIG_RTC_DRV_PCF8583 is not set
# CONFIG_RTC_DRV_M41T80 is not set
# CONFIG_RTC_DRV_BQ32K is not set
# CONFIG_RTC_DRV_S35390A is not set
# CONFIG_RTC_DRV_FM3130 is not set
# CONFIG_RTC_DRV_RX8581 is not set
# CONFIG_RTC_DRV_RX8025 is not set

#
# SPI RTC drivers
#

#
# Platform RTC drivers
#
CONFIG_RTC_DRV_CMOS=y
# CONFIG_RTC_DRV_DS1286 is not set
# CONFIG_RTC_DRV_DS1511 is not set
# CONFIG_RTC_DRV_DS1553 is not set
# CONFIG_RTC_DRV_DS1742 is not set
# CONFIG_RTC_DRV_STK17TA8 is not set
# CONFIG_RTC_DRV_M48T86 is not set
# CONFIG_RTC_DRV_M48T35 is not set
# CONFIG_RTC_DRV_M48T59 is not set
# CONFIG_RTC_DRV_MSM6242 is not set
# CONFIG_RTC_DRV_BQ4802 is not set
# CONFIG_RTC_DRV_RP5C01 is not set
# CONFIG_RTC_DRV_V3020 is not set

#
# on-CPU RTC drivers
#
# CONFIG_DMADEVICES is not set
# CONFIG_AUXDISPLAY is not set
# CONFIG_UIO is not set
# CONFIG_STAGING is not set
# CONFIG_X86_PLATFORM_DEVICES is not set
# CONFIG_HWSPINLOCK is not set

#
# Firmware Drivers
#
# CONFIG_EDD is not set
CONFIG_FIRMWARE_MEMMAP=y
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
# CONFIG_DMIID is not set
# CONFIG_DMI_SYSFS is not set
# CONFIG_ISCSI_IBFT_FIND is not set
# CONFIG_SIGMA is not set

#
# File systems
#
# CONFIG_EXT2_FS is not set
# CONFIG_EXT3_FS is not set
CONFIG_EXT4_FS=y
# CONFIG_EXT4_USE_FOR_EXT23 is not set
# CONFIG_EXT4_FS_XATTR is not set
# CONFIG_EXT4_DEBUG is not set
CONFIG_JBD2=y
# CONFIG_JBD2_DEBUG is not set
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
CONFIG_XFS_FS=y
# CONFIG_XFS_QUOTA is not set
# CONFIG_XFS_POSIX_ACL is not set
# CONFIG_XFS_RT is not set
# CONFIG_XFS_DEBUG is not set
# CONFIG_GFS2_FS is not set
# CONFIG_BTRFS_FS is not set
# CONFIG_NILFS2_FS is not set
# CONFIG_FS_POSIX_ACL is not set
CONFIG_EXPORTFS=y
CONFIG_FILE_LOCKING=y
CONFIG_FSNOTIFY=y
CONFIG_DNOTIFY=y
CONFIG_INOTIFY_USER=y
CONFIG_FANOTIFY=y
# CONFIG_QUOTA is not set
# CONFIG_QUOTACTL is not set
# CONFIG_AUTOFS4_FS is not set
# CONFIG_FUSE_FS is not set

#
# Caches
#
# CONFIG_FSCACHE is not set

#
# CD-ROM/DVD Filesystems
#
CONFIG_ISO9660_FS=y
CONFIG_JOLIET=y
# CONFIG_ZISOFS is not set
# CONFIG_UDF_FS is not set

#
# DOS/FAT/NT Filesystems
#
CONFIG_FAT_FS=y
CONFIG_MSDOS_FS=y
CONFIG_VFAT_FS=y
CONFIG_FAT_DEFAULT_CODEPAGE=437
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
# CONFIG_NTFS_FS is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
CONFIG_PROC_KCORE=y
CONFIG_PROC_SYSCTL=y
CONFIG_PROC_PAGE_MONITOR=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_POSIX_ACL is not set
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
# CONFIG_CONFIGFS_FS is not set
# CONFIG_MISC_FILESYSTEMS is not set
# CONFIG_NETWORK_FILESYSTEMS is not set

#
# Partition Types
#
CONFIG_PARTITION_ADVANCED=y
# CONFIG_ACORN_PARTITION is not set
# CONFIG_OSF_PARTITION is not set
# CONFIG_AMIGA_PARTITION is not set
# CONFIG_ATARI_PARTITION is not set
# CONFIG_MAC_PARTITION is not set
CONFIG_MSDOS_PARTITION=y
# CONFIG_BSD_DISKLABEL is not set
# CONFIG_MINIX_SUBPARTITION is not set
# CONFIG_SOLARIS_X86_PARTITION is not set
# CONFIG_UNIXWARE_DISKLABEL is not set
# CONFIG_LDM_PARTITION is not set
# CONFIG_SGI_PARTITION is not set
# CONFIG_ULTRIX_PARTITION is not set
# CONFIG_SUN_PARTITION is not set
# CONFIG_KARMA_PARTITION is not set
CONFIG_EFI_PARTITION=y
# CONFIG_SYSV68_PARTITION is not set
CONFIG_NLS=y
CONFIG_NLS_DEFAULT="iso8859-1"
CONFIG_NLS_CODEPAGE_437=y
# CONFIG_NLS_CODEPAGE_737 is not set
# CONFIG_NLS_CODEPAGE_775 is not set
# CONFIG_NLS_CODEPAGE_850 is not set
# CONFIG_NLS_CODEPAGE_852 is not set
# CONFIG_NLS_CODEPAGE_855 is not set
# CONFIG_NLS_CODEPAGE_857 is not set
# CONFIG_NLS_CODEPAGE_860 is not set
# CONFIG_NLS_CODEPAGE_861 is not set
# CONFIG_NLS_CODEPAGE_862 is not set
# CONFIG_NLS_CODEPAGE_863 is not set
# CONFIG_NLS_CODEPAGE_864 is not set
# CONFIG_NLS_CODEPAGE_865 is not set
# CONFIG_NLS_CODEPAGE_866 is not set
# CONFIG_NLS_CODEPAGE_869 is not set
# CONFIG_NLS_CODEPAGE_936 is not set
# CONFIG_NLS_CODEPAGE_950 is not set
# CONFIG_NLS_CODEPAGE_932 is not set
# CONFIG_NLS_CODEPAGE_949 is not set
# CONFIG_NLS_CODEPAGE_874 is not set
# CONFIG_NLS_ISO8859_8 is not set
# CONFIG_NLS_CODEPAGE_1250 is not set
# CONFIG_NLS_CODEPAGE_1251 is not set
CONFIG_NLS_ASCII=y
CONFIG_NLS_ISO8859_1=y
# CONFIG_NLS_ISO8859_2 is not set
# CONFIG_NLS_ISO8859_3 is not set
# CONFIG_NLS_ISO8859_4 is not set
# CONFIG_NLS_ISO8859_5 is not set
# CONFIG_NLS_ISO8859_6 is not set
# CONFIG_NLS_ISO8859_7 is not set
# CONFIG_NLS_ISO8859_9 is not set
# CONFIG_NLS_ISO8859_13 is not set
# CONFIG_NLS_ISO8859_14 is not set
CONFIG_NLS_ISO8859_15=y
# CONFIG_NLS_KOI8_R is not set
# CONFIG_NLS_KOI8_U is not set
CONFIG_NLS_UTF8=y

#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_PRINTK_TIME is not set
CONFIG_DEFAULT_MESSAGE_LOGLEVEL=4
# CONFIG_ENABLE_WARN_DEPRECATED is not set
# CONFIG_ENABLE_MUST_CHECK is not set
CONFIG_FRAME_WARN=1024
CONFIG_MAGIC_SYSRQ=y
# CONFIG_STRIP_ASM_SYMS is not set
# CONFIG_UNUSED_SYMBOLS is not set
CONFIG_DEBUG_FS=y
# CONFIG_HEADERS_CHECK is not set
# CONFIG_DEBUG_SECTION_MISMATCH is not set
CONFIG_DEBUG_KERNEL=y
# CONFIG_DEBUG_SHIRQ is not set
# CONFIG_LOCKUP_DETECTOR is not set
# CONFIG_HARDLOCKUP_DETECTOR is not set
# CONFIG_DETECT_HUNG_TASK is not set
CONFIG_SCHED_DEBUG=y
# CONFIG_SCHEDSTATS is not set
# CONFIG_TIMER_STATS is not set
# CONFIG_DEBUG_OBJECTS is not set
# CONFIG_SLUB_STATS is not set
# CONFIG_DEBUG_KMEMLEAK is not set
CONFIG_DEBUG_PREEMPT=y
# CONFIG_DEBUG_RT_MUTEXES is not set
# CONFIG_RT_MUTEX_TESTER is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_MUTEXES is not set
# CONFIG_DEBUG_LOCK_ALLOC is not set
# CONFIG_PROVE_LOCKING is not set
# CONFIG_SPARSE_RCU_POINTER is not set
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
CONFIG_STACKTRACE=y
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_BUGVERBOSE=y
CONFIG_DEBUG_INFO=y
CONFIG_DEBUG_INFO_REDUCED=y
# CONFIG_DEBUG_VM is not set
# CONFIG_DEBUG_VIRTUAL is not set
# CONFIG_DEBUG_WRITECOUNT is not set
# CONFIG_DEBUG_MEMORY_INIT is not set
# CONFIG_DEBUG_LIST is not set
# CONFIG_TEST_LIST_SORT is not set
# CONFIG_DEBUG_SG is not set
# CONFIG_DEBUG_NOTIFIERS is not set
# CONFIG_DEBUG_CREDENTIALS is not set
CONFIG_ARCH_WANT_FRAME_POINTERS=y
CONFIG_FRAME_POINTER=y
# CONFIG_BOOT_PRINTK_DELAY is not set
# CONFIG_RCU_TORTURE_TEST is not set
CONFIG_RCU_CPU_STALL_DETECTOR=y
CONFIG_RCU_CPU_STALL_TIMEOUT=60
CONFIG_RCU_CPU_STALL_DETECTOR_RUNNABLE=y
CONFIG_RCU_CPU_STALL_VERBOSE=y
# CONFIG_BACKTRACE_SELF_TEST is not set
# CONFIG_DEBUG_BLOCK_EXT_DEVT is not set
# CONFIG_DEBUG_FORCE_WEAK_PER_CPU is not set
# CONFIG_LKDTM is not set
# CONFIG_FAULT_INJECTION is not set
# CONFIG_LATENCYTOP is not set
# CONFIG_SYSCTL_SYSCALL_CHECK is not set
# CONFIG_DEBUG_PAGEALLOC is not set
CONFIG_USER_STACKTRACE_SUPPORT=y
CONFIG_NOP_TRACER=y
CONFIG_HAVE_FTRACE_NMI_ENTER=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_FP_TEST=y
CONFIG_HAVE_FUNCTION_TRACE_MCOUNT_TEST=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_TRACER_MAX_TRACE=y
CONFIG_RING_BUFFER=y
CONFIG_FTRACE_NMI_ENTER=y
CONFIG_EVENT_TRACING=y
CONFIG_EVENT_POWER_TRACING_DEPRECATED=y
CONFIG_CONTEXT_SWITCH_TRACER=y
CONFIG_TRACING=y
CONFIG_GENERIC_TRACER=y
CONFIG_TRACING_SUPPORT=y
CONFIG_FTRACE=y
CONFIG_FUNCTION_TRACER=y
CONFIG_FUNCTION_GRAPH_TRACER=y
# CONFIG_IRQSOFF_TRACER is not set
# CONFIG_PREEMPT_TRACER is not set
CONFIG_SCHED_TRACER=y
# CONFIG_FTRACE_SYSCALLS is not set
CONFIG_BRANCH_PROFILE_NONE=y
# CONFIG_PROFILE_ANNOTATED_BRANCHES is not set
# CONFIG_PROFILE_ALL_BRANCHES is not set
# CONFIG_STACK_TRACER is not set
# CONFIG_BLK_DEV_IO_TRACE is not set
CONFIG_DYNAMIC_FTRACE=y
# CONFIG_FUNCTION_PROFILER is not set
CONFIG_FTRACE_MCOUNT_RECORD=y
# CONFIG_FTRACE_STARTUP_TEST is not set
# CONFIG_MMIOTRACE is not set
# CONFIG_RING_BUFFER_BENCHMARK is not set
# CONFIG_PROVIDE_OHCI1394_DMA_INIT is not set
# CONFIG_DYNAMIC_DEBUG is not set
# CONFIG_DMA_API_DEBUG is not set
# CONFIG_ATOMIC64_SELFTEST is not set
# CONFIG_SAMPLES is not set
CONFIG_HAVE_ARCH_KGDB=y
# CONFIG_KGDB is not set
CONFIG_HAVE_ARCH_KMEMCHECK=y
# CONFIG_TEST_KSTRTOX is not set
CONFIG_STRICT_DEVMEM=y
# CONFIG_X86_VERBOSE_BOOTUP is not set
# CONFIG_EARLY_PRINTK is not set
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set
# CONFIG_DEBUG_PER_CPU_MAPS is not set
# CONFIG_X86_PTDUMP is not set
CONFIG_DEBUG_RODATA=y
CONFIG_DEBUG_RODATA_TEST=y
# CONFIG_IOMMU_STRESS is not set
CONFIG_HAVE_MMIOTRACE_SUPPORT=y
CONFIG_IO_DELAY_TYPE_0X80=0
CONFIG_IO_DELAY_TYPE_0XED=1
CONFIG_IO_DELAY_TYPE_UDELAY=2
CONFIG_IO_DELAY_TYPE_NONE=3
CONFIG_IO_DELAY_0X80=y
# CONFIG_IO_DELAY_0XED is not set
# CONFIG_IO_DELAY_UDELAY is not set
# CONFIG_IO_DELAY_NONE is not set
CONFIG_DEFAULT_IO_DELAY_TYPE=0
# CONFIG_DEBUG_BOOT_PARAMS is not set
# CONFIG_CPA_DEBUG is not set
# CONFIG_OPTIMIZE_INLINING is not set
# CONFIG_DEBUG_STRICT_USER_COPY_CHECKS is not set

#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY_DMESG_RESTRICT is not set
# CONFIG_SECURITY is not set
# CONFIG_SECURITYFS is not set
CONFIG_DEFAULT_SECURITY_DAC=y
CONFIG_DEFAULT_SECURITY=""
# CONFIG_CRYPTO is not set
CONFIG_HAVE_KVM=y
CONFIG_HAVE_KVM_IRQCHIP=y
CONFIG_HAVE_KVM_EVENTFD=y
CONFIG_KVM_APIC_ARCHITECTURE=y
CONFIG_KVM_MMIO=y
CONFIG_KVM_ASYNC_PF=y
CONFIG_VIRTUALIZATION=y
CONFIG_KVM=y
# CONFIG_KVM_INTEL is not set
CONFIG_KVM_AMD=y
# CONFIG_KVM_MMU_AUDIT is not set
CONFIG_VHOST_NET=y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_RING=y
CONFIG_VIRTIO_PCI=y
# CONFIG_VIRTIO_BALLOON is not set
CONFIG_BINARY_PRINTF=y

#
# Library routines
#
CONFIG_BITREVERSE=y
CONFIG_GENERIC_FIND_FIRST_BIT=y
CONFIG_GENERIC_FIND_NEXT_BIT=y
CONFIG_GENERIC_FIND_LAST_BIT=y
CONFIG_CRC_CCITT=y
CONFIG_CRC16=y
# CONFIG_CRC_T10DIF is not set
# CONFIG_CRC_ITU_T is not set
CONFIG_CRC32=y
# CONFIG_CRC7 is not set
# CONFIG_LIBCRC32C is not set
# CONFIG_XZ_DEC is not set
# CONFIG_XZ_DEC_BCJ is not set
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y
CONFIG_CPU_RMAP=y
CONFIG_NLATTR=y
# CONFIG_AVERAGE is not set

--
Markus

2011-03-24 18:58:46

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-24 19:54, Markus Trippelsdorf wrote:
> On 2011.03.24 at 19:51 +0100, Jens Axboe wrote:
>> On 2011-03-24 19:36, Jens Axboe wrote:
>>> On 2011-03-24 19:30, Markus Trippelsdorf wrote:
>>>> On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
>>>>>
>>>>> This is the main pull request for the block IO layer and friends for
>>>>> 2.6.39.
>>>>
>>>> This merge results in an early oops on my system (amd64, xfs).
>>>> See the attached photo.
>>>>
>>>
>>> Auch. Can you ensure that you have CONFIG_DEBUGINFO=y in your .config
>>> and then do:
>>>
>>> $ gdb vmlinux
>>> ...
>>> l *cfq_insert_request+0x32
>>>
>>> and send that output?
>>
>> I took a closer look at the oops, and it most likely looks like q ==
>> NULL (offset 0x18 == q->elevator). You left out the Code part, so I
>> can't verify that for certain. Which makes very little sense. I take it
>> this is 100% reproducible? When you send the gdb output, please also
>> attach your .config.
>
> Yes, it's 100% reproducible here. My .config follows:

Can you try this patch and see if it makes a difference?

If you boot without the patch and add elevator=noop, does it then work?


--
Jens Axboe

2011-03-24 19:34:45

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.24 at 19:58 +0100, Jens Axboe wrote:
> On 2011-03-24 19:54, Markus Trippelsdorf wrote:
> > On 2011.03.24 at 19:51 +0100, Jens Axboe wrote:
> >> On 2011-03-24 19:36, Jens Axboe wrote:
> >>> On 2011-03-24 19:30, Markus Trippelsdorf wrote:
> >>>> On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
> >>>>>
> >>>>> This is the main pull request for the block IO layer and friends for
> >>>>> 2.6.39.
> >>>>
> >>>> This merge results in an early oops on my system (amd64, xfs).
> >>>> See the attached photo.
> >>>>
> >>>
> >>> Auch. Can you ensure that you have CONFIG_DEBUGINFO=y in your .config
> >>> and then do:
> >>>
> >>> $ gdb vmlinux
> >>> ...
> >>> l *cfq_insert_request+0x32
> >>>
> >>> and send that output?
> >>
> >> I took a closer look at the oops, and it most likely looks like q ==
> >> NULL (offset 0x18 == q->elevator). You left out the Code part, so I
> >> can't verify that for certain. Which makes very little sense. I take it
> >> this is 100% reproducible? When you send the gdb output, please also
> >> attach your .config.
> >
> > Yes, it's 100% reproducible here. My .config follows:
>
> Can you try this patch and see if it makes a difference?

There's no patch ;-)

> If you boot without the patch and add elevator=noop, does it then work?

It works insofar as the Oops is gone. But my xfs partitions apparently
still get corrupted (I had to run xfs_repair on several of them, because
they would not mount otherwise).

--
Markus

2011-03-24 19:36:23

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-24 20:34, Markus Trippelsdorf wrote:
> On 2011.03.24 at 19:58 +0100, Jens Axboe wrote:
>> On 2011-03-24 19:54, Markus Trippelsdorf wrote:
>>> On 2011.03.24 at 19:51 +0100, Jens Axboe wrote:
>>>> On 2011-03-24 19:36, Jens Axboe wrote:
>>>>> On 2011-03-24 19:30, Markus Trippelsdorf wrote:
>>>>>> On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
>>>>>>>
>>>>>>> This is the main pull request for the block IO layer and friends for
>>>>>>> 2.6.39.
>>>>>>
>>>>>> This merge results in an early oops on my system (amd64, xfs).
>>>>>> See the attached photo.
>>>>>>
>>>>>
>>>>> Auch. Can you ensure that you have CONFIG_DEBUGINFO=y in your .config
>>>>> and then do:
>>>>>
>>>>> $ gdb vmlinux
>>>>> ...
>>>>> l *cfq_insert_request+0x32
>>>>>
>>>>> and send that output?
>>>>
>>>> I took a closer look at the oops, and it most likely looks like q ==
>>>> NULL (offset 0x18 == q->elevator). You left out the Code part, so I
>>>> can't verify that for certain. Which makes very little sense. I take it
>>>> this is 100% reproducible? When you send the gdb output, please also
>>>> attach your .config.
>>>
>>> Yes, it's 100% reproducible here. My .config follows:
>>
>> Can you try this patch and see if it makes a difference?
>
> There's no patch ;-)

Oh, here it is.

>> If you boot without the patch and add elevator=noop, does it then work?
>
> It works insofar as the Oops is gone. But my xfs partitions apparently
> still get corrupted (I had to run xfs_repair on several of them, because
> they would not mount otherwise).

That's very worrying.

diff --git a/block/elevator.c b/block/elevator.c
index c387d31..77669ad 100644
--- a/block/elevator.c
+++ b/block/elevator.c
@@ -691,8 +691,10 @@ void elv_insert(struct request_queue *q, struct request *rq, int where)
* queue already, we are done - rq has now been freed,
* so no need to do anything further.
*/
+#if 0
if (elv_attempt_insert_merge(q, rq))
break;
+#endif
case ELEVATOR_INSERT_SORT:
BUG_ON(rq->cmd_type != REQ_TYPE_FS &&
!(rq->cmd_flags & REQ_DISCARD));

--
Jens Axboe

2011-03-24 19:45:51

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.24 at 20:36 +0100, Jens Axboe wrote:
> On 2011-03-24 20:34, Markus Trippelsdorf wrote:
> > On 2011.03.24 at 19:58 +0100, Jens Axboe wrote:
> >> On 2011-03-24 19:54, Markus Trippelsdorf wrote:
> >>> On 2011.03.24 at 19:51 +0100, Jens Axboe wrote:
> >>>> On 2011-03-24 19:36, Jens Axboe wrote:
> >>>>> On 2011-03-24 19:30, Markus Trippelsdorf wrote:
> >>>>>> On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
> >>>>>>>
> >>>>>>> This is the main pull request for the block IO layer and friends for
> >>>>>>> 2.6.39.
> >>>>>>
> >>>>>> This merge results in an early oops on my system (amd64, xfs).
> >>>>>> See the attached photo.
> >>>>>>
> >>>>>
> >>>>> Auch. Can you ensure that you have CONFIG_DEBUGINFO=y in your .config
> >>>>> and then do:
> >>>>>
> >>>>> $ gdb vmlinux
> >>>>> ...
> >>>>> l *cfq_insert_request+0x32
> >>>>>
> >>>>> and send that output?
> >>>>
> >>>> I took a closer look at the oops, and it most likely looks like q ==
> >>>> NULL (offset 0x18 == q->elevator). You left out the Code part, so I
> >>>> can't verify that for certain. Which makes very little sense. I take it
> >>>> this is 100% reproducible? When you send the gdb output, please also
> >>>> attach your .config.
> >>>
> >>> Yes, it's 100% reproducible here. My .config follows:
> >>
> >> Can you try this patch and see if it makes a difference?
> >
> > There's no patch ;-)
>
> Oh, here it is.
>
It doesn't help, unfortunately. I still get the same BUG at
cfq_insert_request+0x32/0x4d0.

--
Markus

2011-03-24 19:57:49

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-24 20:45, Markus Trippelsdorf wrote:
> On 2011.03.24 at 20:36 +0100, Jens Axboe wrote:
>> On 2011-03-24 20:34, Markus Trippelsdorf wrote:
>>> On 2011.03.24 at 19:58 +0100, Jens Axboe wrote:
>>>> On 2011-03-24 19:54, Markus Trippelsdorf wrote:
>>>>> On 2011.03.24 at 19:51 +0100, Jens Axboe wrote:
>>>>>> On 2011-03-24 19:36, Jens Axboe wrote:
>>>>>>> On 2011-03-24 19:30, Markus Trippelsdorf wrote:
>>>>>>>> On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
>>>>>>>>>
>>>>>>>>> This is the main pull request for the block IO layer and friends for
>>>>>>>>> 2.6.39.
>>>>>>>>
>>>>>>>> This merge results in an early oops on my system (amd64, xfs).
>>>>>>>> See the attached photo.
>>>>>>>>
>>>>>>>
>>>>>>> Auch. Can you ensure that you have CONFIG_DEBUGINFO=y in your .config
>>>>>>> and then do:
>>>>>>>
>>>>>>> $ gdb vmlinux
>>>>>>> ...
>>>>>>> l *cfq_insert_request+0x32
>>>>>>>
>>>>>>> and send that output?
>>>>>>
>>>>>> I took a closer look at the oops, and it most likely looks like q ==
>>>>>> NULL (offset 0x18 == q->elevator). You left out the Code part, so I
>>>>>> can't verify that for certain. Which makes very little sense. I take it
>>>>>> this is 100% reproducible? When you send the gdb output, please also
>>>>>> attach your .config.
>>>>>
>>>>> Yes, it's 100% reproducible here. My .config follows:
>>>>
>>>> Can you try this patch and see if it makes a difference?
>>>
>>> There's no patch ;-)
>>
>> Oh, here it is.
>>
> It doesn't help, unfortunately. I still get the same BUG at
> cfq_insert_request+0x32/0x4d0.

OK, still a data point. What was the last -git kernel you used?

--
Jens Axboe

2011-03-24 20:06:17

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.24 at 20:57 +0100, Jens Axboe wrote:
>
> OK, still a data point. What was the last -git kernel you used?

This one was the last and gave me no problems:

commit b81a618dcd3ea99de292dbe624f41ca68f464376
Merge: 2f284c8 a9712bc
Author: Linus Torvalds <[email protected]>
Date: Wed Mar 23 20:51:42 2011 -0700

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6

--
Markus

2011-03-24 21:01:26

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-24 21:06, Markus Trippelsdorf wrote:
> On 2011.03.24 at 20:57 +0100, Jens Axboe wrote:
>>
>> OK, still a data point. What was the last -git kernel you used?
>
> This one was the last and gave me no problems:
>
> commit b81a618dcd3ea99de292dbe624f41ca68f464376
> Merge: 2f284c8 a9712bc
> Author: Linus Torvalds <[email protected]>
> Date: Wed Mar 23 20:51:42 2011 -0700
>
> Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6

Puzzling... Poking at straws here so far. Does this make any difference
whatsoever?

diff --git a/fs/xfs/linux-2.6/xfs_buf.c b/fs/xfs/linux-2.6/xfs_buf.c
index c05324d..580966f 100644
--- a/fs/xfs/linux-2.6/xfs_buf.c
+++ b/fs/xfs/linux-2.6/xfs_buf.c
@@ -989,8 +989,6 @@ xfs_buf_lock(

if (atomic_read(&bp->b_pin_count) && (bp->b_flags & XBF_STALE))
xfs_log_force(bp->b_target->bt_mount, 0);
- if (atomic_read(&bp->b_io_remaining))
- blk_flush_plug(current);
down(&bp->b_sema);
XB_SET_OWNER(bp);

@@ -1439,8 +1437,6 @@ xfs_buf_iowait(
{
trace_xfs_buf_iowait(bp, _RET_IP_);

- if (atomic_read(&bp->b_io_remaining))
- blk_flush_plug(current);
wait_for_completion(&bp->b_iowait);

trace_xfs_buf_iowait_done(bp, _RET_IP_);
@@ -1943,9 +1939,6 @@ xfsbufd(
xfs_bdstrat_cb(bp);
count++;
}
- if (count)
- blk_flush_plug(current);
-
} while (!kthread_should_stop());

return 0;
@@ -1992,7 +1985,6 @@ xfs_flush_buftarg(

if (wait) {
/* Expedite and wait for IO to complete. */
- blk_flush_plug(current);
while (!list_empty(&wait_list)) {
bp = list_first_entry(&wait_list, struct xfs_buf, b_list);


--
Jens Axboe

2011-03-24 21:41:58

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.24 at 22:01 +0100, Jens Axboe wrote:
> On 2011-03-24 21:06, Markus Trippelsdorf wrote:
> > On 2011.03.24 at 20:57 +0100, Jens Axboe wrote:
> >>
> >> OK, still a data point. What was the last -git kernel you used?
> >
> > This one was the last and gave me no problems:
> >
> > commit b81a618dcd3ea99de292dbe624f41ca68f464376
> > Merge: 2f284c8 a9712bc
> > Author: Linus Torvalds <[email protected]>
> > Date: Wed Mar 23 20:51:42 2011 -0700
> >
> > Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6
>
> Puzzling... Poking at straws here so far. Does this make any difference
> whatsoever?

I will test your patch later.

Git-bisect gave me this result thus far:

9026e521c0da0731eb31f9f9022dd00cc3cd8885 is bad
82f04ab47e1d94d78503591a7460b2cad9601ede is good

When I continue the bisection with 4345caba340f051e10847924fc078ae18ed6695c
the system will start normally, but it then silently corrupts my xfs
partitions. And on next (re)boot I get this (only fixable with
xfs_repair):

BUG: unable to handle kernel NULL pointer dereference at 00000000000000f8
IP: [<ffffffff8123cb97>] xfs_cmn_err+0x27/0xc0
PGD 21c54c067 PUD 21c6bb067 PMD 0
Oops: 0000 [#1] PREEMPT SMP
last sysfs file: /sys/devices/pci0000:00/0000:00:11.0/host1/target1:0:0/1:0:0:0/block/sdb/sdb2/alignment_offset
CPU 3
Pid: 1294, comm: rm Not tainted 2.6.38-rc6-00279-g4345cab #25 System manufacturer System Product Name/M4A78T-E
RIP: 0010:[<ffffffff8123cb97>] [<ffffffff8123cb97>] xfs_cmn_err+0x27/0xc0
RSP: 0018:ffff88021c7b9ab8 EFLAGS: 00010246
RAX: ffff88021c7b9b38 RBX: ffff88021dd14118 RCX: ffffffff8167a348
RDX: 0000000000000000 RSI: ffffffff816501f0 RDI: 0000000000000008
RBP: ffff88021c7b9b28 R08: ffffffff81650119 R09: 000000000000058e
R10: 0000000000000001 R11: 0000000000012de8 R12: ffff88021dcc3340
R13: 0000000000000075 R14: ffff88021e126c80 R15: 00000000000b0208
FS: 00007fef28aec700(0000) GS:ffff8800dfd80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00000000000000f8 CR3: 000000021c5ae000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process rm (pid: 1294, threadinfo ffff88021c7b8000, task ffff88021c566710)
Stack:
ffffffff811f2362 ffff88021dcc3340 ffff88021c7b9b08 ffffffff811f7dab
000000000000ea60 ffff88021e173e00 ffff88021c7b9bb4 ffff88021c7b9bb0
ffff88021c7b9bac ffff88021e126c80 ffff88021c7b9b48 ffffffff811dadfe
Call Trace:
[<ffffffff811f2362>] ? xfs_btree_rec_addr+0x12/0x20
[<ffffffff811f7dab>] ? xfs_btree_get_rec+0x5b/0x90
[<ffffffff811dadfe>] ? xfs_alloc_get_rec+0x2e/0x70
[<ffffffff812072f0>] xfs_error_report+0x40/0x50
[<ffffffff811de274>] ? xfs_free_extent+0x94/0xc0
[<ffffffff811dc120>] xfs_free_ag_extent+0x4e0/0x7d0
[<ffffffff811de274>] xfs_free_extent+0x94/0xc0
[<ffffffff8122d4d5>] ? kmem_zone_alloc+0x85/0xd0
[<ffffffff811ee3e4>] xfs_bmap_finish+0x164/0x1b0
[<ffffffff8120e6b0>] xfs_itruncate_finish+0x150/0x3f0
[<ffffffff8122d4d5>] ? kmem_zone_alloc+0x85/0xd0
[<ffffffff8122ae46>] xfs_inactive+0x2d6/0x440
[<ffffffff812391ba>] xfs_fs_evict_inode+0xaa/0x130
[<ffffffff81133d14>] evict+0x24/0xc0
[<ffffffff81134a1b>] iput+0x1ab/0x280
[<ffffffff8112a0c6>] do_unlinkat+0x116/0x1c0
[<ffffffff811208fa>] ? sys_newfstatat+0x2a/0x40
[<ffffffff8112a192>] sys_unlinkat+0x22/0x40
[<ffffffff8103ddeb>] system_call_fastpath+0x16/0x1b
Code: 00 00 00 00 55 48 89 e5 48 83 ec 70 66 66 66 66 90 8b 05 59 d6 4b 00 4c 89 45 f0 4c 89 4d f8 85 c0 74 04 85 c7 75 3e 48 8d 45 10 <48> 8b b2 f8
00 00 00 48 8d 55 c0 48 c7 c7 ce 11 65 81 c7 45 a8
RIP [<ffffffff8123cb97>] xfs_cmn_err+0x27/0xc0
RSP <ffff88021c7b9ab8>
CR2: 00000000000000f8
---[ end trace 43fa8028bd7b575e ]--

--
Markus

2011-03-24 22:06:41

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.24 at 22:01 +0100, Jens Axboe wrote:
> On 2011-03-24 21:06, Markus Trippelsdorf wrote:
> > On 2011.03.24 at 20:57 +0100, Jens Axboe wrote:
> >>
> >> OK, still a data point. What was the last -git kernel you used?
> >
> > This one was the last and gave me no problems:
> >
> > commit b81a618dcd3ea99de292dbe624f41ca68f464376
> > Merge: 2f284c8 a9712bc
> > Author: Linus Torvalds <[email protected]>
> > Date: Wed Mar 23 20:51:42 2011 -0700
> >
> > Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6
>
> Puzzling... Poking at straws here so far. Does this make any difference
> whatsoever?

No, it doesn't make any difference at all.

--
Markus

2011-03-25 04:41:47

by Dave Chinner

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On Thu, Mar 24, 2011 at 08:34:41PM +0100, Markus Trippelsdorf wrote:
> On 2011.03.24 at 19:58 +0100, Jens Axboe wrote:
> > On 2011-03-24 19:54, Markus Trippelsdorf wrote:
> > > On 2011.03.24 at 19:51 +0100, Jens Axboe wrote:
> > >> On 2011-03-24 19:36, Jens Axboe wrote:
> > >>> On 2011-03-24 19:30, Markus Trippelsdorf wrote:
> > >>>> On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
> > >>>>>
> > >>>>> This is the main pull request for the block IO layer and friends for
> > >>>>> 2.6.39.
> > >>>>
> > >>>> This merge results in an early oops on my system (amd64, xfs).
> > >>>> See the attached photo.
> > >>>>
> > >>>
> > >>> Auch. Can you ensure that you have CONFIG_DEBUGINFO=y in your .config
> > >>> and then do:
> > >>>
> > >>> $ gdb vmlinux
> > >>> ...
> > >>> l *cfq_insert_request+0x32
> > >>>
> > >>> and send that output?
> > >>
> > >> I took a closer look at the oops, and it most likely looks like q ==
> > >> NULL (offset 0x18 == q->elevator). You left out the Code part, so I
> > >> can't verify that for certain. Which makes very little sense. I take it
> > >> this is 100% reproducible? When you send the gdb output, please also
> > >> attach your .config.
> > >
> > > Yes, it's 100% reproducible here. My .config follows:
> >
> > Can you try this patch and see if it makes a difference?
>
> There's no patch ;-)
>
> > If you boot without the patch and add elevator=noop, does it then work?
>
> It works insofar as the Oops is gone. But my xfs partitions apparently
> still get corrupted (I had to run xfs_repair on several of them, because
> they would not mount otherwise).

So the patchset is causing repeatable filesystem corruption? Sounds
to me like this series is not yet ready for mainline merging. Last
thing I want to spend the .39 cycle helping people recover busted
filesystems as a result of undercooked block layer changes...

Cheers,

Dave.
--
Dave Chinner
[email protected]

2011-03-25 07:23:54

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-24 22:41, Markus Trippelsdorf wrote:
> On 2011.03.24 at 22:01 +0100, Jens Axboe wrote:
>> On 2011-03-24 21:06, Markus Trippelsdorf wrote:
>>> On 2011.03.24 at 20:57 +0100, Jens Axboe wrote:
>>>>
>>>> OK, still a data point. What was the last -git kernel you used?
>>>
>>> This one was the last and gave me no problems:
>>>
>>> commit b81a618dcd3ea99de292dbe624f41ca68f464376
>>> Merge: 2f284c8 a9712bc
>>> Author: Linus Torvalds <[email protected]>
>>> Date: Wed Mar 23 20:51:42 2011 -0700
>>>
>>> Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6
>>
>> Puzzling... Poking at straws here so far. Does this make any difference
>> whatsoever?
>
> I will test your patch later.
>
> Git-bisect gave me this result thus far:
>
> 9026e521c0da0731eb31f9f9022dd00cc3cd8885 is bad
> 82f04ab47e1d94d78503591a7460b2cad9601ede is good
>
> When I continue the bisection with 4345caba340f051e10847924fc078ae18ed6695c
> the system will start normally, but it then silently corrupts my xfs
> partitions. And on next (re)boot I get this (only fixable with
> xfs_repair):
>
> BUG: unable to handle kernel NULL pointer dereference at 00000000000000f8
> IP: [<ffffffff8123cb97>] xfs_cmn_err+0x27/0xc0
> PGD 21c54c067 PUD 21c6bb067 PMD 0
> Oops: 0000 [#1] PREEMPT SMP
> last sysfs file: /sys/devices/pci0000:00/0000:00:11.0/host1/target1:0:0/1:0:0:0/block/sdb/sdb2/alignment_offset
> CPU 3
> Pid: 1294, comm: rm Not tainted 2.6.38-rc6-00279-g4345cab #25 System manufacturer System Product Name/M4A78T-E
> RIP: 0010:[<ffffffff8123cb97>] [<ffffffff8123cb97>] xfs_cmn_err+0x27/0xc0
> RSP: 0018:ffff88021c7b9ab8 EFLAGS: 00010246
> RAX: ffff88021c7b9b38 RBX: ffff88021dd14118 RCX: ffffffff8167a348
> RDX: 0000000000000000 RSI: ffffffff816501f0 RDI: 0000000000000008
> RBP: ffff88021c7b9b28 R08: ffffffff81650119 R09: 000000000000058e
> R10: 0000000000000001 R11: 0000000000012de8 R12: ffff88021dcc3340
> R13: 0000000000000075 R14: ffff88021e126c80 R15: 00000000000b0208
> FS: 00007fef28aec700(0000) GS:ffff8800dfd80000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> CR2: 00000000000000f8 CR3: 000000021c5ae000 CR4: 00000000000006e0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Process rm (pid: 1294, threadinfo ffff88021c7b8000, task ffff88021c566710)
> Stack:
> ffffffff811f2362 ffff88021dcc3340 ffff88021c7b9b08 ffffffff811f7dab
> 000000000000ea60 ffff88021e173e00 ffff88021c7b9bb4 ffff88021c7b9bb0
> ffff88021c7b9bac ffff88021e126c80 ffff88021c7b9b48 ffffffff811dadfe
> Call Trace:
> [<ffffffff811f2362>] ? xfs_btree_rec_addr+0x12/0x20
> [<ffffffff811f7dab>] ? xfs_btree_get_rec+0x5b/0x90
> [<ffffffff811dadfe>] ? xfs_alloc_get_rec+0x2e/0x70
> [<ffffffff812072f0>] xfs_error_report+0x40/0x50
> [<ffffffff811de274>] ? xfs_free_extent+0x94/0xc0
> [<ffffffff811dc120>] xfs_free_ag_extent+0x4e0/0x7d0
> [<ffffffff811de274>] xfs_free_extent+0x94/0xc0
> [<ffffffff8122d4d5>] ? kmem_zone_alloc+0x85/0xd0
> [<ffffffff811ee3e4>] xfs_bmap_finish+0x164/0x1b0
> [<ffffffff8120e6b0>] xfs_itruncate_finish+0x150/0x3f0
> [<ffffffff8122d4d5>] ? kmem_zone_alloc+0x85/0xd0
> [<ffffffff8122ae46>] xfs_inactive+0x2d6/0x440
> [<ffffffff812391ba>] xfs_fs_evict_inode+0xaa/0x130
> [<ffffffff81133d14>] evict+0x24/0xc0
> [<ffffffff81134a1b>] iput+0x1ab/0x280
> [<ffffffff8112a0c6>] do_unlinkat+0x116/0x1c0
> [<ffffffff811208fa>] ? sys_newfstatat+0x2a/0x40
> [<ffffffff8112a192>] sys_unlinkat+0x22/0x40
> [<ffffffff8103ddeb>] system_call_fastpath+0x16/0x1b
> Code: 00 00 00 00 55 48 89 e5 48 83 ec 70 66 66 66 66 90 8b 05 59 d6 4b 00 4c 89 45 f0 4c 89 4d f8 85 c0 74 04 85 c7 75 3e 48 8d 45 10 <48> 8b b2 f8
> 00 00 00 48 8d 55 c0 48 c7 c7 ce 11 65 81 c7 45 a8
> RIP [<ffffffff8123cb97>] xfs_cmn_err+0x27/0xc0
> RSP <ffff88021c7b9ab8>
> CR2: 00000000000000f8
> ---[ end trace 43fa8028bd7b575e ]--

How confident are you in those bisection results? Not trying to put you
on the spot, just wondering whether you tested and it's completely
consistent, or whether it was a one-off.

In any case, between those commits we the below. Since you get
corruption with noop as well as with cfq, then we can rule out the cfq
and blk-cgroup changes. I'm assuming you don't use the integrity stuff,
so that goes too. And the accounting fix is very straight forward.

Dan Carpenter (1):
block: NULL dereference on error path in __blkdev_get()

Jens Axboe (2):
fs: assign sb->s_bdi to default_backing_dev_info if the bdi is
going away
block: attempt to merge with existing requests on plug flush

Justin TerAvest (3):
cfq-iosched: Don't update group weights when on service tree
cfq-iosched: Don't set active queue in preempt
blk-cgroup: Only give unaccounted_time under debug

Martin K. Petersen (1):
block: Require subsystems to explicitly allocate bio_set integrity
mempool

Shaohua Li (1):
block: fix non-atomic access to genhd inflight structures

So the lineup should be down to these three:

Dan Carpenter (1):
block: NULL dereference on error path in __blkdev_get()

Jens Axboe (2):
fs: assign sb->s_bdi to default_backing_dev_info if the bdi is
going away
block: attempt to merge with existing requests on plug flush

Since we already tested the plug merge theory by disabling that part in
elevator.c, it's really down to the sb->s_bdi change or the NULL fix from
Dan.

The sb->s_bdi change is 95f28604a65b1c40b6c6cd95e58439cd7ded3add
The __blkdev_get() is 4345caba340f051e10847924fc078ae18ed6695c

Can you try Linus' tree and just back out both of those, then test? If
it looks good, then apply one then the other to see which one is
screwing this up.

Thanks a lot for your testing!

--
Jens Axboe

2011-03-25 07:26:30

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-25 05:41, Dave Chinner wrote:
> On Thu, Mar 24, 2011 at 08:34:41PM +0100, Markus Trippelsdorf wrote:
>> On 2011.03.24 at 19:58 +0100, Jens Axboe wrote:
>>> On 2011-03-24 19:54, Markus Trippelsdorf wrote:
>>>> On 2011.03.24 at 19:51 +0100, Jens Axboe wrote:
>>>>> On 2011-03-24 19:36, Jens Axboe wrote:
>>>>>> On 2011-03-24 19:30, Markus Trippelsdorf wrote:
>>>>>>> On 2011.03.24 at 14:43 +0100, Jens Axboe wrote:
>>>>>>>>
>>>>>>>> This is the main pull request for the block IO layer and friends for
>>>>>>>> 2.6.39.
>>>>>>>
>>>>>>> This merge results in an early oops on my system (amd64, xfs).
>>>>>>> See the attached photo.
>>>>>>>
>>>>>>
>>>>>> Auch. Can you ensure that you have CONFIG_DEBUGINFO=y in your .config
>>>>>> and then do:
>>>>>>
>>>>>> $ gdb vmlinux
>>>>>> ...
>>>>>> l *cfq_insert_request+0x32
>>>>>>
>>>>>> and send that output?
>>>>>
>>>>> I took a closer look at the oops, and it most likely looks like q ==
>>>>> NULL (offset 0x18 == q->elevator). You left out the Code part, so I
>>>>> can't verify that for certain. Which makes very little sense. I take it
>>>>> this is 100% reproducible? When you send the gdb output, please also
>>>>> attach your .config.
>>>>
>>>> Yes, it's 100% reproducible here. My .config follows:
>>>
>>> Can you try this patch and see if it makes a difference?
>>
>> There's no patch ;-)
>>
>>> If you boot without the patch and add elevator=noop, does it then work?
>>
>> It works insofar as the Oops is gone. But my xfs partitions apparently
>> still get corrupted (I had to run xfs_repair on several of them, because
>> they would not mount otherwise).
>
> So the patchset is causing repeatable filesystem corruption? Sounds
> to me like this series is not yet ready for mainline merging. Last
> thing I want to spend the .39 cycle helping people recover busted
> filesystems as a result of undercooked block layer changes...

Well, the last thing I want to do is be responsible for screwing peoples
file systems. I have been running these changes on my laptop, desktop,
and test machines for the last month at least. It's been in linux-next
for about that long, too. I'm extremely puzzled at this issue that
Markus reports.

So believe me, if we can't resolve this very quickly then we'll pull it
back out.

--
Jens Axboe

2011-03-25 08:38:04

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.25 at 08:23 +0100, Jens Axboe wrote:
> On 2011-03-24 22:41, Markus Trippelsdorf wrote:
> > On 2011.03.24 at 22:01 +0100, Jens Axboe wrote:
> >> On 2011-03-24 21:06, Markus Trippelsdorf wrote:
> >>> On 2011.03.24 at 20:57 +0100, Jens Axboe wrote:
> >>>>
> >>>> OK, still a data point. What was the last -git kernel you used?
> >>>
> >>> This one was the last and gave me no problems:
> >>>
> >>> commit b81a618dcd3ea99de292dbe624f41ca68f464376
> >>> Merge: 2f284c8 a9712bc
> >>> Author: Linus Torvalds <[email protected]>
> >>> Date: Wed Mar 23 20:51:42 2011 -0700
> >>>
> >>> Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6
> >>
> >> Puzzling... Poking at straws here so far. Does this make any difference
> >> whatsoever?
> >
> > I will test your patch later.
> >
> > Git-bisect gave me this result thus far:
> >
> > 9026e521c0da0731eb31f9f9022dd00cc3cd8885 is bad
> > 82f04ab47e1d94d78503591a7460b2cad9601ede is good
> >
> > When I continue the bisection with 4345caba340f051e10847924fc078ae18ed6695c
> > the system will start normally, but it then silently corrupts my xfs
> > partitions. And on next (re)boot I get this (only fixable with
> > xfs_repair):
> >
> How confident are you in those bisection results? Not trying to put you
> on the spot, just wondering whether you tested and it's completely
> consistent, or whether it was a one-off.

Just double checked and 82f04ab47e1d94d78503591a7460b2cad9601ede is also
bad. It just silently corrupts the file system (without a BUG) and I
didn't notice.
So back to square one.

How can I tell git-bisect just to try the commits in the block merge and
not to take wild swings in history?

--
Markus

2011-03-25 08:44:15

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-25 09:37, Markus Trippelsdorf wrote:
> On 2011.03.25 at 08:23 +0100, Jens Axboe wrote:
>> On 2011-03-24 22:41, Markus Trippelsdorf wrote:
>>> On 2011.03.24 at 22:01 +0100, Jens Axboe wrote:
>>>> On 2011-03-24 21:06, Markus Trippelsdorf wrote:
>>>>> On 2011.03.24 at 20:57 +0100, Jens Axboe wrote:
>>>>>>
>>>>>> OK, still a data point. What was the last -git kernel you used?
>>>>>
>>>>> This one was the last and gave me no problems:
>>>>>
>>>>> commit b81a618dcd3ea99de292dbe624f41ca68f464376
>>>>> Merge: 2f284c8 a9712bc
>>>>> Author: Linus Torvalds <[email protected]>
>>>>> Date: Wed Mar 23 20:51:42 2011 -0700
>>>>>
>>>>> Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6
>>>>
>>>> Puzzling... Poking at straws here so far. Does this make any difference
>>>> whatsoever?
>>>
>>> I will test your patch later.
>>>
>>> Git-bisect gave me this result thus far:
>>>
>>> 9026e521c0da0731eb31f9f9022dd00cc3cd8885 is bad
>>> 82f04ab47e1d94d78503591a7460b2cad9601ede is good
>>>
>>> When I continue the bisection with 4345caba340f051e10847924fc078ae18ed6695c
>>> the system will start normally, but it then silently corrupts my xfs
>>> partitions. And on next (re)boot I get this (only fixable with
>>> xfs_repair):
>>>
>> How confident are you in those bisection results? Not trying to put you
>> on the spot, just wondering whether you tested and it's completely
>> consistent, or whether it was a one-off.
>
> Just double checked and 82f04ab47e1d94d78503591a7460b2cad9601ede is also
> bad. It just silently corrupts the file system (without a BUG) and I
> didn't notice.
> So back to square one.
>
> How can I tell git-bisect just to try the commits in the block merge and
> not to take wild swings in history?

Something like:

$ git bisect start
$ git bisect good 3dab04e6978e358ad2307bca563fabd6c5d2c58b
$ git bisect bad 6c5103890057b1bb781b26b7aae38d33e4c517d8

--
Jens Axboe

2011-03-25 09:27:57

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.25 at 09:44 +0100, Jens Axboe wrote:
> On 2011-03-25 09:37, Markus Trippelsdorf wrote:
> > On 2011.03.25 at 08:23 +0100, Jens Axboe wrote:
> >> On 2011-03-24 22:41, Markus Trippelsdorf wrote:
> >>> On 2011.03.24 at 22:01 +0100, Jens Axboe wrote:
> >>>> On 2011-03-24 21:06, Markus Trippelsdorf wrote:
> >>>>> On 2011.03.24 at 20:57 +0100, Jens Axboe wrote:
> >>>>>>
> >>>>>> OK, still a data point. What was the last -git kernel you used?
> >>>>>
> >>>>> This one was the last and gave me no problems:
> >>>>>
> >>>>> commit b81a618dcd3ea99de292dbe624f41ca68f464376
> >>>>> Merge: 2f284c8 a9712bc
> >>>>> Author: Linus Torvalds <[email protected]>
> >>>>> Date: Wed Mar 23 20:51:42 2011 -0700
> >>>>>
> >>>>> Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6
> >>>>
> >>>> Puzzling... Poking at straws here so far. Does this make any difference
> >>>> whatsoever?
> >>>
> >>> I will test your patch later.
> >>>
> >>> Git-bisect gave me this result thus far:
> >>>
> >>> 9026e521c0da0731eb31f9f9022dd00cc3cd8885 is bad
> >>> 82f04ab47e1d94d78503591a7460b2cad9601ede is good
> >>>
> >>> When I continue the bisection with 4345caba340f051e10847924fc078ae18ed6695c
> >>> the system will start normally, but it then silently corrupts my xfs
> >>> partitions. And on next (re)boot I get this (only fixable with
> >>> xfs_repair):
> >>>
> >> How confident are you in those bisection results? Not trying to put you
> >> on the spot, just wondering whether you tested and it's completely
> >> consistent, or whether it was a one-off.
> >
> > Just double checked and 82f04ab47e1d94d78503591a7460b2cad9601ede is also
> > bad. It just silently corrupts the file system (without a BUG) and I
> > didn't notice.
> > So back to square one.
> >
> > How can I tell git-bisect just to try the commits in the block merge and
> > not to take wild swings in history?
>
> Something like:
>
> $ git bisect start
> $ git bisect good 3dab04e6978e358ad2307bca563fabd6c5d2c58b
> $ git bisect bad 6c5103890057b1bb781b26b7aae38d33e4c517d8

Thanks. To give you a flavor of the corruption here is the output of a
xfs_repair run:

# xfs_repair /dev/sda1
Phase 1 - find and verify superblock...
Phase 2 - using internal log
- zero log...
- scan filesystem freespace and inode maps...
out-of-order bno btree record 14 (132755 23) block 0/146613
block (0,132756-132756) multiply claimed by bno space tree, state - 1
agf_freeblks 8432187, counted 8432188 in ag 0
agf_freeblks 9784880, counted 9784881 in ag 3
sb_fdblocks 39149623, counted 39149625
- found root inode chunk
Phase 3 - for each AG...
- scan and clear agi unlinked lists...
- process known inodes and perform inode discovery...
- agno = 0
imap claims a free inode 2106112 is in use, correcting imap and clearing inode
cleared inode 2106112
imap claims a free inode 2106114 is in use, correcting imap and clearing inode
cleared inode 2106114
imap claims a free inode 2106115 is in use, correcting imap and clearing inode
cleared inode 2106115
- agno = 1
imap claims a free inode 268641903 is in use, correcting imap and clearing inode
cleared inode 268641903
imap claims a free inode 268711919 is in use, correcting imap and clearing inode
cleared inode 268711919
data fork in ino 289942300 claims free block 34476047
data fork in ino 289942300 claims free block 34476048
data fork in ino 295265710 claims free block 34469734
data fork in ino 295265735 claims free block 34469461
- agno = 2
imap claims a free inode 548585504 is in use, correcting imap and clearing inode
cleared inode 548585504
imap claims a free inode 548585509 is in use, correcting imap and clearing inode
cleared inode 548585509
imap claims a free inode 548585510 is in use, correcting imap and clearing inode
cleared inode 548585510
imap claims a free inode 548923508 is in use, correcting imap and clearing inode
cleared inode 548923508
imap claims a free inode 548923509 is in use, correcting imap and clearing inode
cleared inode 548923509
imap claims a free inode 570640866 is in use, correcting imap and clearing inode
cleared inode 570640866
- agno = 3
imap claims a free inode 845126191 is in use, correcting imap and clearing inode
cleared inode 845126191
imap claims a free inode 845130982 is in use, correcting imap and clearing inode
cleared inode 845130982
- process newly discovered inodes...
Phase 4 - check for duplicate blocks...
- setting up duplicate extent list...
- check for inodes claiming duplicate blocks...
- agno = 3
- agno = 1
- agno = 0
- agno = 2
entry "fcron.pid" at block 0 offset 1144 in directory inode 199
references free inode 2106112
clearing inode number in entry at offset 1144...
entry "fcron.fifo" at block 0 offset 1168 in directory inode 199
references free inode 2106114
clearing inode number in entry at offset 1168...
entry "syslog-ng.pid" at block 0 offset 1192 in directory inode 199
references free inode 2106115
clearing inode number in entry at offset 1192...
Phase 5 - rebuild AG headers and trees...
- reset superblock...
Phase 6 - check inode connectivity...
- resetting contents of realtime bitmap and summary inodes
- traversing filesystem ...
bad hash table for directory inode 199 (no data entry): rebuilding
rebuilding directory inode 199
- traversal finished ...
- moving disconnected inodes to lost+found ...
Phase 7 - verify and correct link counts...
done


--
Markus

2011-03-25 09:57:09

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.25 at 09:44 +0100, Jens Axboe wrote:
> On 2011-03-25 09:37, Markus Trippelsdorf wrote:
> > On 2011.03.25 at 08:23 +0100, Jens Axboe wrote:
> >> On 2011-03-24 22:41, Markus Trippelsdorf wrote:
> >>> On 2011.03.24 at 22:01 +0100, Jens Axboe wrote:
> >>>> On 2011-03-24 21:06, Markus Trippelsdorf wrote:
> >>>>> On 2011.03.24 at 20:57 +0100, Jens Axboe wrote:
> >>>>>>
> >>>>>> OK, still a data point. What was the last -git kernel you used?
> >>>>>
> >>>>> This one was the last and gave me no problems:
> >>>>>
> >>>>> commit b81a618dcd3ea99de292dbe624f41ca68f464376
> >>>>> Merge: 2f284c8 a9712bc
> >>>>> Author: Linus Torvalds <[email protected]>
> >>>>> Date: Wed Mar 23 20:51:42 2011 -0700
> >>>>>
> >>>>> Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6
> >>>>
> >>>> Puzzling... Poking at straws here so far. Does this make any difference
> >>>> whatsoever?
> >>>
> >>> I will test your patch later.
> >>>
> >>> Git-bisect gave me this result thus far:
> >>>
> >>> 9026e521c0da0731eb31f9f9022dd00cc3cd8885 is bad
> >>> 82f04ab47e1d94d78503591a7460b2cad9601ede is good
> >>>
> >>> When I continue the bisection with 4345caba340f051e10847924fc078ae18ed6695c
> >>> the system will start normally, but it then silently corrupts my xfs
> >>> partitions. And on next (re)boot I get this (only fixable with
> >>> xfs_repair):
> >>>
> >> How confident are you in those bisection results? Not trying to put you
> >> on the spot, just wondering whether you tested and it's completely
> >> consistent, or whether it was a one-off.
> >
> > Just double checked and 82f04ab47e1d94d78503591a7460b2cad9601ede is also
> > bad. It just silently corrupts the file system (without a BUG) and I
> > didn't notice.
> > So back to square one.
> >
> > How can I tell git-bisect just to try the commits in the block merge and
> > not to take wild swings in history?
>
> Something like:
>
> $ git bisect start
> $ git bisect good 3dab04e6978e358ad2307bca563fabd6c5d2c58b
> $ git bisect bad 6c5103890057b1bb781b26b7aae38d33e4c517d8

Ok this time I've found the commit:

9b6096a65f99a89dfd8328c4e469e7b53b3ae04a is the first bad commit
commit 9b6096a65f99a89dfd8328c4e469e7b53b3ae04a
Author: Shaohua Li <[email protected]>
Date: Thu Mar 17 10:47:06 2011 +0100

mm: make generic_writepages() use plugging

This recovers a performance regression caused by the removal
of the per-device plugging.

Signed-off-by: Jens Axboe <[email protected]>

Reverting it solves all problems here.

--
Markus

2011-03-25 10:11:45

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-25 10:57, Markus Trippelsdorf wrote:
> On 2011.03.25 at 09:44 +0100, Jens Axboe wrote:
>> On 2011-03-25 09:37, Markus Trippelsdorf wrote:
>>> On 2011.03.25 at 08:23 +0100, Jens Axboe wrote:
>>>> On 2011-03-24 22:41, Markus Trippelsdorf wrote:
>>>>> On 2011.03.24 at 22:01 +0100, Jens Axboe wrote:
>>>>>> On 2011-03-24 21:06, Markus Trippelsdorf wrote:
>>>>>>> On 2011.03.24 at 20:57 +0100, Jens Axboe wrote:
>>>>>>>>
>>>>>>>> OK, still a data point. What was the last -git kernel you used?
>>>>>>>
>>>>>>> This one was the last and gave me no problems:
>>>>>>>
>>>>>>> commit b81a618dcd3ea99de292dbe624f41ca68f464376
>>>>>>> Merge: 2f284c8 a9712bc
>>>>>>> Author: Linus Torvalds <[email protected]>
>>>>>>> Date: Wed Mar 23 20:51:42 2011 -0700
>>>>>>>
>>>>>>> Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6
>>>>>>
>>>>>> Puzzling... Poking at straws here so far. Does this make any difference
>>>>>> whatsoever?
>>>>>
>>>>> I will test your patch later.
>>>>>
>>>>> Git-bisect gave me this result thus far:
>>>>>
>>>>> 9026e521c0da0731eb31f9f9022dd00cc3cd8885 is bad
>>>>> 82f04ab47e1d94d78503591a7460b2cad9601ede is good
>>>>>
>>>>> When I continue the bisection with 4345caba340f051e10847924fc078ae18ed6695c
>>>>> the system will start normally, but it then silently corrupts my xfs
>>>>> partitions. And on next (re)boot I get this (only fixable with
>>>>> xfs_repair):
>>>>>
>>>> How confident are you in those bisection results? Not trying to put you
>>>> on the spot, just wondering whether you tested and it's completely
>>>> consistent, or whether it was a one-off.
>>>
>>> Just double checked and 82f04ab47e1d94d78503591a7460b2cad9601ede is also
>>> bad. It just silently corrupts the file system (without a BUG) and I
>>> didn't notice.
>>> So back to square one.
>>>
>>> How can I tell git-bisect just to try the commits in the block merge and
>>> not to take wild swings in history?
>>
>> Something like:
>>
>> $ git bisect start
>> $ git bisect good 3dab04e6978e358ad2307bca563fabd6c5d2c58b
>> $ git bisect bad 6c5103890057b1bb781b26b7aae38d33e4c517d8
>
> Ok this time I've found the commit:
>
> 9b6096a65f99a89dfd8328c4e469e7b53b3ae04a is the first bad commit
> commit 9b6096a65f99a89dfd8328c4e469e7b53b3ae04a
> Author: Shaohua Li <[email protected]>
> Date: Thu Mar 17 10:47:06 2011 +0100
>
> mm: make generic_writepages() use plugging
>
> This recovers a performance regression caused by the removal
> of the per-device plugging.
>
> Signed-off-by: Jens Axboe <[email protected]>
>
> Reverting it solves all problems here.

Great! I'll see if I can find out why. The patch itself should not cause
problems. I'm suspecting some interaction with nested plugging and the
auto-flush.

--
Jens Axboe

2011-03-25 11:59:09

by Theodore Ts'o

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops


On Mar 25, 2011, at 12:41 AM, Dave Chinner wrote:

>>
>> It works insofar as the Oops is gone. But my xfs partitions apparently
>> still get corrupted (I had to run xfs_repair on several of them, because
>> they would not mount otherwise).
>
> So the patchset is causing repeatable filesystem corruption? Sounds
> to me like this series is not yet ready for mainline merging. Last
> thing I want to spend the .39 cycle helping people recover busted
> filesystems as a result of undercooked block layer changes...

FYI. I did a trial merge last night of the ext4 changes last night with
the tip of Linus's tree. The ext4 changes (based on 2.6.38-rc5)
survived xfstests -g auto before I merged in Linus's 2.6.39 master
branch. After I merged with 2.6.39-tip, I reran xfstests, and it got
past test #13 (fsstress), which normally means that everything is
OK, so I sent a pull request to Linus. Much later, (-g auto takes a
long time) I got an OOPS inside the virtio driver. Ext4 was nowhere
in the stack trace, but of course the block layer was. Grumbling
that someone had broke virtio during the merge window, I switched
my KVM setup to use SATA emulation and used the sda devices
instead. This time I got an oops in the block I/O layer, again quite
late in xfstests. Somewhere around test #224 or so if I remember
correctly.

It was too late last night to do any more investigating, which is why
I hadn't sent a formal report yet, but next up is for me to retry xfstests
before merging in my changes, and then to start a git bisect.

So before accusing some patch series which hasn't been merged
into 2.6.39 yet, you might want to also worry about some change
that already has been merged. Of course the symptoms for me are
quite different. I'm not seeing an early oops, but only something
which shows up when the the system is put under a lot of stress
by xfstests. So it could be a different problem....

- Ted

P.S. And of course there is the chance that there is some
subtle bug in the ext4 branch, which worked just fine when
it was just based on 2.6.38-rc5, but which only manifested
itself when I merged in the tip of Linus's branch. So I'm not
__accusing__ the block layer yet, even though the stack traces
seem to point that way, because I don't have a smoking gun
yet. But I do have to admit I'm suspicious....

2011-03-25 12:14:43

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-25 12:59, Theodore Tso wrote:
>
> On Mar 25, 2011, at 12:41 AM, Dave Chinner wrote:
>
>>>
>>> It works insofar as the Oops is gone. But my xfs partitions apparently
>>> still get corrupted (I had to run xfs_repair on several of them, because
>>> they would not mount otherwise).
>>
>> So the patchset is causing repeatable filesystem corruption? Sounds
>> to me like this series is not yet ready for mainline merging. Last
>> thing I want to spend the .39 cycle helping people recover busted
>> filesystems as a result of undercooked block layer changes...
>
> FYI. I did a trial merge last night of the ext4 changes last night with
> the tip of Linus's tree. The ext4 changes (based on 2.6.38-rc5)
> survived xfstests -g auto before I merged in Linus's 2.6.39 master
> branch. After I merged with 2.6.39-tip, I reran xfstests, and it got
> past test #13 (fsstress), which normally means that everything is
> OK, so I sent a pull request to Linus. Much later, (-g auto takes a
> long time) I got an OOPS inside the virtio driver. Ext4 was nowhere
> in the stack trace, but of course the block layer was. Grumbling
> that someone had broke virtio during the merge window, I switched
> my KVM setup to use SATA emulation and used the sda devices
> instead. This time I got an oops in the block I/O layer, again quite
> late in xfstests. Somewhere around test #224 or so if I remember
> correctly.
>
> It was too late last night to do any more investigating, which is why
> I hadn't sent a formal report yet, but next up is for me to retry xfstests
> before merging in my changes, and then to start a git bisect.
>
> So before accusing some patch series which hasn't been merged
> into 2.6.39 yet, you might want to also worry about some change
> that already has been merged. Of course the symptoms for me are
> quite different. I'm not seeing an early oops, but only something
> which shows up when the the system is put under a lot of stress
> by xfstests. So it could be a different problem....
>
> - Ted
>
> P.S. And of course there is the chance that there is some
> subtle bug in the ext4 branch, which worked just fine when
> it was just based on 2.6.38-rc5, but which only manifested
> itself when I merged in the tip of Linus's branch. So I'm not
> __accusing__ the block layer yet, even though the stack traces
> seem to point that way, because I don't have a smoking gun
> yet. But I do have to admit I'm suspicious....

But this plugging change is merged, so it is a very likely candidate.
With the oddness going on, I suspect that we end up flushing a plug that
resides on a stack that is no longer valid.

Is there a way to check whether a given pointer is valid on the current
stack for this process?

I think we can rule out stack overflows, since the plug context itself
is very small (28 bytes). But if we have something like:

blk_start_plug(&plug1);
...
blk_start_plug(&plug2);
...
flush(&plug2);

then that could explain the corruption and lockups.

So I'd really like to have something ala:

if (is_str_ptr_valid(current, ptr, size))
...

to aid the debugging.

--
Jens Axboe

2011-03-25 12:34:15

by Theodore Ts'o

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On Fri, Mar 25, 2011 at 01:14:36PM +0100, Jens Axboe wrote:
> But this plugging change is merged, so it is a very likely candidate.
> With the oddness going on, I suspect that we end up flushing a plug that
> resides on a stack that is no longer valid.

Ah, sorry, I was typing this while I was still doing my morning e-mail
reading/deleting in bed, so I couldn't check to see if it had been
merged already.

I'm downstairs now, and you could very well be right, since here's the
the stack trace from the kvm console output, and it features
flush_plug_list very prominently in the stack trace (see attached).

> So I'd really like to have something ala:
>
> if (is_str_ptr_valid(current, ptr, size))
> ...
>
> to aid the debugging.

Well, I can repro the problem in an hour or two of running xfstests
under KVM, so I'm happy to try any debugging patches. Let me know
where you'd like that placed, and I'll do a run.

- Ted

P.S. Hmm, and I this one does have ext4 in the stack trace. My bad;
I misremembered. The other one didn't, though. (Oh, and this one
died in test #130, but if it's related to some kind of plug/unplug
race, it's probably quite random when it happens.)

BUG: unable to handle kernel NULL pointer dereference at 0000000c
IP: [<c036429a>] cfq_insert_request+0x39/0x4a2
*pdpt = 00000000017f0001 *pde = 0000000000000000
Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
last sysfs file:
Modules linked in:

Pid: 15723, comm: xfs_io Not tainted 2.6.38-08184-g108052f #1489 Bochs Bochs
EIP: 0060:[<c036429a>] EFLAGS: 00010046 CPU: 0
EIP is at cfq_insert_request+0x39/0x4a2
EAX: 00000000 EBX: 00000000 ECX: c096e710 EDX: cb599870
ESI: f596c000 EDI: cb599870 EBP: f6db3bfc ESP: f6db3bd0
DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
Process xfs_io (pid: 15723, ti=f6db2000 task=c5f6e7c0 task.ti=f6db2000)
Stack:
f6db3c04 00000046 00000028 f6db3c14 00000046 00000000 c03573c8 c03573c8
cb599870 f5839d60 f6db3e84 f6db3c14 c0354613 00000006 02a00091 f5839d60
f6db3e84 f6db3c24 c03546a9 cb599870 f5839d60 f6db3c40 c03573dd f6db3e88
Call Trace:
[<c03573c8>] ? flush_plug_list+0xa5/0x114
[<c03573c8>] ? flush_plug_list+0xa5/0x114
[<c0354613>] elv_insert+0x173/0x1a6
[<c03546a9>] __elv_add_request+0x63/0x67
[<c03573dd>] flush_plug_list+0xba/0x114
[<c035744c>] __blk_flush_plug+0x15/0x31
[<c0686899>] schedule+0x25b/0x6d9
[<c017c71d>] ? sched_clock_cpu+0x134/0x144
[<c03594f4>] ? __make_request+0x1fa/0x231
[<c06871ab>] schedule_timeout+0x1b/0x9b
[<c01876df>] ? mark_lock+0x1e/0x1df
[<c01878e3>] ? mark_held_locks+0x43/0x5b
[<c0688d3a>] ? _raw_spin_unlock_irq+0x27/0x30
[<c0187b3f>] ? trace_hardirqs_on_caller+0x104/0x125
[<c0187b6b>] ? trace_hardirqs_on+0xb/0xd
[<c0687042>] wait_for_common+0xc6/0x11f
[<c015d4d4>] ? default_wake_function+0x0/0x12
[<c068713a>] wait_for_completion+0x17/0x19
[<c035a837>] blkdev_issue_flush+0x96/0xcf
[<c0686fb2>] ? wait_for_common+0x36/0x11f
[<c025c06d>] ext4_sync_file+0x205/0x250
[<c021a04b>] vfs_fsync_range+0x46/0x68
[<c021a0c2>] generic_write_sync+0x55/0x64
[<c01cfd78>] generic_file_aio_write+0x99/0xb8
[<c025bd7c>] ext4_file_write+0x1df/0x22f
[<c01fbbd0>] do_sync_write+0x8f/0xca
[<c033622f>] ? security_file_permission+0x27/0x2b
[<c01fbd9f>] ? rw_verify_area+0xca/0xed
[<c01fbb41>] ? do_sync_write+0x0/0xca
[<c01fc51c>] vfs_write+0x85/0xe3
[<c0688ef6>] ? restore_all_notrace+0x0/0x18
[<c01fc5c2>] sys_pwrite64+0x48/0x61
[<c0688ebd>] syscall_call+0x7/0xb
Code: 8b 40 0c 8b 5a 5c 89 d7 8b 70 04 8b 06 8b 80 74 04 00 00 85 c0 74 11 ff 73 74 68 33 0a 87 c0 50 e8 ce a7 e5 ff 83 c4 0c 8b 47 58 <8b> 50 0c 89 d8 e8 68 f0 ff ff 8b 57 20 a1 40 6a 92 c0 83 e2 11
EIP: [<c036429a>] cfq_insert_request+0x39/0x4a2 SS:ESP 0068:f6db3bd0
CR2: 000000000000000c
---[ end trace da176424940e586f ]---

2011-03-25 12:43:34

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-25 13:33, Ted Ts'o wrote:
> On Fri, Mar 25, 2011 at 01:14:36PM +0100, Jens Axboe wrote:
>> But this plugging change is merged, so it is a very likely candidate.
>> With the oddness going on, I suspect that we end up flushing a plug that
>> resides on a stack that is no longer valid.
>
> Ah, sorry, I was typing this while I was still doing my morning e-mail
> reading/deleting in bed, so I couldn't check to see if it had been
> merged already.
>
> I'm downstairs now, and you could very well be right, since here's the
> the stack trace from the kvm console output, and it features
> flush_plug_list very prominently in the stack trace (see attached).
>
>> So I'd really like to have something ala:
>>
>> if (is_str_ptr_valid(current, ptr, size))
>> ...
>>
>> to aid the debugging.
>
> Well, I can repro the problem in an hour or two of running xfstests
> under KVM, so I'm happy to try any debugging patches. Let me know
> where you'd like that placed, and I'll do a run.

Tests are running here locally too, so far not much luck. Can you give
the patch at the end of this email a whirl?

> - Ted
>
> P.S. Hmm, and I this one does have ext4 in the stack trace. My bad;
> I misremembered. The other one didn't, though. (Oh, and this one
> died in test #130, but if it's related to some kind of plug/unplug
> race, it's probably quite random when it happens.)
>
> BUG: unable to handle kernel NULL pointer dereference at 0000000c
> IP: [<c036429a>] cfq_insert_request+0x39/0x4a2
> *pdpt = 00000000017f0001 *pde = 0000000000000000
> Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
> last sysfs file:
> Modules linked in:
>
> Pid: 15723, comm: xfs_io Not tainted 2.6.38-08184-g108052f #1489 Bochs Bochs
> EIP: 0060:[<c036429a>] EFLAGS: 00010046 CPU: 0
> EIP is at cfq_insert_request+0x39/0x4a2
> EAX: 00000000 EBX: 00000000 ECX: c096e710 EDX: cb599870
> ESI: f596c000 EDI: cb599870 EBP: f6db3bfc ESP: f6db3bd0
> DS: 007b ES: 007b FS: 00d8 GS: 0033 SS: 0068
> Process xfs_io (pid: 15723, ti=f6db2000 task=c5f6e7c0 task.ti=f6db2000)
> Stack:
> f6db3c04 00000046 00000028 f6db3c14 00000046 00000000 c03573c8 c03573c8
> cb599870 f5839d60 f6db3e84 f6db3c14 c0354613 00000006 02a00091 f5839d60
> f6db3e84 f6db3c24 c03546a9 cb599870 f5839d60 f6db3c40 c03573dd f6db3e88
> Call Trace:
> [<c03573c8>] ? flush_plug_list+0xa5/0x114
> [<c03573c8>] ? flush_plug_list+0xa5/0x114
> [<c0354613>] elv_insert+0x173/0x1a6
> [<c03546a9>] __elv_add_request+0x63/0x67
> [<c03573dd>] flush_plug_list+0xba/0x114
> [<c035744c>] __blk_flush_plug+0x15/0x31
> [<c0686899>] schedule+0x25b/0x6d9
> [<c017c71d>] ? sched_clock_cpu+0x134/0x144
> [<c03594f4>] ? __make_request+0x1fa/0x231
> [<c06871ab>] schedule_timeout+0x1b/0x9b
> [<c01876df>] ? mark_lock+0x1e/0x1df
> [<c01878e3>] ? mark_held_locks+0x43/0x5b
> [<c0688d3a>] ? _raw_spin_unlock_irq+0x27/0x30
> [<c0187b3f>] ? trace_hardirqs_on_caller+0x104/0x125
> [<c0187b6b>] ? trace_hardirqs_on+0xb/0xd
> [<c0687042>] wait_for_common+0xc6/0x11f
> [<c015d4d4>] ? default_wake_function+0x0/0x12
> [<c068713a>] wait_for_completion+0x17/0x19
> [<c035a837>] blkdev_issue_flush+0x96/0xcf
> [<c0686fb2>] ? wait_for_common+0x36/0x11f
> [<c025c06d>] ext4_sync_file+0x205/0x250
> [<c021a04b>] vfs_fsync_range+0x46/0x68
> [<c021a0c2>] generic_write_sync+0x55/0x64
> [<c01cfd78>] generic_file_aio_write+0x99/0xb8
> [<c025bd7c>] ext4_file_write+0x1df/0x22f
> [<c01fbbd0>] do_sync_write+0x8f/0xca
> [<c033622f>] ? security_file_permission+0x27/0x2b
> [<c01fbd9f>] ? rw_verify_area+0xca/0xed
> [<c01fbb41>] ? do_sync_write+0x0/0xca
> [<c01fc51c>] vfs_write+0x85/0xe3
> [<c0688ef6>] ? restore_all_notrace+0x0/0x18
> [<c01fc5c2>] sys_pwrite64+0x48/0x61
> [<c0688ebd>] syscall_call+0x7/0xb
> Code: 8b 40 0c 8b 5a 5c 89 d7 8b 70 04 8b 06 8b 80 74 04 00 00 85 c0 74 11 ff 73 74 68 33 0a 87 c0 50 e8 ce a7 e5 ff 83 c4 0c 8b 47 58 <8b> 50 0c 89 d8 e8 68 f0 ff ff 8b 57 20 a1 40 6a 92 c0 83 e2 11
> EIP: [<c036429a>] cfq_insert_request+0x39/0x4a2 SS:ESP 0068:f6db3bd0
> CR2: 000000000000000c
> ---[ end trace da176424940e586f ]---

This looks somewhat similar to what Markus is reporting, essentially the
badness is coming out of the flush-on-schedule. Whether that is what
triggers this bug or if it's just a normal place to flush the plug, hard
to say. For this particular case it looks quite normal -
wait_for_completion() will block, so we flush it out.


diff --git a/block/blk-core.c b/block/blk-core.c
index 59b5c00..8906ff1 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -1197,6 +1197,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
if (!plug)
goto out;

+ preempt_disable();
list_for_each_entry_reverse(rq, &plug->list, queuelist) {
int el_ret;

@@ -1214,6 +1215,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
break;
}
}
+ preempt_enable();
out:
return ret;
}

--
Jens Axboe

2011-03-25 12:44:18

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-25 10:57, Markus Trippelsdorf wrote:
> On 2011.03.25 at 09:44 +0100, Jens Axboe wrote:
>> On 2011-03-25 09:37, Markus Trippelsdorf wrote:
>>> On 2011.03.25 at 08:23 +0100, Jens Axboe wrote:
>>>> On 2011-03-24 22:41, Markus Trippelsdorf wrote:
>>>>> On 2011.03.24 at 22:01 +0100, Jens Axboe wrote:
>>>>>> On 2011-03-24 21:06, Markus Trippelsdorf wrote:
>>>>>>> On 2011.03.24 at 20:57 +0100, Jens Axboe wrote:
>>>>>>>>
>>>>>>>> OK, still a data point. What was the last -git kernel you used?
>>>>>>>
>>>>>>> This one was the last and gave me no problems:
>>>>>>>
>>>>>>> commit b81a618dcd3ea99de292dbe624f41ca68f464376
>>>>>>> Merge: 2f284c8 a9712bc
>>>>>>> Author: Linus Torvalds <[email protected]>
>>>>>>> Date: Wed Mar 23 20:51:42 2011 -0700
>>>>>>>
>>>>>>> Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs-2.6
>>>>>>
>>>>>> Puzzling... Poking at straws here so far. Does this make any difference
>>>>>> whatsoever?
>>>>>
>>>>> I will test your patch later.
>>>>>
>>>>> Git-bisect gave me this result thus far:
>>>>>
>>>>> 9026e521c0da0731eb31f9f9022dd00cc3cd8885 is bad
>>>>> 82f04ab47e1d94d78503591a7460b2cad9601ede is good
>>>>>
>>>>> When I continue the bisection with 4345caba340f051e10847924fc078ae18ed6695c
>>>>> the system will start normally, but it then silently corrupts my xfs
>>>>> partitions. And on next (re)boot I get this (only fixable with
>>>>> xfs_repair):
>>>>>
>>>> How confident are you in those bisection results? Not trying to put you
>>>> on the spot, just wondering whether you tested and it's completely
>>>> consistent, or whether it was a one-off.
>>>
>>> Just double checked and 82f04ab47e1d94d78503591a7460b2cad9601ede is also
>>> bad. It just silently corrupts the file system (without a BUG) and I
>>> didn't notice.
>>> So back to square one.
>>>
>>> How can I tell git-bisect just to try the commits in the block merge and
>>> not to take wild swings in history?
>>
>> Something like:
>>
>> $ git bisect start
>> $ git bisect good 3dab04e6978e358ad2307bca563fabd6c5d2c58b
>> $ git bisect bad 6c5103890057b1bb781b26b7aae38d33e4c517d8
>
> Ok this time I've found the commit:
>
> 9b6096a65f99a89dfd8328c4e469e7b53b3ae04a is the first bad commit
> commit 9b6096a65f99a89dfd8328c4e469e7b53b3ae04a
> Author: Shaohua Li <[email protected]>
> Date: Thu Mar 17 10:47:06 2011 +0100
>
> mm: make generic_writepages() use plugging
>
> This recovers a performance regression caused by the removal
> of the per-device plugging.
>
> Signed-off-by: Jens Axboe <[email protected]>
>
> Reverting it solves all problems here.

Can you try this one?

diff --git a/block/blk-core.c b/block/blk-core.c
index 59b5c00..8906ff1 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -1197,6 +1197,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
if (!plug)
goto out;

+ preempt_disable();
list_for_each_entry_reverse(rq, &plug->list, queuelist) {
int el_ret;

@@ -1214,6 +1215,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
break;
}
}
+ preempt_enable();
out:
return ret;
}

--
Jens Axboe

2011-03-25 13:02:58

by Chris Mason

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

Excerpts from Dave Chinner's message of 2011-03-25 00:41:28 -0400:
> On Thu, Mar 24, 2011 at 08:34:41PM +0100, Markus Trippelsdorf wrote:

[ oopsen and fs corruptions in linus master ]

>
> So the patchset is causing repeatable filesystem corruption? Sounds
> to me like this series is not yet ready for mainline merging. Last
> thing I want to spend the .39 cycle helping people recover busted
> filesystems as a result of undercooked block layer changes...
>

I tried to reproduce this, but didn't get farther than make
modules_install. My setup works like this:

build new kernel
start virtual machine off new kernel
make modules_install over NFS to get new modules (just the filesystems)

make modules_install instantly did this:
BUG: spinlock recursion on CPU#0, as/2014
lock: ffff88006e939d08, .magic: dead4ead, .owner: as/2014, .owner_cpu: 0
Pid: 2014, comm: as Tainted: G W 2.6.38-josef+ #228
Call Trace:
[<ffffffff812712e5>] spin_bug+0x9c/0xa3
[<ffffffff81271333>] do_raw_spin_lock+0x47/0x13c
[<ffffffff81068329>] ? get_parent_ip+0x11/0x41
[<ffffffff815dd796>] _raw_spin_lock+0x23/0x27
[<ffffffff811258f9>] igrab+0x1b/0x48
[<ffffffff811b2249>] nfs_updatepage+0x2ac/0x469
[<ffffffff811a4eff>] nfs_write_end+0x26d/0x298
[<ffffffff81068329>] ? get_parent_ip+0x11/0x41
[<ffffffff810d43b7>] generic_file_buffered_write+0x189/0x26a
[<ffffffff810db611>] ? __alloc_pages_nodemask+0x14d/0x7a8
[<ffffffff810d5b2e>] __generic_file_aio_write+0x32b/0x35b
[<ffffffff810d5bbf>] generic_file_aio_write+0x61/0xba
[<ffffffff811a5ab7>] nfs_file_write+0xde/0x161
[<ffffffff8110f3ba>] do_sync_write+0xcb/0x108
[<ffffffff810f78d4>] ? do_mmap_pgoff+0x29a/0x2f4
[<ffffffff8110f9f7>] vfs_write+0xb5/0x131
[<ffffffff8110fc5c>] sys_write+0x4a/0x71
[<ffffffff815e34ab>] system_call_fastpath+0x16/0x1b

-chris

2011-03-25 13:10:03

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.25 at 13:44 +0100, Jens Axboe wrote:
> On 2011-03-25 10:57, Markus Trippelsdorf wrote:
> >
> > Reverting it solves all problems here.
>
> Can you try this one?

This one doesn't help; I still get the same BUG.

BTW if you're having trouble reproducing this, here is the only non
stock xfs option that I use on the affected partitions:

noatime,logbsize=262144

--
Markus

2011-03-25 14:10:52

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-25 14:09, Markus Trippelsdorf wrote:
> On 2011.03.25 at 13:44 +0100, Jens Axboe wrote:
>> On 2011-03-25 10:57, Markus Trippelsdorf wrote:
>>>
>>> Reverting it solves all problems here.
>>
>> Can you try this one?
>
> This one doesn't help; I still get the same BUG.
>
> BTW if you're having trouble reproducing this, here is the only non
> stock xfs option that I use on the affected partitions:
>
> noatime,logbsize=262144

This?

diff --git a/block/blk-core.c b/block/blk-core.c
index 59b5c00..121df87 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -1197,6 +1197,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
if (!plug)
goto out;

+ preempt_disable();
list_for_each_entry_reverse(rq, &plug->list, queuelist) {
int el_ret;

@@ -1214,6 +1215,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
break;
}
}
+ preempt_enable();
out:
return ret;
}
@@ -1322,7 +1324,9 @@ get_rq:
* Debug flag, kill later
*/
req->cmd_flags |= REQ_ON_PLUG;
+ preempt_disable();
list_add_tail(&req->queuelist, &plug->list);
+ preempt_enable();
drive_stat_acct(req, 1);
} else {
spin_lock_irq(q->queue_lock);

--
Jens Axboe

2011-03-25 14:14:19

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.25 at 15:10 +0100, Jens Axboe wrote:
> On 2011-03-25 14:09, Markus Trippelsdorf wrote:
> > On 2011.03.25 at 13:44 +0100, Jens Axboe wrote:
> >> On 2011-03-25 10:57, Markus Trippelsdorf wrote:
> >>>
> >>> Reverting it solves all problems here.
> >>
> >> Can you try this one?
> >
> > This one doesn't help; I still get the same BUG.
> >
> > BTW if you're having trouble reproducing this, here is the only non
> > stock xfs option that I use on the affected partitions:
> >
> > noatime,logbsize=262144
>
> This?

No.

--
Markus

2011-03-25 14:19:24

by Chris Mason

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

Excerpts from Markus Trippelsdorf's message of 2011-03-25 10:14:13 -0400:
> On 2011.03.25 at 15:10 +0100, Jens Axboe wrote:
> > On 2011-03-25 14:09, Markus Trippelsdorf wrote:
> > > On 2011.03.25 at 13:44 +0100, Jens Axboe wrote:
> > >> On 2011-03-25 10:57, Markus Trippelsdorf wrote:
> > >>>
> > >>> Reverting it solves all problems here.
> > >>
> > >> Can you try this one?
> > >
> > > This one doesn't help; I still get the same BUG.
> > >
> > > BTW if you're having trouble reproducing this, here is the only non
> > > stock xfs option that I use on the affected partitions:
> > >
> > > noatime,logbsize=262144
> >
> > This?
>
> No.
>

I was able to reproduce it here. This is working for me:


diff --git a/block/blk-core.c b/block/blk-core.c
index 59b5c00..88f5f6a 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -271,7 +271,7 @@ EXPORT_SYMBOL(blk_start_queue);
**/
void blk_stop_queue(struct request_queue *q)
{
- cancel_delayed_work(&q->delay_work);
+ __cancel_delayed_work(&q->delay_work);
queue_flag_set(QUEUE_FLAG_STOPPED, q);
}
EXPORT_SYMBOL(blk_stop_queue);
@@ -1193,6 +1193,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
struct request *rq;
bool ret = false;

+ preempt_disable();
plug = tsk->plug;
if (!plug)
goto out;
@@ -1215,6 +1216,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
}
}
out:
+ preempt_enable();
return ret;
}

@@ -1309,6 +1311,7 @@ get_rq:
put_cpu();
}

+ preempt_disable();
plug = current->plug;
if (plug) {
if (!plug->should_sort && !list_empty(&plug->list)) {
@@ -1324,7 +1327,9 @@ get_rq:
req->cmd_flags |= REQ_ON_PLUG;
list_add_tail(&req->queuelist, &plug->list);
drive_stat_acct(req, 1);
+ preempt_enable();
} else {
+ preempt_enable();
spin_lock_irq(q->queue_lock);
add_acct_request(q, req, where);
__blk_run_queue(q, false);

2011-03-25 14:20:16

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-25 15:14, Markus Trippelsdorf wrote:
> On 2011.03.25 at 15:10 +0100, Jens Axboe wrote:
>> On 2011-03-25 14:09, Markus Trippelsdorf wrote:
>>> On 2011.03.25 at 13:44 +0100, Jens Axboe wrote:
>>>> On 2011-03-25 10:57, Markus Trippelsdorf wrote:
>>>>>
>>>>> Reverting it solves all problems here.
>>>>
>>>> Can you try this one?
>>>
>>> This one doesn't help; I still get the same BUG.
>>>
>>> BTW if you're having trouble reproducing this, here is the only non
>>> stock xfs option that I use on the affected partitions:
>>>
>>> noatime,logbsize=262144
>>
>> This?
>
> No.

Lets expand the scope a bit, this one?

diff --git a/block/blk-core.c b/block/blk-core.c
index 59b5c00..5876c73 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -1193,6 +1193,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
struct request *rq;
bool ret = false;

+ preempt_disable();
plug = tsk->plug;
if (!plug)
goto out;
@@ -1215,6 +1216,7 @@ static bool attempt_plug_merge(struct task_struct *tsk, struct request_queue *q,
}
}
out:
+ preempt_enable();
return ret;
}

@@ -1309,6 +1311,7 @@ get_rq:
put_cpu();
}

+ preempt_disable();
plug = current->plug;
if (plug) {
if (!plug->should_sort && !list_empty(&plug->list)) {
@@ -1324,7 +1327,9 @@ get_rq:
req->cmd_flags |= REQ_ON_PLUG;
list_add_tail(&req->queuelist, &plug->list);
drive_stat_acct(req, 1);
+ preempt_enable();
} else {
+ preempt_enable();
spin_lock_irq(q->queue_lock);
add_acct_request(q, req, where);
__blk_run_queue(q, false);


--
Jens Axboe

2011-03-25 14:20:53

by Chris Mason

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

Excerpts from Chris Mason's message of 2011-03-25 10:18:19 -0400:
> Excerpts from Markus Trippelsdorf's message of 2011-03-25 10:14:13 -0400:
> > On 2011.03.25 at 15:10 +0100, Jens Axboe wrote:
> > > On 2011-03-25 14:09, Markus Trippelsdorf wrote:
> > > > On 2011.03.25 at 13:44 +0100, Jens Axboe wrote:
> > > >> On 2011-03-25 10:57, Markus Trippelsdorf wrote:
> > > >>>
> > > >>> Reverting it solves all problems here.
> > > >>
> > > >> Can you try this one?
> > > >
> > > > This one doesn't help; I still get the same BUG.
> > > >
> > > > BTW if you're having trouble reproducing this, here is the only non
> > > > stock xfs option that I use on the affected partitions:
> > > >
> > > > noatime,logbsize=262144
> > >
> > > This?
> >
> > No.
> >
>
> I was able to reproduce it here. This is working for me:
>

Sorry, this is working better but still explodes later. Don't bother
with this one.

-chris

2011-03-25 14:24:54

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.25 at 10:19 -0400, Chris Mason wrote:
> Excerpts from Chris Mason's message of 2011-03-25 10:18:19 -0400:
> > Excerpts from Markus Trippelsdorf's message of 2011-03-25 10:14:13 -0400:
> > > On 2011.03.25 at 15:10 +0100, Jens Axboe wrote:
> > > > On 2011-03-25 14:09, Markus Trippelsdorf wrote:
> > > > > On 2011.03.25 at 13:44 +0100, Jens Axboe wrote:
> > > > >> On 2011-03-25 10:57, Markus Trippelsdorf wrote:
> > > > >>>
> > > > >>> Reverting it solves all problems here.
> > > > >>
> > > > >> Can you try this one?
> > > > >
> > > > > This one doesn't help; I still get the same BUG.
> > > > >
> > > > > BTW if you're having trouble reproducing this, here is the only non
> > > > > stock xfs option that I use on the affected partitions:
> > > > >
> > > > > noatime,logbsize=262144
> > > >
> > > > This?
> > >
> > > No.
> > >
> >
> > I was able to reproduce it here. This is working for me:
> >
>
> Sorry, this is working better but still explodes later. Don't bother
> with this one.

Yeah, doesn't help here also.
--
Markus

2011-03-25 14:28:31

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.25 at 15:20 +0100, Jens Axboe wrote:
> On 2011-03-25 15:14, Markus Trippelsdorf wrote:
> > On 2011.03.25 at 15:10 +0100, Jens Axboe wrote:
> >> On 2011-03-25 14:09, Markus Trippelsdorf wrote:
> >>> On 2011.03.25 at 13:44 +0100, Jens Axboe wrote:
> >>>> On 2011-03-25 10:57, Markus Trippelsdorf wrote:
> >>>>>
> >>>>> Reverting it solves all problems here.
> >>>>
> >>>> Can you try this one?
> >>>
> >>> This one doesn't help; I still get the same BUG.
> >>>
> >>> BTW if you're having trouble reproducing this, here is the only non
> >>> stock xfs option that I use on the affected partitions:
> >>>
> >>> noatime,logbsize=262144
> >>
> >> This?
> >
> > No.
>
> Lets expand the scope a bit, this one?

No. And setting CONFIG_PREEMPT_NONE also doesn't help.

--
Markus

2011-03-25 15:51:56

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-25 15:28, Markus Trippelsdorf wrote:
> On 2011.03.25 at 15:20 +0100, Jens Axboe wrote:
>> On 2011-03-25 15:14, Markus Trippelsdorf wrote:
>>> On 2011.03.25 at 15:10 +0100, Jens Axboe wrote:
>>>> On 2011-03-25 14:09, Markus Trippelsdorf wrote:
>>>>> On 2011.03.25 at 13:44 +0100, Jens Axboe wrote:
>>>>>> On 2011-03-25 10:57, Markus Trippelsdorf wrote:
>>>>>>>
>>>>>>> Reverting it solves all problems here.
>>>>>>
>>>>>> Can you try this one?
>>>>>
>>>>> This one doesn't help; I still get the same BUG.
>>>>>
>>>>> BTW if you're having trouble reproducing this, here is the only non
>>>>> stock xfs option that I use on the affected partitions:
>>>>>
>>>>> noatime,logbsize=262144
>>>>
>>>> This?
>>>
>>> No.
>>
>> Lets expand the scope a bit, this one?
>
> No. And setting CONFIG_PREEMPT_NONE also doesn't help.
>

Does this work? I'm actually relieved that the preempt was a red
herring. I had earlier convinced myself that it was all safe without
that mucking around with preempt counts.

diff --git a/block/blk-core.c b/block/blk-core.c
index 59b5c00..64e96ee 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -2702,7 +2702,10 @@ static void flush_plug_list(struct blk_plug *plug)
/*
* rq is already accounted, so use raw insert
*/
- __elv_add_request(q, rq, ELEVATOR_INSERT_SORT_MERGE);
+ if (rq->cmd_flags & (REQ_FLUSH | REQ_FUA))
+ __elv_add_request(q, rq, ELEVATOR_INSERT_FLUSH);
+ else
+ __elv_add_request(q, rq, ELEVATOR_INSERT_SORT_MERGE);
}

if (q) {


--
Jens Axboe

2011-03-25 15:58:25

by Markus Trippelsdorf

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011.03.25 at 16:51 +0100, Jens Axboe wrote:
> On 2011-03-25 15:28, Markus Trippelsdorf wrote:
> > On 2011.03.25 at 15:20 +0100, Jens Axboe wrote:
> >> On 2011-03-25 15:14, Markus Trippelsdorf wrote:
> >>> On 2011.03.25 at 15:10 +0100, Jens Axboe wrote:
> >>>> On 2011-03-25 14:09, Markus Trippelsdorf wrote:
> >>>>> On 2011.03.25 at 13:44 +0100, Jens Axboe wrote:
> >>>>>> On 2011-03-25 10:57, Markus Trippelsdorf wrote:
> >>>>>>>
> >>>>>>> Reverting it solves all problems here.
> >>>>>>
> >>>>>> Can you try this one?
> >>>>>
> >>>>> This one doesn't help; I still get the same BUG.
> >>>>>
> >>>>> BTW if you're having trouble reproducing this, here is the only non
> >>>>> stock xfs option that I use on the affected partitions:
> >>>>>
> >>>>> noatime,logbsize=262144
> >>>>
> >>>> This?
> >>>
> >>> No.
> >>
> >> Lets expand the scope a bit, this one?
> >
> > No. And setting CONFIG_PREEMPT_NONE also doesn't help.
> >
>
> Does this work? I'm actually relieved that the preempt was a red
> herring. I had earlier convinced myself that it was all safe without
> that mucking around with preempt counts.

Finally: YES.
Many thanks Jens.

--
Markus

2011-03-25 16:01:25

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39 - early Oops

On 2011-03-25 16:58, Markus Trippelsdorf wrote:
> On 2011.03.25 at 16:51 +0100, Jens Axboe wrote:
>> On 2011-03-25 15:28, Markus Trippelsdorf wrote:
>>> On 2011.03.25 at 15:20 +0100, Jens Axboe wrote:
>>>> On 2011-03-25 15:14, Markus Trippelsdorf wrote:
>>>>> On 2011.03.25 at 15:10 +0100, Jens Axboe wrote:
>>>>>> On 2011-03-25 14:09, Markus Trippelsdorf wrote:
>>>>>>> On 2011.03.25 at 13:44 +0100, Jens Axboe wrote:
>>>>>>>> On 2011-03-25 10:57, Markus Trippelsdorf wrote:
>>>>>>>>>
>>>>>>>>> Reverting it solves all problems here.
>>>>>>>>
>>>>>>>> Can you try this one?
>>>>>>>
>>>>>>> This one doesn't help; I still get the same BUG.
>>>>>>>
>>>>>>> BTW if you're having trouble reproducing this, here is the only non
>>>>>>> stock xfs option that I use on the affected partitions:
>>>>>>>
>>>>>>> noatime,logbsize=262144
>>>>>>
>>>>>> This?
>>>>>
>>>>> No.
>>>>
>>>> Lets expand the scope a bit, this one?
>>>
>>> No. And setting CONFIG_PREEMPT_NONE also doesn't help.
>>>
>>
>> Does this work? I'm actually relieved that the preempt was a red
>> herring. I had earlier convinced myself that it was all safe without
>> that mucking around with preempt counts.
>
> Finally: YES.
> Many thanks Jens.

Awesome. And thanks goes to you for doing tons of testing and having a
rapid turn-around time, and to Sergey for finding the right commit that
allowed a genuine fix.

I'll prepare a pull for Linus immediately.

--
Jens Axboe

2011-03-25 21:35:51

by Geert Uytterhoeven

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

Hi Jens,

On Thu, Mar 24, 2011 at 14:43, Jens Axboe <[email protected]> wrote:
> Jens Axboe (20):
>      block: remove per-queue plugging

This one (commit 7eaceaccab5f40bbfda044629a6298616aeaed50) breaks IDE
on Atari/m68k under ARAnyM. It hangs on:

| ide: Falcon IDE controller
| Probing IDE interface ide0...
| hda: Sarge m68k, ATA DISK drive
| ide0 at 0xfff00000 on irq 15 (serialized)
| ide-gd driver 1.18
| hda: max request size: 128KiB
| hda: 2118816 sectors (1084 MB) w/256KiB Cache, CHS=2102/16/63

The next expected line is the partition parsing:

| hda: AHDI hda1 hda2

I tried reverting it, but it caused too many conflicts.
The parent of 7eaceaccab5f40bbfda044629a6298616aeaed50 works.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

2011-03-26 06:29:18

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

On 2011-03-25 22:35, Geert Uytterhoeven wrote:
> Hi Jens,
>
> On Thu, Mar 24, 2011 at 14:43, Jens Axboe <[email protected]> wrote:
>> Jens Axboe (20):
>> block: remove per-queue plugging
>
> This one (commit 7eaceaccab5f40bbfda044629a6298616aeaed50) breaks IDE
> on Atari/m68k under ARAnyM. It hangs on:
>
> | ide: Falcon IDE controller
> | Probing IDE interface ide0...
> | hda: Sarge m68k, ATA DISK drive
> | ide0 at 0xfff00000 on irq 15 (serialized)
> | ide-gd driver 1.18
> | hda: max request size: 128KiB
> | hda: 2118816 sectors (1084 MB) w/256KiB Cache, CHS=2102/16/63
>
> The next expected line is the partition parsing:
>
> | hda: AHDI hda1 hda2

Geert, does this work for you?

diff --git a/drivers/ide/ide-io.c b/drivers/ide/ide-io.c
index f407784..381017c 100644
--- a/drivers/ide/ide-io.c
+++ b/drivers/ide/ide-io.c
@@ -549,6 +549,9 @@ plug_device_2:

if (rq)
blk_requeue_request(q, rq);
+
+ /* Use 3ms as that was the old plug delay */
+ blk_delay_queue(q, msecs_to_jiffies(3));
}

void ide_requeue_and_plug(ide_drive_t *drive, struct request *rq)
@@ -561,6 +564,8 @@ void ide_requeue_and_plug(ide_drive_t *drive, struct request *rq)
if (rq)
blk_requeue_request(q, rq);

+ /* Use 3ms as that was the old plug delay */
+ blk_delay_queue(q, msecs_to_jiffies(3));
spin_unlock_irqrestore(q->queue_lock, flags);
}


--
Jens Axboe

2011-03-26 07:21:13

by Geert Uytterhoeven

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

On Sat, Mar 26, 2011 at 07:29, Jens Axboe <[email protected]> wrote:
> On 2011-03-25 22:35, Geert Uytterhoeven wrote:
>> On Thu, Mar 24, 2011 at 14:43, Jens Axboe <[email protected]> wrote:
>>> Jens Axboe (20):
>>>      block: remove per-queue plugging
>>
>> This one (commit 7eaceaccab5f40bbfda044629a6298616aeaed50) breaks IDE
>> on Atari/m68k under ARAnyM. It hangs on:
>>
>> | ide: Falcon IDE controller
>> | Probing IDE interface ide0...
>> | hda: Sarge m68k, ATA DISK drive
>> | ide0 at 0xfff00000 on irq 15 (serialized)
>> | ide-gd driver 1.18
>> | hda: max request size: 128KiB
>> | hda: 2118816 sectors (1084 MB) w/256KiB Cache, CHS=2102/16/63
>>
>> The next expected line is the partition parsing:
>>
>> | hda: AHDI hda1 hda2
>
> Geert, does this work for you?

Yep.Thanks!

Tested-by: Geert Uytterhoeven <[email protected]>

> diff --git a/drivers/ide/ide-io.c b/drivers/ide/ide-io.c
> index f407784..381017c 100644
> --- a/drivers/ide/ide-io.c
> +++ b/drivers/ide/ide-io.c
> @@ -549,6 +549,9 @@ plug_device_2:
>
>        if (rq)
>                blk_requeue_request(q, rq);
> +
> +       /* Use 3ms as that was the old plug delay */
> +       blk_delay_queue(q, msecs_to_jiffies(3));
>  }
>
>  void ide_requeue_and_plug(ide_drive_t *drive, struct request *rq)
> @@ -561,6 +564,8 @@ void ide_requeue_and_plug(ide_drive_t *drive, struct request *rq)
>        if (rq)
>                blk_requeue_request(q, rq);
>
> +       /* Use 3ms as that was the old plug delay */
> +       blk_delay_queue(q, msecs_to_jiffies(3));
>        spin_unlock_irqrestore(q->queue_lock, flags);
>  }
>

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

2011-03-26 08:25:33

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

On 2011-03-26 08:21, Geert Uytterhoeven wrote:
> On Sat, Mar 26, 2011 at 07:29, Jens Axboe <[email protected]> wrote:
>> On 2011-03-25 22:35, Geert Uytterhoeven wrote:
>>> On Thu, Mar 24, 2011 at 14:43, Jens Axboe <[email protected]> wrote:
>>>> Jens Axboe (20):
>>>> block: remove per-queue plugging
>>>
>>> This one (commit 7eaceaccab5f40bbfda044629a6298616aeaed50) breaks IDE
>>> on Atari/m68k under ARAnyM. It hangs on:
>>>
>>> | ide: Falcon IDE controller
>>> | Probing IDE interface ide0...
>>> | hda: Sarge m68k, ATA DISK drive
>>> | ide0 at 0xfff00000 on irq 15 (serialized)
>>> | ide-gd driver 1.18
>>> | hda: max request size: 128KiB
>>> | hda: 2118816 sectors (1084 MB) w/256KiB Cache, CHS=2102/16/63
>>>
>>> The next expected line is the partition parsing:
>>>
>>> | hda: AHDI hda1 hda2
>>
>> Geert, does this work for you?
>
> Yep.Thanks!

Great! I think we should place those blk_delay_queue() calls under the
if (rq), that should workd to and be more optimal. Can I ask you to
check that, too?

So:

if (rq) {
blk_requeue_request(q, rq);
blk_delay_queue(q, msecs_to_jiffies(3));
}

in both locations.

--
Jens Axboe

2011-03-26 08:34:16

by Geert Uytterhoeven

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

On Sat, Mar 26, 2011 at 09:25, Jens Axboe <[email protected]> wrote:
> On 2011-03-26 08:21, Geert Uytterhoeven wrote:
>> On Sat, Mar 26, 2011 at 07:29, Jens Axboe <[email protected]> wrote:
>>> On 2011-03-25 22:35, Geert Uytterhoeven wrote:
>>>> On Thu, Mar 24, 2011 at 14:43, Jens Axboe <[email protected]> wrote:
>>>>> Jens Axboe (20):
>>>>>      block: remove per-queue plugging
>>>>
>>>> This one (commit 7eaceaccab5f40bbfda044629a6298616aeaed50) breaks IDE
>>>> on Atari/m68k under ARAnyM. It hangs on:
>>>>
>>>> | ide: Falcon IDE controller
>>>> | Probing IDE interface ide0...
>>>> | hda: Sarge m68k, ATA DISK drive
>>>> | ide0 at 0xfff00000 on irq 15 (serialized)
>>>> | ide-gd driver 1.18
>>>> | hda: max request size: 128KiB
>>>> | hda: 2118816 sectors (1084 MB) w/256KiB Cache, CHS=2102/16/63
>>>>
>>>> The next expected line is the partition parsing:
>>>>
>>>> | hda: AHDI hda1 hda2
>>>
>>> Geert, does this work for you?
>>
>> Yep.Thanks!
>
> Great! I think we should place those blk_delay_queue() calls under the
> if (rq), that should workd to and be more optimal. Can I ask you to
> check that, too?
>
> So:
>
> if (rq) {
>        blk_requeue_request(q, rq);
>        blk_delay_queue(q, msecs_to_jiffies(3));
> }
>
> in both locations.

Works, too.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

2011-03-26 09:26:33

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

On 2011-03-26 09:34, Geert Uytterhoeven wrote:
> On Sat, Mar 26, 2011 at 09:25, Jens Axboe <[email protected]> wrote:
>> On 2011-03-26 08:21, Geert Uytterhoeven wrote:
>>> On Sat, Mar 26, 2011 at 07:29, Jens Axboe <[email protected]> wrote:
>>>> On 2011-03-25 22:35, Geert Uytterhoeven wrote:
>>>>> On Thu, Mar 24, 2011 at 14:43, Jens Axboe <[email protected]> wrote:
>>>>>> Jens Axboe (20):
>>>>>> block: remove per-queue plugging
>>>>>
>>>>> This one (commit 7eaceaccab5f40bbfda044629a6298616aeaed50) breaks IDE
>>>>> on Atari/m68k under ARAnyM. It hangs on:
>>>>>
>>>>> | ide: Falcon IDE controller
>>>>> | Probing IDE interface ide0...
>>>>> | hda: Sarge m68k, ATA DISK drive
>>>>> | ide0 at 0xfff00000 on irq 15 (serialized)
>>>>> | ide-gd driver 1.18
>>>>> | hda: max request size: 128KiB
>>>>> | hda: 2118816 sectors (1084 MB) w/256KiB Cache, CHS=2102/16/63
>>>>>
>>>>> The next expected line is the partition parsing:
>>>>>
>>>>> | hda: AHDI hda1 hda2
>>>>
>>>> Geert, does this work for you?
>>>
>>> Yep.Thanks!
>>
>> Great! I think we should place those blk_delay_queue() calls under the
>> if (rq), that should workd to and be more optimal. Can I ask you to
>> check that, too?
>>
>> So:
>>
>> if (rq) {
>> blk_requeue_request(q, rq);
>> blk_delay_queue(q, msecs_to_jiffies(3));
>> }
>>
>> in both locations.
>
> Works, too.

Great, thanks for testing! I'll queue this up and submit it so it makes
2.6.39-rc1.

--
Jens Axboe

2011-03-26 16:49:00

by Linus Torvalds

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

On Fri, Mar 25, 2011 at 11:29 PM, Jens Axboe <[email protected]> wrote:
> +
> + ? ? ? /* Use 3ms as that was the old plug delay */
> + ? ? ? blk_delay_queue(q, msecs_to_jiffies(3));

That's bogus. blk_delay_queue() already takes msecs, not jiffies.

Also, do we really need to delay every unplug like this? It seems sad.
A 3ms delay is a long time these days - admittedly most people
hopefully use ATA these days if they have an SSD or something, but
still..

Linus

2011-03-26 16:53:54

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

On 2011-03-26 17:48, Linus Torvalds wrote:
> On Fri, Mar 25, 2011 at 11:29 PM, Jens Axboe <[email protected]> wrote:
>> +
>> + /* Use 3ms as that was the old plug delay */
>> + blk_delay_queue(q, msecs_to_jiffies(3));
>
> That's bogus. blk_delay_queue() already takes msecs, not jiffies.

You are right, braino.

> Also, do we really need to delay every unplug like this? It seems sad.
> A 3ms delay is a long time these days - admittedly most people
> hopefully use ATA these days if they have an SSD or something, but
> still..

Depends on whether you want the 'call me back, I ran into busy this
time' or the 'recall me immediately'.

I'll take a look at the IDE case tonight and submit a real fix.

--
Jens Axboe

2011-03-26 18:48:39

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

On 2011-03-26 17:53, Jens Axboe wrote:
> On 2011-03-26 17:48, Linus Torvalds wrote:
>> On Fri, Mar 25, 2011 at 11:29 PM, Jens Axboe <[email protected]> wrote:
>>> +
>>> + /* Use 3ms as that was the old plug delay */
>>> + blk_delay_queue(q, msecs_to_jiffies(3));
>>
>> That's bogus. blk_delay_queue() already takes msecs, not jiffies.
>
> You are right, braino.
>
>> Also, do we really need to delay every unplug like this? It seems sad.
>> A 3ms delay is a long time these days - admittedly most people
>> hopefully use ATA these days if they have an SSD or something, but
>> still..
>
> Depends on whether you want the 'call me back, I ran into busy this
> time' or the 'recall me immediately'.
>
> I'll take a look at the IDE case tonight and submit a real fix.

It's not completely clear cut what the delays should be in the various
old replug cases. Only for the drive sleeping can I make an educated
guess since we now the timeout period.

So something like this. Retains the same recall delay as we had before.
Someone who cares is free to go and optimize this, if the queue rerun
delay should be shorter in some cases.

diff --git a/drivers/ide/ide-io.c b/drivers/ide/ide-io.c
index f407784..0e406d73 100644
--- a/drivers/ide/ide-io.c
+++ b/drivers/ide/ide-io.c
@@ -440,6 +440,7 @@ void do_ide_request(struct request_queue *q)
struct ide_host *host = hwif->host;
struct request *rq = NULL;
ide_startstop_t startstop;
+ unsigned long queue_run_ms = 3; /* old plug delay */

spin_unlock_irq(q->queue_lock);

@@ -459,6 +460,9 @@ repeat:
prev_port = hwif->host->cur_port;
if (drive->dev_flags & IDE_DFLAG_SLEEPING &&
time_after(drive->sleep, jiffies)) {
+ unsigned long left = jiffies - drive->sleep;
+
+ queue_run_ms = jiffies_to_msecs(left + 1);
ide_unlock_port(hwif);
goto plug_device;
}
@@ -547,8 +551,10 @@ plug_device:
plug_device_2:
spin_lock_irq(q->queue_lock);

- if (rq)
+ if (rq) {
blk_requeue_request(q, rq);
+ blk_delay_queue(q, queue_run_ms);
+ }
}

void ide_requeue_and_plug(ide_drive_t *drive, struct request *rq)
@@ -562,6 +568,10 @@ void ide_requeue_and_plug(ide_drive_t *drive, struct request *rq)
blk_requeue_request(q, rq);

spin_unlock_irqrestore(q->queue_lock, flags);
+
+ /* Use 3ms as that was the old plug delay */
+ if (rq)
+ blk_delay_queue(q, 3);
}

static int drive_is_ready(ide_drive_t *drive)


--
Jens Axboe

2011-03-27 11:49:52

by Avi Kivity

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

On 03/24/2011 03:43 PM, Jens Axboe wrote:
> Hi Linus,
>
> This is the main pull request for the block IO layer and friends for
> 2.6.39.
>
> There are two major things in this tree:
>
> - The removal of the per-device plugging state for disks. On fast
> devices, it ended up hammering the queue lock quite hard. The new
> scheme puts the plugging state on the stack and allows an IO submitter
> to finish his batch of IO before pushing it to the queue. Once that
> push starts, we'll insert/merge with the existing queue.
>
> A pointer to this plugging context is stored in the task structure. If
> a task ends up blocking before it has submitted it's IO (usual cause
> would be memory allocation of some sort), the plugged list is
> auto-submitted before the task goes to sleep.

This is the fourth "do something if preempted" hook (the other three are
kvm, cmwq, and perf). Why not use sched notifiers for this?

--
error compiling committee.c: too many arguments to function

2011-03-27 12:01:13

by Jens Axboe

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

On 2011-03-27 13:49, Avi Kivity wrote:
> On 03/24/2011 03:43 PM, Jens Axboe wrote:
>> Hi Linus,
>>
>> This is the main pull request for the block IO layer and friends for
>> 2.6.39.
>>
>> There are two major things in this tree:
>>
>> - The removal of the per-device plugging state for disks. On fast
>> devices, it ended up hammering the queue lock quite hard. The new
>> scheme puts the plugging state on the stack and allows an IO submitter
>> to finish his batch of IO before pushing it to the queue. Once that
>> push starts, we'll insert/merge with the existing queue.
>>
>> A pointer to this plugging context is stored in the task structure. If
>> a task ends up blocking before it has submitted it's IO (usual cause
>> would be memory allocation of some sort), the plugged list is
>> auto-submitted before the task goes to sleep.
>
> This is the fourth "do something if preempted" hook (the other three are
> kvm, cmwq, and perf). Why not use sched notifiers for this?

It's a 'if preempted', it's 'if going to sleep'. Two issues with the
preempt notifiers for this, I did look into it:

- Not unconditionally available. Not a big problem, we could just do
that.

- Sched out is called under the runqueue lock, and it's not safe to drop
it.

But it's not a bad point, it would be nice if we could unify all these
use cases and provide a notifier that works for all.

--
Jens Axboe

2011-03-27 13:21:55

by Alan

[permalink] [raw]
Subject: Re: [GIT PULL] Core block IO bits for 2.6.39

On Sat, 26 Mar 2011 09:48:37 -0700
Linus Torvalds <[email protected]> wrote:

> On Fri, Mar 25, 2011 at 11:29 PM, Jens Axboe <[email protected]> wrote:
> > +
> > + ? ? ? /* Use 3ms as that was the old plug delay */
> > + ? ? ? blk_delay_queue(q, msecs_to_jiffies(3));
>
> That's bogus. blk_delay_queue() already takes msecs, not jiffies.
>
> Also, do we really need to delay every unplug like this? It seems sad.
> A 3ms delay is a long time these days - admittedly most people
> hopefully use ATA these days if they have an SSD or something, but
> still..

The ATA code contains several other bogus delays btw. So out of the box
unless you are using AHCI you are continually running small pointless
delays every command issued.