Hi all,
This patchset ("stable page writes, part 2") makes some key modifications to
the original 'stable page writes' patchset. First, it provides creators
(devices and filesystems) of a backing_dev_info a flag that declares whether or
not it is necessary to ensure that page contents cannot change during writeout.
It is no longer assumed that this is true of all devices (which was never true
anyway). Second, the flag is used to relaxed the wait_on_page_writeback calls
so that wait only occurs if the device needs it. Third, it fixes up the
remaining disk-backed filesystems to use this improved conditional-wait logic
to provide stable page writes on those filesystems.
It is hoped that (for people not using checksumming devices, anyway) this
patchset will give back unnecessary performance decreases since the original
stable page write patchset went into 3.0. Sorry about not fixing it sooner.
Complaints were registered by several people about the long write latencies
introduced by the original stable page write patchset. Generally speaking, the
kernel ought to allocate as little extra memory as possible to facilitate
writeout, but for people who simply cannot wait, a second page stability
strategy is (re)introduced: snapshotting page contents. The waiting behavior
is still the default strategy; to enable page snapshotting, a superblock flag
(MS_SNAP_STABLE) must be set. This flag is used to bandaid^Henable stable page
writeback on ext3[1], and is not used anywhere else.
Given that there are already a few storage devices and network FSes that have
rolled their own page stability wait/page snapshot code, it would be nice to
move towards consolidating all of these. It seems possible that iscsi and
raid5 may wish to use the new stable page write support to enable zero-copy
writeout.
Thank you to Jan Kara for helping fix a couple more filesystems.
Per Andrew Morton's request, here are the result of using dbench to measure
latencies on ext2:
3.8.0-rc3:
Operation Count AvgLat MaxLat
----------------------------------------
WriteX 109347 0.028 59.817
ReadX 347180 0.004 3.391
Flush 15514 29.828 287.283
Throughput 57.429 MB/sec 4 clients 4 procs max_latency=287.290 ms
3.8.0-rc3 + patches:
WriteX 105556 0.029 4.273
ReadX 335004 0.005 4.112
Flush 14982 30.540 298.634
Throughput 55.4496 MB/sec 4 clients 4 procs max_latency=298.650 ms
As you can see, for ext2 the maximum write latency decreases from ~60ms on a
laptop hard disk to ~4ms. I'm not sure why the flush latencies increase,
though I suspect that being able to dirty pages faster gives the flusher more
work to do.
On ext4, the average write latency decreases as well as all the maximum
latencies:
3.8.0-rc3:
WriteX 85624 0.152 33.078
ReadX 272090 0.010 61.210
Flush 12129 36.219 168.260
Throughput 44.8618 MB/sec 4 clients 4 procs max_latency=168.276 ms
3.8.0-rc3 + patches:
WriteX 86082 0.141 30.928
ReadX 273358 0.010 36.124
Flush 12214 34.800 165.689
Throughput 44.9941 MB/sec 4 clients 4 procs max_latency=165.722 ms
XFS seems to exhibit similar latency improvements as ext2:
3.8.0-rc3:
WriteX 125739 0.028 104.343
ReadX 399070 0.005 4.115
Flush 17851 25.004 131.390
Throughput 66.0024 MB/sec 4 clients 4 procs max_latency=131.406 ms
3.8.0-rc3 + patches:
WriteX 123529 0.028 6.299
ReadX 392434 0.005 4.287
Flush 17549 25.120 188.687
Throughput 64.9113 MB/sec 4 clients 4 procs max_latency=188.704 ms
...and btrfs, just to round things out, also shows some latency decreases:
3.8.0-rc3:
WriteX 67122 0.083 82.355
ReadX 212719 0.005 2.828
Flush 9547 47.561 147.418
Throughput 35.3391 MB/sec 4 clients 4 procs max_latency=147.433 ms
3.8.0-rc3 + patches:
WriteX 64898 0.101 71.631
ReadX 206673 0.005 7.123
Flush 9190 47.963 219.034
Throughput 34.0795 MB/sec 4 clients 4 procs max_latency=219.044 ms
Before this patchset, all filesystems would block, regardless of whether or not
it was necessary. ext3 would wait, but still generate occasional checksum
errors. The network filesystems were left to do their own thing, so they'd
wait too.
After this patchset, all the disk filesystems except ext3 and btrfs will wait
only if the hardware requires it. ext3 (if necessary) snapshots pages instead
of blocking, and btrfs provides its own bdi so the mm will never wait. Network
filesystems haven't been touched, so either they provide their own wait code,
or they don't block at all. The blocking behavior is back to what it was
before 3.0 if you don't have a disk requiring stable page writes.
This patchset has been tested on 3.8.0-rc3 on x64 with ext3, ext4, and xfs.
I've spot-checked 3.8.0-rc4 and seem to be getting the same results as -rc3.
What does everyone think about queueing this for 3.9?
--D
[1] The alternative fixes to ext3 include fixing the locking order and page bit
handling like we did for ext4 (but then why not just use ext4?), or setting
PG_writeback so early that ext3 becomes extremely slow. I tried that, but the
number of write()s I could initiate dropped by nearly an order of magnitude.
That was a bit much even for the author of the stable page series! :)
Create a helper function to check if a backing device requires stable page
writes and, if so, performs the necessary wait. Then, make it so that all
points in the memory manager that handle making pages writable use the helper
function. This should provide stable page write support to most filesystems,
while eliminating unnecessary waiting for devices that don't require the
feature.
Before this patchset, all filesystems would block, regardless of whether or not
it was necessary. ext3 would wait, but still generate occasional checksum
errors. The network filesystems were left to do their own thing, so they'd
wait too.
After this patchset, all the disk filesystems except ext3 and btrfs will wait
only if the hardware requires it. ext3 (if necessary) snapshots pages instead
of blocking, and btrfs provides its own bdi so the mm will never wait. Network
filesystems haven't been touched, so either they provide their own stable page
guarantees or they don't block at all. The blocking behavior is back to what
it was before 3.0 if you don't have a disk requiring stable page writes.
Here's the result of using dbench to test latency on ext2:
3.8.0-rc3:
Operation Count AvgLat MaxLat
----------------------------------------
WriteX 109347 0.028 59.817
ReadX 347180 0.004 3.391
Flush 15514 29.828 287.283
Throughput 57.429 MB/sec 4 clients 4 procs max_latency=287.290 ms
3.8.0-rc3 + patches:
WriteX 105556 0.029 4.273
ReadX 335004 0.005 4.112
Flush 14982 30.540 298.634
Throughput 55.4496 MB/sec 4 clients 4 procs max_latency=298.650 ms
As you can see, the maximum write latency drops considerably with this patch
enabled. The other filesystems (ext3/ext4/xfs/btrfs) behave similarly, but see
the cover letter for those results.
Acked-by: Steven Whitehouse <[email protected]>
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Darrick J. Wong <[email protected]>
---
fs/buffer.c | 2 +-
fs/ext4/inode.c | 2 +-
fs/gfs2/file.c | 2 +-
fs/nilfs2/file.c | 2 +-
include/linux/pagemap.h | 1 +
mm/filemap.c | 3 ++-
mm/page-writeback.c | 20 ++++++++++++++++++++
7 files changed, 27 insertions(+), 5 deletions(-)
diff --git a/fs/buffer.c b/fs/buffer.c
index 7a75c3e..2ea9cd44 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -2359,7 +2359,7 @@ int __block_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf,
if (unlikely(ret < 0))
goto out_unlock;
set_page_dirty(page);
- wait_on_page_writeback(page);
+ wait_for_stable_page(page);
return 0;
out_unlock:
unlock_page(page);
diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
index cbfe13b..cd818d8b 100644
--- a/fs/ext4/inode.c
+++ b/fs/ext4/inode.c
@@ -4968,7 +4968,7 @@ int ext4_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf)
0, len, NULL,
ext4_bh_unmapped)) {
/* Wait so that we don't change page under IO */
- wait_on_page_writeback(page);
+ wait_for_stable_page(page);
ret = VM_FAULT_LOCKED;
goto out;
}
diff --git a/fs/gfs2/file.c b/fs/gfs2/file.c
index 991ab2d..b9e0ca2 100644
--- a/fs/gfs2/file.c
+++ b/fs/gfs2/file.c
@@ -483,7 +483,7 @@ out:
gfs2_holder_uninit(&gh);
if (ret == 0) {
set_page_dirty(page);
- wait_on_page_writeback(page);
+ wait_for_stable_page(page);
}
sb_end_pagefault(inode->i_sb);
return block_page_mkwrite_return(ret);
diff --git a/fs/nilfs2/file.c b/fs/nilfs2/file.c
index 6194688..bec4af6 100644
--- a/fs/nilfs2/file.c
+++ b/fs/nilfs2/file.c
@@ -126,7 +126,7 @@ static int nilfs_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf)
nilfs_transaction_commit(inode->i_sb);
mapped:
- wait_on_page_writeback(page);
+ wait_for_stable_page(page);
out:
sb_end_pagefault(inode->i_sb);
return block_page_mkwrite_return(ret);
diff --git a/include/linux/pagemap.h b/include/linux/pagemap.h
index 6da609d..0e38e13 100644
--- a/include/linux/pagemap.h
+++ b/include/linux/pagemap.h
@@ -414,6 +414,7 @@ static inline void wait_on_page_writeback(struct page *page)
}
extern void end_page_writeback(struct page *page);
+void wait_for_stable_page(struct page *page);
/*
* Add an arbitrary waiter to a page's wait queue
diff --git a/mm/filemap.c b/mm/filemap.c
index 83efee7..5577dc8 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -1728,6 +1728,7 @@ int filemap_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf)
* see the dirty page and writeprotect it again.
*/
set_page_dirty(page);
+ wait_for_stable_page(page);
out:
sb_end_pagefault(inode->i_sb);
return ret;
@@ -2274,7 +2275,7 @@ repeat:
return NULL;
}
found:
- wait_on_page_writeback(page);
+ wait_for_stable_page(page);
return page;
}
EXPORT_SYMBOL(grab_cache_page_write_begin);
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 0713bfb..9c5af4d 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -2289,3 +2289,23 @@ int mapping_tagged(struct address_space *mapping, int tag)
return radix_tree_tagged(&mapping->page_tree, tag);
}
EXPORT_SYMBOL(mapping_tagged);
+
+/**
+ * wait_for_stable_page() - wait for writeback to finish, if necessary.
+ * @page: The page to wait on.
+ *
+ * This function determines if the given page is related to a backing device
+ * that requires page contents to be held stable during writeback. If so, then
+ * it will wait for any pending writeback to complete.
+ */
+void wait_for_stable_page(struct page *page)
+{
+ struct address_space *mapping = page_mapping(page);
+ struct backing_dev_info *bdi = mapping->backing_dev_info;
+
+ if (!bdi_cap_stable_pages_required(bdi))
+ return;
+
+ wait_on_page_writeback(page);
+}
+EXPORT_SYMBOL_GPL(wait_for_stable_page);
When stable pages are required, we have to wait if the page is just
going to disk and we want to modify it. Add proper callback to
ubifs_vm_page_mkwrite().
CC: Artem Bityutskiy <[email protected]>
CC: Adrian Hunter <[email protected]>
CC: [email protected]
From: Jan Kara <[email protected]>
Signed-off-by: Jan Kara <[email protected]>
Signed-off-by: Darrick J. Wong <[email protected]>
---
fs/ubifs/file.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/fs/ubifs/file.c b/fs/ubifs/file.c
index 5bc7781..4f6493c 100644
--- a/fs/ubifs/file.c
+++ b/fs/ubifs/file.c
@@ -1522,6 +1522,7 @@ static int ubifs_vm_page_mkwrite(struct vm_area_struct *vma,
ubifs_release_dirty_inode_budget(c, ui);
}
+ wait_for_stable_page(page);
unlock_page(page);
return 0;
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
This creates a per-backing-device flag that tracks whether or not pages must be
held immutable during writeout. Eventually it will be used to waive
wait_for_page_writeback() if nothing requires stable pages.
Reviewed-by: Jan Kara <[email protected]>
Signed-off-by: Darrick J. Wong <[email protected]>
---
Documentation/ABI/testing/sysfs-class-bdi | 5 +++++
block/blk-integrity.c | 4 ++++
include/linux/backing-dev.h | 6 ++++++
mm/backing-dev.c | 11 +++++++++++
4 files changed, 26 insertions(+)
diff --git a/Documentation/ABI/testing/sysfs-class-bdi b/Documentation/ABI/testing/sysfs-class-bdi
index 5f50097..d773d56 100644
--- a/Documentation/ABI/testing/sysfs-class-bdi
+++ b/Documentation/ABI/testing/sysfs-class-bdi
@@ -48,3 +48,8 @@ max_ratio (read-write)
most of the write-back cache. For example in case of an NFS
mount that is prone to get stuck, or a FUSE mount which cannot
be trusted to play fair.
+
+stable_pages_required (read-only)
+
+ If set, the backing device requires that all pages comprising a write
+ request must not be changed until writeout is complete.
diff --git a/block/blk-integrity.c b/block/blk-integrity.c
index da2a818..dabd221 100644
--- a/block/blk-integrity.c
+++ b/block/blk-integrity.c
@@ -420,6 +420,8 @@ int blk_integrity_register(struct gendisk *disk, struct blk_integrity *template)
} else
bi->name = bi_unsupported_name;
+ disk->queue->backing_dev_info.capabilities |= BDI_CAP_STABLE_WRITES;
+
return 0;
}
EXPORT_SYMBOL(blk_integrity_register);
@@ -438,6 +440,8 @@ void blk_integrity_unregister(struct gendisk *disk)
if (!disk || !disk->integrity)
return;
+ disk->queue->backing_dev_info.capabilities &= ~BDI_CAP_STABLE_WRITES;
+
bi = disk->integrity;
kobject_uevent(&bi->kobj, KOBJ_REMOVE);
diff --git a/include/linux/backing-dev.h b/include/linux/backing-dev.h
index 12731a1..3504599 100644
--- a/include/linux/backing-dev.h
+++ b/include/linux/backing-dev.h
@@ -254,6 +254,7 @@ int bdi_set_max_ratio(struct backing_dev_info *bdi, unsigned int max_ratio);
#define BDI_CAP_EXEC_MAP 0x00000040
#define BDI_CAP_NO_ACCT_WB 0x00000080
#define BDI_CAP_SWAP_BACKED 0x00000100
+#define BDI_CAP_STABLE_WRITES 0x00000200
#define BDI_CAP_VMFLAGS \
(BDI_CAP_READ_MAP | BDI_CAP_WRITE_MAP | BDI_CAP_EXEC_MAP)
@@ -308,6 +309,11 @@ long wait_iff_congested(struct zone *zone, int sync, long timeout);
int pdflush_proc_obsolete(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp, loff_t *ppos);
+static inline bool bdi_cap_stable_pages_required(struct backing_dev_info *bdi)
+{
+ return bdi->capabilities & BDI_CAP_STABLE_WRITES;
+}
+
static inline bool bdi_cap_writeback_dirty(struct backing_dev_info *bdi)
{
return !(bdi->capabilities & BDI_CAP_NO_WRITEBACK);
diff --git a/mm/backing-dev.c b/mm/backing-dev.c
index d3ca2b3..41733c5 100644
--- a/mm/backing-dev.c
+++ b/mm/backing-dev.c
@@ -221,12 +221,23 @@ static ssize_t max_ratio_store(struct device *dev,
}
BDI_SHOW(max_ratio, bdi->max_ratio)
+static ssize_t stable_pages_required_show(struct device *dev,
+ struct device_attribute *attr,
+ char *page)
+{
+ struct backing_dev_info *bdi = dev_get_drvdata(dev);
+
+ return snprintf(page, PAGE_SIZE-1, "%d\n",
+ bdi_cap_stable_pages_required(bdi) ? 1 : 0);
+}
+
#define __ATTR_RW(attr) __ATTR(attr, 0644, attr##_show, attr##_store)
static struct device_attribute bdi_dev_attrs[] = {
__ATTR_RW(read_ahead_kb),
__ATTR_RW(min_ratio),
__ATTR_RW(max_ratio),
+ __ATTR_RO(stable_pages_required),
__ATTR_NULL,
};
Fix up the ->page_mkwrite handler to provide stable page writes if necessary.
Signed-off-by: Darrick J. Wong <[email protected]>
---
fs/9p/vfs_file.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/fs/9p/vfs_file.c b/fs/9p/vfs_file.c
index c2483e9..357260b 100644
--- a/fs/9p/vfs_file.c
+++ b/fs/9p/vfs_file.c
@@ -620,6 +620,7 @@ v9fs_vm_page_mkwrite(struct vm_area_struct *vma, struct vm_fault *vmf)
lock_page(page);
if (page->mapping != inode->i_mapping)
goto out_unlock;
+ wait_for_stable_page(page);
return VM_FAULT_LOCKED;
out_unlock:
This provides a band-aid to provide stable page writes on jbd without needing
to backport the fixed locking and page writeback bit handling schemes of jbd2.
The band-aid works by using bounce buffers to snapshot page contents instead of
waiting.
For those wondering about the ext3 bandage -- fixing the jbd locking (which was
done as part of ext4dev years ago) is a lot of surgery, and setting
PG_writeback on data pages when we actually hold the page lock dropped ext3
performance by nearly an order of magnitude. If we're going to migrate iscsi
and raid to use stable page writes, the complaints about high latency will
likely return. We might as well centralize their page snapshotting thing to
one place.
Tested-by: Andy Lutomirski <[email protected]>
Signed-off-by: Darrick J. Wong <[email protected]>
---
arch/tile/Kconfig | 6 ------
block/blk-core.c | 8 +++++---
fs/ext3/super.c | 1 +
include/uapi/linux/fs.h | 3 +++
mm/Kconfig | 13 +++++++++++++
mm/bounce.c | 48 +++++++++++++++++++++++++++++++++++++++++++----
mm/page-writeback.c | 4 ++++
7 files changed, 70 insertions(+), 13 deletions(-)
diff --git a/arch/tile/Kconfig b/arch/tile/Kconfig
index 875d008..c671fda 100644
--- a/arch/tile/Kconfig
+++ b/arch/tile/Kconfig
@@ -410,12 +410,6 @@ config TILE_USB
Provides USB host adapter support for the built-in EHCI and OHCI
interfaces on TILE-Gx chips.
-# USB OHCI needs the bounce pool since tilegx will often have more
-# than 4GB of memory, but we don't currently use the IOTLB to present
-# a 32-bit address to OHCI. So we need to use a bounce pool instead.
-config NEED_BOUNCE_POOL
- def_bool USB_OHCI_HCD
-
source "drivers/pci/hotplug/Kconfig"
endmenu
diff --git a/block/blk-core.c b/block/blk-core.c
index c973249..277134c 100644
--- a/block/blk-core.c
+++ b/block/blk-core.c
@@ -1474,6 +1474,11 @@ void blk_queue_bio(struct request_queue *q, struct bio *bio)
*/
blk_queue_bounce(q, &bio);
+ if (bio_integrity_enabled(bio) && bio_integrity_prep(bio)) {
+ bio_endio(bio, -EIO);
+ return;
+ }
+
if (bio->bi_rw & (REQ_FLUSH | REQ_FUA)) {
spin_lock_irq(q->queue_lock);
where = ELEVATOR_INSERT_FLUSH;
@@ -1714,9 +1719,6 @@ generic_make_request_checks(struct bio *bio)
*/
blk_partition_remap(bio);
- if (bio_integrity_enabled(bio) && bio_integrity_prep(bio))
- goto end_io;
-
if (bio_check_eod(bio, nr_sectors))
goto end_io;
diff --git a/fs/ext3/super.c b/fs/ext3/super.c
index 6e50223..4ba2683 100644
--- a/fs/ext3/super.c
+++ b/fs/ext3/super.c
@@ -2065,6 +2065,7 @@ static int ext3_fill_super (struct super_block *sb, void *data, int silent)
test_opt(sb,DATA_FLAGS) == EXT3_MOUNT_JOURNAL_DATA ? "journal":
test_opt(sb,DATA_FLAGS) == EXT3_MOUNT_ORDERED_DATA ? "ordered":
"writeback");
+ sb->s_flags |= MS_SNAP_STABLE;
return 0;
diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
index 780d4c6..c7fc1e6 100644
--- a/include/uapi/linux/fs.h
+++ b/include/uapi/linux/fs.h
@@ -86,6 +86,9 @@ struct inodes_stat_t {
#define MS_KERNMOUNT (1<<22) /* this is a kern_mount call */
#define MS_I_VERSION (1<<23) /* Update inode I_version field */
#define MS_STRICTATIME (1<<24) /* Always perform atime updates */
+
+/* These sb flags are internal to the kernel */
+#define MS_SNAP_STABLE (1<<27) /* Snapshot pages during writeback, if needed */
#define MS_NOSEC (1<<28)
#define MS_BORN (1<<29)
#define MS_ACTIVE (1<<30)
diff --git a/mm/Kconfig b/mm/Kconfig
index 278e3ab..7901d83 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -258,6 +258,19 @@ config BOUNCE
def_bool y
depends on BLOCK && MMU && (ZONE_DMA || HIGHMEM)
+# On the 'tile' arch, USB OHCI needs the bounce pool since tilegx will often
+# have more than 4GB of memory, but we don't currently use the IOTLB to present
+# a 32-bit address to OHCI. So we need to use a bounce pool instead.
+#
+# We also use the bounce pool to provide stable page writes for jbd. jbd
+# initiates buffer writeback without locking the page or setting PG_writeback,
+# and fixing that behavior (a second time; jbd2 doesn't have this problem) is
+# a major rework effort. Instead, use the bounce buffer to snapshot pages
+# (until jbd goes away). The only jbd user is ext3.
+config NEED_BOUNCE_POOL
+ bool
+ default y if (TILE && USB_OHCI_HCD) || (BLK_DEV_INTEGRITY && JBD)
+
config NR_QUICK
int
depends on QUICKLIST
diff --git a/mm/bounce.c b/mm/bounce.c
index 0420867..5f89017 100644
--- a/mm/bounce.c
+++ b/mm/bounce.c
@@ -178,8 +178,45 @@ static void bounce_end_io_read_isa(struct bio *bio, int err)
__bounce_end_io_read(bio, isa_page_pool, err);
}
+#ifdef CONFIG_NEED_BOUNCE_POOL
+static int must_snapshot_stable_pages(struct request_queue *q, struct bio *bio)
+{
+ struct page *page;
+ struct backing_dev_info *bdi;
+ struct address_space *mapping;
+ struct bio_vec *from;
+ int i;
+
+ if (bio_data_dir(bio) != WRITE)
+ return 0;
+
+ if (!bdi_cap_stable_pages_required(&q->backing_dev_info))
+ return 0;
+
+ /*
+ * Based on the first page that has a valid mapping, decide whether or
+ * not we have to employ bounce buffering to guarantee stable pages.
+ */
+ bio_for_each_segment(from, bio, i) {
+ page = from->bv_page;
+ mapping = page_mapping(page);
+ if (!mapping)
+ continue;
+ bdi = mapping->backing_dev_info;
+ return mapping->host->i_sb->s_flags & MS_SNAP_STABLE;
+ }
+
+ return 0;
+}
+#else
+static int must_snapshot_stable_pages(struct request_queue *q, struct bio *bio)
+{
+ return 0;
+}
+#endif /* CONFIG_NEED_BOUNCE_POOL */
+
static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
- mempool_t *pool)
+ mempool_t *pool, int force)
{
struct page *page;
struct bio *bio = NULL;
@@ -192,7 +229,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
/*
* is destination page below bounce pfn?
*/
- if (page_to_pfn(page) <= queue_bounce_pfn(q))
+ if (page_to_pfn(page) <= queue_bounce_pfn(q) && !force)
continue;
/*
@@ -270,6 +307,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)
{
+ int must_bounce;
mempool_t *pool;
/*
@@ -278,13 +316,15 @@ void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)
if (!bio_has_data(*bio_orig))
return;
+ must_bounce = must_snapshot_stable_pages(q, *bio_orig);
+
/*
* for non-isa bounce case, just check if the bounce pfn is equal
* to or bigger than the highest pfn in the system -- in that case,
* don't waste time iterating over bio segments
*/
if (!(q->bounce_gfp & GFP_DMA)) {
- if (queue_bounce_pfn(q) >= blk_max_pfn)
+ if (queue_bounce_pfn(q) >= blk_max_pfn && !must_bounce)
return;
pool = page_pool;
} else {
@@ -295,7 +335,7 @@ void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)
/*
* slow path
*/
- __blk_queue_bounce(q, bio_orig, pool);
+ __blk_queue_bounce(q, bio_orig, pool, must_bounce);
}
EXPORT_SYMBOL(blk_queue_bounce);
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 9c5af4d..75a13f3 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -2305,6 +2305,10 @@ void wait_for_stable_page(struct page *page)
if (!bdi_cap_stable_pages_required(bdi))
return;
+#ifdef CONFIG_NEED_BOUNCE_POOL
+ if (mapping->host->i_sb->s_flags & MS_SNAP_STABLE)
+ return;
+#endif /* CONFIG_NEED_BOUNCE_POOL */
wait_on_page_writeback(page);
}
When stable pages are required, we have to wait if the page is just
going to disk and we want to modify it. Add proper callback to
ocfs2_grab_pages_for_write().
CC: [email protected]
CC: Joel Becker <[email protected]>
CC: Mark Fasheh <[email protected]>
From: Jan Kara <[email protected]>
Signed-off-by: Jan Kara <[email protected]>
Signed-off-by: Darrick J. Wong <[email protected]>
---
fs/ocfs2/aops.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/fs/ocfs2/aops.c b/fs/ocfs2/aops.c
index 6577432..9796330 100644
--- a/fs/ocfs2/aops.c
+++ b/fs/ocfs2/aops.c
@@ -1194,6 +1194,7 @@ static int ocfs2_grab_pages_for_write(struct address_space *mapping,
goto out;
}
}
+ wait_for_stable_page(wc->w_pages[i]);
if (index == target_index)
wc->w_target_page = wc->w_pages[i];
On Fri 18-01-13 17:13:01, Darrick J. Wong wrote:
> This provides a band-aid to provide stable page writes on jbd without needing
> to backport the fixed locking and page writeback bit handling schemes of jbd2.
> The band-aid works by using bounce buffers to snapshot page contents instead of
> waiting.
>
> For those wondering about the ext3 bandage -- fixing the jbd locking (which was
> done as part of ext4dev years ago) is a lot of surgery, and setting
> PG_writeback on data pages when we actually hold the page lock dropped ext3
> performance by nearly an order of magnitude. If we're going to migrate iscsi
> and raid to use stable page writes, the complaints about high latency will
> likely return. We might as well centralize their page snapshotting thing to
> one place.
>
> Tested-by: Andy Lutomirski <[email protected]>
> Signed-off-by: Darrick J. Wong <[email protected]>
The patch looks good. You can add:
Reviewed-by: Jan Kara <[email protected]>
Honza
> ---
> arch/tile/Kconfig | 6 ------
> block/blk-core.c | 8 +++++---
> fs/ext3/super.c | 1 +
> include/uapi/linux/fs.h | 3 +++
> mm/Kconfig | 13 +++++++++++++
> mm/bounce.c | 48 +++++++++++++++++++++++++++++++++++++++++++----
> mm/page-writeback.c | 4 ++++
> 7 files changed, 70 insertions(+), 13 deletions(-)
>
>
> diff --git a/arch/tile/Kconfig b/arch/tile/Kconfig
> index 875d008..c671fda 100644
> --- a/arch/tile/Kconfig
> +++ b/arch/tile/Kconfig
> @@ -410,12 +410,6 @@ config TILE_USB
> Provides USB host adapter support for the built-in EHCI and OHCI
> interfaces on TILE-Gx chips.
>
> -# USB OHCI needs the bounce pool since tilegx will often have more
> -# than 4GB of memory, but we don't currently use the IOTLB to present
> -# a 32-bit address to OHCI. So we need to use a bounce pool instead.
> -config NEED_BOUNCE_POOL
> - def_bool USB_OHCI_HCD
> -
> source "drivers/pci/hotplug/Kconfig"
>
> endmenu
> diff --git a/block/blk-core.c b/block/blk-core.c
> index c973249..277134c 100644
> --- a/block/blk-core.c
> +++ b/block/blk-core.c
> @@ -1474,6 +1474,11 @@ void blk_queue_bio(struct request_queue *q, struct bio *bio)
> */
> blk_queue_bounce(q, &bio);
>
> + if (bio_integrity_enabled(bio) && bio_integrity_prep(bio)) {
> + bio_endio(bio, -EIO);
> + return;
> + }
> +
> if (bio->bi_rw & (REQ_FLUSH | REQ_FUA)) {
> spin_lock_irq(q->queue_lock);
> where = ELEVATOR_INSERT_FLUSH;
> @@ -1714,9 +1719,6 @@ generic_make_request_checks(struct bio *bio)
> */
> blk_partition_remap(bio);
>
> - if (bio_integrity_enabled(bio) && bio_integrity_prep(bio))
> - goto end_io;
> -
> if (bio_check_eod(bio, nr_sectors))
> goto end_io;
>
> diff --git a/fs/ext3/super.c b/fs/ext3/super.c
> index 6e50223..4ba2683 100644
> --- a/fs/ext3/super.c
> +++ b/fs/ext3/super.c
> @@ -2065,6 +2065,7 @@ static int ext3_fill_super (struct super_block *sb, void *data, int silent)
> test_opt(sb,DATA_FLAGS) == EXT3_MOUNT_JOURNAL_DATA ? "journal":
> test_opt(sb,DATA_FLAGS) == EXT3_MOUNT_ORDERED_DATA ? "ordered":
> "writeback");
> + sb->s_flags |= MS_SNAP_STABLE;
>
> return 0;
>
> diff --git a/include/uapi/linux/fs.h b/include/uapi/linux/fs.h
> index 780d4c6..c7fc1e6 100644
> --- a/include/uapi/linux/fs.h
> +++ b/include/uapi/linux/fs.h
> @@ -86,6 +86,9 @@ struct inodes_stat_t {
> #define MS_KERNMOUNT (1<<22) /* this is a kern_mount call */
> #define MS_I_VERSION (1<<23) /* Update inode I_version field */
> #define MS_STRICTATIME (1<<24) /* Always perform atime updates */
> +
> +/* These sb flags are internal to the kernel */
> +#define MS_SNAP_STABLE (1<<27) /* Snapshot pages during writeback, if needed */
> #define MS_NOSEC (1<<28)
> #define MS_BORN (1<<29)
> #define MS_ACTIVE (1<<30)
> diff --git a/mm/Kconfig b/mm/Kconfig
> index 278e3ab..7901d83 100644
> --- a/mm/Kconfig
> +++ b/mm/Kconfig
> @@ -258,6 +258,19 @@ config BOUNCE
> def_bool y
> depends on BLOCK && MMU && (ZONE_DMA || HIGHMEM)
>
> +# On the 'tile' arch, USB OHCI needs the bounce pool since tilegx will often
> +# have more than 4GB of memory, but we don't currently use the IOTLB to present
> +# a 32-bit address to OHCI. So we need to use a bounce pool instead.
> +#
> +# We also use the bounce pool to provide stable page writes for jbd. jbd
> +# initiates buffer writeback without locking the page or setting PG_writeback,
> +# and fixing that behavior (a second time; jbd2 doesn't have this problem) is
> +# a major rework effort. Instead, use the bounce buffer to snapshot pages
> +# (until jbd goes away). The only jbd user is ext3.
> +config NEED_BOUNCE_POOL
> + bool
> + default y if (TILE && USB_OHCI_HCD) || (BLK_DEV_INTEGRITY && JBD)
> +
> config NR_QUICK
> int
> depends on QUICKLIST
> diff --git a/mm/bounce.c b/mm/bounce.c
> index 0420867..5f89017 100644
> --- a/mm/bounce.c
> +++ b/mm/bounce.c
> @@ -178,8 +178,45 @@ static void bounce_end_io_read_isa(struct bio *bio, int err)
> __bounce_end_io_read(bio, isa_page_pool, err);
> }
>
> +#ifdef CONFIG_NEED_BOUNCE_POOL
> +static int must_snapshot_stable_pages(struct request_queue *q, struct bio *bio)
> +{
> + struct page *page;
> + struct backing_dev_info *bdi;
> + struct address_space *mapping;
> + struct bio_vec *from;
> + int i;
> +
> + if (bio_data_dir(bio) != WRITE)
> + return 0;
> +
> + if (!bdi_cap_stable_pages_required(&q->backing_dev_info))
> + return 0;
> +
> + /*
> + * Based on the first page that has a valid mapping, decide whether or
> + * not we have to employ bounce buffering to guarantee stable pages.
> + */
> + bio_for_each_segment(from, bio, i) {
> + page = from->bv_page;
> + mapping = page_mapping(page);
> + if (!mapping)
> + continue;
> + bdi = mapping->backing_dev_info;
> + return mapping->host->i_sb->s_flags & MS_SNAP_STABLE;
> + }
> +
> + return 0;
> +}
> +#else
> +static int must_snapshot_stable_pages(struct request_queue *q, struct bio *bio)
> +{
> + return 0;
> +}
> +#endif /* CONFIG_NEED_BOUNCE_POOL */
> +
> static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
> - mempool_t *pool)
> + mempool_t *pool, int force)
> {
> struct page *page;
> struct bio *bio = NULL;
> @@ -192,7 +229,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
> /*
> * is destination page below bounce pfn?
> */
> - if (page_to_pfn(page) <= queue_bounce_pfn(q))
> + if (page_to_pfn(page) <= queue_bounce_pfn(q) && !force)
> continue;
>
> /*
> @@ -270,6 +307,7 @@ static void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig,
>
> void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)
> {
> + int must_bounce;
> mempool_t *pool;
>
> /*
> @@ -278,13 +316,15 @@ void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)
> if (!bio_has_data(*bio_orig))
> return;
>
> + must_bounce = must_snapshot_stable_pages(q, *bio_orig);
> +
> /*
> * for non-isa bounce case, just check if the bounce pfn is equal
> * to or bigger than the highest pfn in the system -- in that case,
> * don't waste time iterating over bio segments
> */
> if (!(q->bounce_gfp & GFP_DMA)) {
> - if (queue_bounce_pfn(q) >= blk_max_pfn)
> + if (queue_bounce_pfn(q) >= blk_max_pfn && !must_bounce)
> return;
> pool = page_pool;
> } else {
> @@ -295,7 +335,7 @@ void blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)
> /*
> * slow path
> */
> - __blk_queue_bounce(q, bio_orig, pool);
> + __blk_queue_bounce(q, bio_orig, pool, must_bounce);
> }
>
> EXPORT_SYMBOL(blk_queue_bounce);
> diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> index 9c5af4d..75a13f3 100644
> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -2305,6 +2305,10 @@ void wait_for_stable_page(struct page *page)
>
> if (!bdi_cap_stable_pages_required(bdi))
> return;
> +#ifdef CONFIG_NEED_BOUNCE_POOL
> + if (mapping->host->i_sb->s_flags & MS_SNAP_STABLE)
> + return;
> +#endif /* CONFIG_NEED_BOUNCE_POOL */
>
> wait_on_page_writeback(page);
> }
>
--
Jan Kara <[email protected]>
SUSE Labs, CR
On Fri, 18 Jan 2013 17:13:16 -0800
"Darrick J. Wong" <[email protected]> wrote:
> When stable pages are required, we have to wait if the page is just
> going to disk and we want to modify it. Add proper callback to
> ubifs_vm_page_mkwrite().
>
> CC: Artem Bityutskiy <[email protected]>
> CC: Adrian Hunter <[email protected]>
> CC: [email protected]
> From: Jan Kara <[email protected]>
> Signed-off-by: Jan Kara <[email protected]>
> Signed-off-by: Darrick J. Wong <[email protected]>
A couple of these patches had this From:Jan strangely embedded in the
signoff area. I have assumed that they were indeed authored by Jan.
Please note that authorship is indicated by putting the From: line
right at the start of the chagnelog.
I grabbed the patches. They should appear in linux-next tomorrow if I
can get the current pooppile to build.
Acked-by: Joel Becker <[email protected]>
On Fri, Jan 18, 2013 at 05:13:08PM -0800, Darrick J. Wong wrote:
> When stable pages are required, we have to wait if the page is just
> going to disk and we want to modify it. Add proper callback to
> ocfs2_grab_pages_for_write().
>
> CC: [email protected]
> CC: Joel Becker <[email protected]>
> CC: Mark Fasheh <[email protected]>
> From: Jan Kara <[email protected]>
> Signed-off-by: Jan Kara <[email protected]>
> Signed-off-by: Darrick J. Wong <[email protected]>
> ---
> fs/ocfs2/aops.c | 1 +
> 1 file changed, 1 insertion(+)
>
>
> diff --git a/fs/ocfs2/aops.c b/fs/ocfs2/aops.c
> index 6577432..9796330 100644
> --- a/fs/ocfs2/aops.c
> +++ b/fs/ocfs2/aops.c
> @@ -1194,6 +1194,7 @@ static int ocfs2_grab_pages_for_write(struct address_space *mapping,
> goto out;
> }
> }
> + wait_for_stable_page(wc->w_pages[i]);
>
> if (index == target_index)
> wc->w_target_page = wc->w_pages[i];
>
--
"This is the end, beautiful friend.
This is the end, my only friend the end
Of our elaborate plans, the end
Of everything that stands, the end
No safety or surprise, the end
I'll never look into your eyes again."
http://www.jlbec.org/
[email protected]
On Wed, Jan 23, 2013 at 01:43:12PM -0800, Andrew Morton wrote:
> On Fri, 18 Jan 2013 17:13:16 -0800
> "Darrick J. Wong" <[email protected]> wrote:
>
> > When stable pages are required, we have to wait if the page is just
> > going to disk and we want to modify it. Add proper callback to
> > ubifs_vm_page_mkwrite().
> >
> > CC: Artem Bityutskiy <[email protected]>
> > CC: Adrian Hunter <[email protected]>
> > CC: [email protected]
> > From: Jan Kara <[email protected]>
> > Signed-off-by: Jan Kara <[email protected]>
> > Signed-off-by: Darrick J. Wong <[email protected]>
>
> A couple of these patches had this From:Jan strangely embedded in the
> signoff area. I have assumed that they were indeed authored by Jan.
>
> Please note that authorship is indicated by putting the From: line
> right at the start of the chagnelog.
>
>
> I grabbed the patches. They should appear in linux-next tomorrow if I
> can get the current pooppile to build.
Well... these patches have been banging around in -next for a month or so now.
As far as I know there haven't been any complaints. Can we push these for 3.9?
--D
On 02/21/2013 05:48 AM, Darrick J. Wong wrote:
> On Wed, Jan 23, 2013 at 01:43:12PM -0800, Andrew Morton wrote:
>> On Fri, 18 Jan 2013 17:13:16 -0800
>> "Darrick J. Wong" <[email protected]> wrote:
>>
>>> When stable pages are required, we have to wait if the page is just
>>> going to disk and we want to modify it. Add proper callback to
>>> ubifs_vm_page_mkwrite().
>>>
>>> CC: Artem Bityutskiy <[email protected]>
>>> CC: Adrian Hunter <[email protected]>
>>> CC: [email protected]
>>> From: Jan Kara <[email protected]>
>>> Signed-off-by: Jan Kara <[email protected]>
>>> Signed-off-by: Darrick J. Wong <[email protected]>
>>
>> A couple of these patches had this From:Jan strangely embedded in the
>> signoff area. I have assumed that they were indeed authored by Jan.
>>
>> Please note that authorship is indicated by putting the From: line
>> right at the start of the chagnelog.
>>
>>
>> I grabbed the patches. They should appear in linux-next tomorrow if I
>> can get the current pooppile to build.
>
> Well... these patches have been banging around in -next for a month or so now.
> As far as I know there haven't been any complaints. Can we push these for 3.9?
>
Yes, please I'm waiting for these patches as well. Lets push them this merge
window. I was sure they would get in at 3.8, but they didn't. What's the delay?
[Using this I can fix a theoretical raid corruption in exofs local access, which
no one really cared because exofs is always accessed via pnfs, which does not
have that bug]
Thanks
Boaz
> --D
>
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/
On Wed, 20 Feb 2013 19:48:34 -0800
"Darrick J. Wong" <[email protected]> wrote:
> > I grabbed the patches. They should appear in linux-next tomorrow if I
> > can get the current pooppile to build.
>
> Well... these patches have been banging around in -next for a month or so now.
> As far as I know there haven't been any complaints. Can we push these for 3.9?
yup. You can normally assume that this is the case, unless the patches have
a) been causing bugs or
b) been getting rude review comments or
c) been getting a great string of fix-fix-fix-fix patches in -mm or
d) acquired rude akpm comments or unresolved questions against them
in http://ozlabs.org/~akpm/mmots/series
On Thu, Feb 21, 2013 at 02:32:43PM -0800, Andrew Morton wrote:
> On Wed, 20 Feb 2013 19:48:34 -0800
> "Darrick J. Wong" <[email protected]> wrote:
>
> > > I grabbed the patches. They should appear in linux-next tomorrow if I
> > > can get the current pooppile to build.
> >
> > Well... these patches have been banging around in -next for a month or so now.
> > As far as I know there haven't been any complaints. Can we push these for 3.9?
>
> yup. You can normally assume that this is the case, unless the patches have
>
> a) been causing bugs or
>
> b) been getting rude review comments or
>
> c) been getting a great string of fix-fix-fix-fix patches in -mm or
>
> d) acquired rude akpm comments or unresolved questions against them
> in http://ozlabs.org/~akpm/mmots/series
I was simply making sure that I hadn't missed anything. :)
--D
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/