This patch series fixes a bug reported by Hui Zhu; see proposed
patches v1 and v2:
https://lore.kernel.org/linux-fsdevel/[email protected]/
https://lore.kernel.org/linux-fsdevel/[email protected]/
I decided to go in a rather different direction for this fix, and
fix a related problem at the same time. I don't think there's any
urgency to rush this into Linus' tree, nor have I marked it for stable.
Reasonable people may disagree.
Matthew Wilcox (Oracle) (3):
buffer: Pass GFP flags to folio_alloc_buffers()
buffer: Hoist GFP flags from grow_dev_page() to __getblk_gfp()
ext4: Use bdev_getblk() to avoid memory reclaim in readahead path
fs/buffer.c | 75 +++++++++++++++++++++++--------------
fs/ext4/super.c | 3 +-
include/linux/buffer_head.h | 4 +-
3 files changed, 52 insertions(+), 30 deletions(-)
--
2.40.1
Instead of creating entirely new flags, inherit them from grow_dev_page().
The other callers create the same flags that this function used to create.
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
---
fs/buffer.c | 17 +++++++++--------
include/linux/buffer_head.h | 2 +-
2 files changed, 10 insertions(+), 9 deletions(-)
diff --git a/fs/buffer.c b/fs/buffer.c
index bd091329026c..7326acc29541 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -920,16 +920,12 @@ int remove_inode_buffers(struct inode *inode)
* which may not fail from ordinary buffer allocations.
*/
struct buffer_head *folio_alloc_buffers(struct folio *folio, unsigned long size,
- bool retry)
+ gfp_t gfp)
{
struct buffer_head *bh, *head;
- gfp_t gfp = GFP_NOFS | __GFP_ACCOUNT;
long offset;
struct mem_cgroup *memcg, *old_memcg;
- if (retry)
- gfp |= __GFP_NOFAIL;
-
/* The folio lock pins the memcg */
memcg = folio_memcg(folio);
old_memcg = set_active_memcg(memcg);
@@ -972,7 +968,11 @@ EXPORT_SYMBOL_GPL(folio_alloc_buffers);
struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size,
bool retry)
{
- return folio_alloc_buffers(page_folio(page), size, retry);
+ gfp_t gfp = GFP_NOFS | __GFP_ACCOUNT;
+ if (retry)
+ gfp |= __GFP_NOFAIL;
+
+ return folio_alloc_buffers(page_folio(page), size, gfp);
}
EXPORT_SYMBOL_GPL(alloc_page_buffers);
@@ -1074,7 +1074,7 @@ grow_dev_page(struct block_device *bdev, sector_t block,
goto failed;
}
- bh = folio_alloc_buffers(folio, size, true);
+ bh = folio_alloc_buffers(folio, size, gfp_mask);
/*
* Link the folio to the buffers and initialise them. Take the
@@ -1665,8 +1665,9 @@ void folio_create_empty_buffers(struct folio *folio, unsigned long blocksize,
unsigned long b_state)
{
struct buffer_head *bh, *head, *tail;
+ gfp_t gfp = GFP_NOFS | __GFP_ACCOUNT | __GFP_NOFAIL;
- head = folio_alloc_buffers(folio, blocksize, true);
+ head = folio_alloc_buffers(folio, blocksize, gfp);
bh = head;
do {
bh->b_state |= b_state;
diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h
index 6cb3e9af78c9..d17efb8b7976 100644
--- a/include/linux/buffer_head.h
+++ b/include/linux/buffer_head.h
@@ -200,7 +200,7 @@ void folio_set_bh(struct buffer_head *bh, struct folio *folio,
unsigned long offset);
bool try_to_free_buffers(struct folio *);
struct buffer_head *folio_alloc_buffers(struct folio *folio, unsigned long size,
- bool retry);
+ gfp_t gfp);
struct buffer_head *alloc_page_buffers(struct page *page, unsigned long size,
bool retry);
void create_empty_buffers(struct page *, unsigned long,
--
2.40.1
sb_getblk_gfp adds __GFP_NOFAIL, which is unnecessary for readahead;
we're quite comfortable with the possibility that we may not get a bh
back. Switch to bdev_getblk() which does not include __GFP_NOFAIL.
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Reported-by: Hui Zhu <[email protected]>
Link: https://lore.kernel.org/linux-fsdevel/[email protected]/
---
fs/ext4/super.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/fs/ext4/super.c b/fs/ext4/super.c
index c94ebf704616..48524314be97 100644
--- a/fs/ext4/super.c
+++ b/fs/ext4/super.c
@@ -254,7 +254,8 @@ struct buffer_head *ext4_sb_bread_unmovable(struct super_block *sb,
void ext4_sb_breadahead_unmovable(struct super_block *sb, sector_t block)
{
- struct buffer_head *bh = sb_getblk_gfp(sb, block, 0);
+ struct buffer_head *bh = bdev_getblk(sb->s_bdev, block,
+ sb->s_blocksize, GFP_NOWAIT);
if (likely(bh)) {
if (trylock_buffer(bh))
--
2.40.1
grow_dev_page() is only called by grow_buffers(). grow_buffers()
is only called by __getblk_slow() and __getblk_slow() is only called
from __getblk_gfp(), so it is safe to move the GFP flags setting
all the way up. With that done, add a new bdev_getblk() entry point
that leaves the GFP flags the way the caller specified them.
Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
---
fs/buffer.c | 60 ++++++++++++++++++++++++-------------
include/linux/buffer_head.h | 2 ++
2 files changed, 41 insertions(+), 21 deletions(-)
diff --git a/fs/buffer.c b/fs/buffer.c
index 7326acc29541..122b7d16befb 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -1048,20 +1048,11 @@ grow_dev_page(struct block_device *bdev, sector_t block,
struct buffer_head *bh;
sector_t end_block;
int ret = 0;
- gfp_t gfp_mask;
-
- gfp_mask = mapping_gfp_constraint(inode->i_mapping, ~__GFP_FS) | gfp;
-
- /*
- * XXX: __getblk_slow() can not really deal with failure and
- * will endlessly loop on improvised global reclaim. Prefer
- * looping in the allocator rather than here, at least that
- * code knows what it's doing.
- */
- gfp_mask |= __GFP_NOFAIL;
folio = __filemap_get_folio(inode->i_mapping, index,
- FGP_LOCK | FGP_ACCESSED | FGP_CREAT, gfp_mask);
+ FGP_LOCK | FGP_ACCESSED | FGP_CREAT, gfp);
+ if (IS_ERR(folio))
+ return PTR_ERR(folio);
bh = folio_buffers(folio);
if (bh) {
@@ -1074,7 +1065,9 @@ grow_dev_page(struct block_device *bdev, sector_t block,
goto failed;
}
- bh = folio_alloc_buffers(folio, size, gfp_mask);
+ bh = folio_alloc_buffers(folio, size, gfp);
+ if (!bh)
+ goto failed;
/*
* Link the folio to the buffers and initialise them. Take the
@@ -1426,24 +1419,49 @@ __find_get_block(struct block_device *bdev, sector_t block, unsigned size)
}
EXPORT_SYMBOL(__find_get_block);
+/**
+ * bdev_getblk - Get a buffer_head in a block device's buffer cache.
+ * @bdev: The block device.
+ * @block: The block number.
+ * @size: The size of buffer_heads for this @bdev.
+ * @gfp: The memory allocation flags to use.
+ *
+ * In contrast to __getblk_gfp(), the @gfp flags must be all of the flags;
+ * they are not augmented with the mapping's GFP flags.
+ *
+ * Return: The buffer head, or NULL if memory could not be allocated.
+ */
+struct buffer_head *bdev_getblk(struct block_device *bdev, sector_t block,
+ unsigned size, gfp_t gfp)
+{
+ struct buffer_head *bh = __find_get_block(bdev, block, size);
+
+ might_alloc(gfp);
+ if (bh)
+ return bh;
+
+ return __getblk_slow(bdev, block, size, gfp);
+}
+EXPORT_SYMBOL(bdev_getblk);
+
/*
* __getblk_gfp() will locate (and, if necessary, create) the buffer_head
* which corresponds to the passed block_device, block and size. The
* returned buffer has its reference count incremented.
- *
- * __getblk_gfp() will lock up the machine if grow_dev_page's
- * try_to_free_buffers() attempt is failing. FIXME, perhaps?
*/
struct buffer_head *
__getblk_gfp(struct block_device *bdev, sector_t block,
unsigned size, gfp_t gfp)
{
- struct buffer_head *bh = __find_get_block(bdev, block, size);
+ gfp |= mapping_gfp_constraint(bdev->bd_inode->i_mapping, ~__GFP_FS);
- might_sleep();
- if (bh == NULL)
- bh = __getblk_slow(bdev, block, size, gfp);
- return bh;
+ /*
+ * Prefer looping in the allocator rather than here, at least that
+ * code knows what it's doing.
+ */
+ gfp |= __GFP_NOFAIL;
+
+ return bdev_getblk(bdev, block, size, gfp);
}
EXPORT_SYMBOL(__getblk_gfp);
diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h
index d17efb8b7976..01110db9213c 100644
--- a/include/linux/buffer_head.h
+++ b/include/linux/buffer_head.h
@@ -233,6 +233,8 @@ void __wait_on_buffer(struct buffer_head *);
wait_queue_head_t *bh_waitq_head(struct buffer_head *bh);
struct buffer_head *__find_get_block(struct block_device *bdev, sector_t block,
unsigned size);
+struct buffer_head *bdev_getblk(struct block_device *bdev, sector_t block,
+ unsigned size, gfp_t gfp);
struct buffer_head *__getblk_gfp(struct block_device *bdev, sector_t block,
unsigned size, gfp_t gfp);
void __brelse(struct buffer_head *);
--
2.40.1
On Fri, Aug 11, 2023 at 05:15:28PM +0100, Matthew Wilcox (Oracle) wrote:
> + struct buffer_head *bh = bdev_getblk(sb->s_bdev, block,
> + sb->s_blocksize, GFP_NOWAIT);
Doh, I missed the __GFP_ACCOUNT from that. Will wait a day or two
for other feedback.
It occurs to me that this would also be useful and I'll include it in
the next version:
diff --git a/fs/buffer.c b/fs/buffer.c
index 122b7d16befb..b551a5b1196b 100644
--- a/fs/buffer.c
+++ b/fs/buffer.c
@@ -1470,7 +1470,9 @@ EXPORT_SYMBOL(__getblk_gfp);
*/
void __breadahead(struct block_device *bdev, sector_t block, unsigned size)
{
- struct buffer_head *bh = __getblk(bdev, block, size);
+ struct buffer_head *bh = bdev_getblk(bdev, block, size,
+ GFP_NOWAIT | __GFP_MOVABLE | __GFP_ACCOUNT);
+
if (likely(bh)) {
bh_readahead(bh, REQ_RAHEAD);
brelse(bh);
On Fri 11-08-23 17:15:27, Matthew Wilcox (Oracle) wrote:
> grow_dev_page() is only called by grow_buffers(). grow_buffers()
> is only called by __getblk_slow() and __getblk_slow() is only called
> from __getblk_gfp(), so it is safe to move the GFP flags setting
> all the way up. With that done, add a new bdev_getblk() entry point
> that leaves the GFP flags the way the caller specified them.
>
> Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
Can't we just finish this gfp parameter conversion for all the users?
There are five __getblk_gfp() users, three in buffer_head.h directly
generate gfp mask, two (__bread_gfp() and sb_getblk_gfp()) pass it from the
caller. All three __bread_gfp() callers are in buffer_head.h and directly
generate gfp mask. sb_getblk_gfp() has five callers, all in ext4 and easily
convertable as well.
This results not only in cleaner code but also just checking
sb_getblk_gfp() callers shows how confused they currently are about the gfp
argument (passing NOFS, NOFAIL and other pointless flags). Secondly, we can
keep using sb_getblk_gfp() from the filesystems instead of having to decide
between sb_getblk_gfp() and bdev_getblk().
If you don't have time for this, I guess I can find some...
Honza
> ---
> fs/buffer.c | 60 ++++++++++++++++++++++++-------------
> include/linux/buffer_head.h | 2 ++
> 2 files changed, 41 insertions(+), 21 deletions(-)
>
> diff --git a/fs/buffer.c b/fs/buffer.c
> index 7326acc29541..122b7d16befb 100644
> --- a/fs/buffer.c
> +++ b/fs/buffer.c
> @@ -1048,20 +1048,11 @@ grow_dev_page(struct block_device *bdev, sector_t block,
> struct buffer_head *bh;
> sector_t end_block;
> int ret = 0;
> - gfp_t gfp_mask;
> -
> - gfp_mask = mapping_gfp_constraint(inode->i_mapping, ~__GFP_FS) | gfp;
> -
> - /*
> - * XXX: __getblk_slow() can not really deal with failure and
> - * will endlessly loop on improvised global reclaim. Prefer
> - * looping in the allocator rather than here, at least that
> - * code knows what it's doing.
> - */
> - gfp_mask |= __GFP_NOFAIL;
>
> folio = __filemap_get_folio(inode->i_mapping, index,
> - FGP_LOCK | FGP_ACCESSED | FGP_CREAT, gfp_mask);
> + FGP_LOCK | FGP_ACCESSED | FGP_CREAT, gfp);
> + if (IS_ERR(folio))
> + return PTR_ERR(folio);
>
> bh = folio_buffers(folio);
> if (bh) {
> @@ -1074,7 +1065,9 @@ grow_dev_page(struct block_device *bdev, sector_t block,
> goto failed;
> }
>
> - bh = folio_alloc_buffers(folio, size, gfp_mask);
> + bh = folio_alloc_buffers(folio, size, gfp);
> + if (!bh)
> + goto failed;
>
> /*
> * Link the folio to the buffers and initialise them. Take the
> @@ -1426,24 +1419,49 @@ __find_get_block(struct block_device *bdev, sector_t block, unsigned size)
> }
> EXPORT_SYMBOL(__find_get_block);
>
> +/**
> + * bdev_getblk - Get a buffer_head in a block device's buffer cache.
> + * @bdev: The block device.
> + * @block: The block number.
> + * @size: The size of buffer_heads for this @bdev.
> + * @gfp: The memory allocation flags to use.
> + *
> + * In contrast to __getblk_gfp(), the @gfp flags must be all of the flags;
> + * they are not augmented with the mapping's GFP flags.
> + *
> + * Return: The buffer head, or NULL if memory could not be allocated.
> + */
> +struct buffer_head *bdev_getblk(struct block_device *bdev, sector_t block,
> + unsigned size, gfp_t gfp)
> +{
> + struct buffer_head *bh = __find_get_block(bdev, block, size);
> +
> + might_alloc(gfp);
> + if (bh)
> + return bh;
> +
> + return __getblk_slow(bdev, block, size, gfp);
> +}
> +EXPORT_SYMBOL(bdev_getblk);
> +
> /*
> * __getblk_gfp() will locate (and, if necessary, create) the buffer_head
> * which corresponds to the passed block_device, block and size. The
> * returned buffer has its reference count incremented.
> - *
> - * __getblk_gfp() will lock up the machine if grow_dev_page's
> - * try_to_free_buffers() attempt is failing. FIXME, perhaps?
> */
> struct buffer_head *
> __getblk_gfp(struct block_device *bdev, sector_t block,
> unsigned size, gfp_t gfp)
> {
> - struct buffer_head *bh = __find_get_block(bdev, block, size);
> + gfp |= mapping_gfp_constraint(bdev->bd_inode->i_mapping, ~__GFP_FS);
>
> - might_sleep();
> - if (bh == NULL)
> - bh = __getblk_slow(bdev, block, size, gfp);
> - return bh;
> + /*
> + * Prefer looping in the allocator rather than here, at least that
> + * code knows what it's doing.
> + */
> + gfp |= __GFP_NOFAIL;
> +
> + return bdev_getblk(bdev, block, size, gfp);
> }
> EXPORT_SYMBOL(__getblk_gfp);
>
> diff --git a/include/linux/buffer_head.h b/include/linux/buffer_head.h
> index d17efb8b7976..01110db9213c 100644
> --- a/include/linux/buffer_head.h
> +++ b/include/linux/buffer_head.h
> @@ -233,6 +233,8 @@ void __wait_on_buffer(struct buffer_head *);
> wait_queue_head_t *bh_waitq_head(struct buffer_head *bh);
> struct buffer_head *__find_get_block(struct block_device *bdev, sector_t block,
> unsigned size);
> +struct buffer_head *bdev_getblk(struct block_device *bdev, sector_t block,
> + unsigned size, gfp_t gfp);
> struct buffer_head *__getblk_gfp(struct block_device *bdev, sector_t block,
> unsigned size, gfp_t gfp);
> void __brelse(struct buffer_head *);
> --
> 2.40.1
>
--
Jan Kara <[email protected]>
SUSE Labs, CR
On Thu, Sep 14, 2023 at 11:16:25AM +0200, Jan Kara wrote:
> On Fri 11-08-23 17:15:27, Matthew Wilcox (Oracle) wrote:
> > grow_dev_page() is only called by grow_buffers(). grow_buffers()
> > is only called by __getblk_slow() and __getblk_slow() is only called
> > from __getblk_gfp(), so it is safe to move the GFP flags setting
> > all the way up. With that done, add a new bdev_getblk() entry point
> > that leaves the GFP flags the way the caller specified them.
> >
> > Signed-off-by: Matthew Wilcox (Oracle) <[email protected]>
>
> Can't we just finish this gfp parameter conversion for all the users?
> There are five __getblk_gfp() users, three in buffer_head.h directly
> generate gfp mask, two (__bread_gfp() and sb_getblk_gfp()) pass it from the
> caller. All three __bread_gfp() callers are in buffer_head.h and directly
> generate gfp mask. sb_getblk_gfp() has five callers, all in ext4 and easily
> convertable as well.
>
> This results not only in cleaner code but also just checking
> sb_getblk_gfp() callers shows how confused they currently are about the gfp
> argument (passing NOFS, NOFAIL and other pointless flags). Secondly, we can
> keep using sb_getblk_gfp() from the filesystems instead of having to decide
> between sb_getblk_gfp() and bdev_getblk().
I didn't do __bread_gfp() because it's basically an internal interface.
All users call sb_bread(), sb_bread_unmovable() or __bread().
It doesn't seem worth doing. Now, if we start to see people actually
using __bread_gfp() outside of those three interfaces, I'd agree we need
to make it use GFP flags properly.
BTW, Andrew has taken the bdev_getblk() series into the mm tree, so
testing that tree might be a good idea for the ext4 developers (and other
filesystems; an earlier revision of this patchset had a bug which would
have only affected nilfs2).