From: Trond Myklebust <[email protected]>
The following patch series performs a number of cleanups on the readdir
code.
It also adds support for 1MB readdir RPC calls on-the-wire, and modifies
the caching code to ensure that we cache the entire contents of that
1MB call (instead of discarding the data that doesn't fit into a single
page).
For filesystems that use ordered readdir cookie schemes (e.g. XFS), it
optimises searching for cookies in the client's page cache by skipping
over pages that contain cookie values that are not in the range we are
searching for.
Finally, it improves scalability when dealing with very large
directories by turning off caching when those directories are changing,
so as to avoid the need for a linear search on the client of the entire
directory when looking for the first entry pointed to by the current
file descriptor offset.
v2: Fix the handling of the NFSv3/v4 directory verifier.
v3: Optimise searching when the readdir cookies are seen to be ordered.
v4: Optimise performance for large directories that are changing.
Add in llseek dependency patches.
Trond Myklebust (21):
NFS: Remove unnecessary inode locking in nfs_llseek_dir()
NFS: Remove unnecessary inode lock in nfs_fsync_dir()
NFS: Ensure contents of struct nfs_open_dir_context are consistent
NFS: Clean up readdir struct nfs_cache_array
NFS: Clean up nfs_readdir_page_filler()
NFS: Clean up directory array handling
NFS: Don't discard readdir results
NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array()
NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array()
NFS: Simplify struct nfs_cache_array_entry
NFS: Support larger readdir buffers
NFS: More readdir cleanups
NFS: nfs_do_filldir() does not return a value
NFS: Reduce readdir stack usage
NFS: Cleanup to remove nfs_readdir_descriptor_t typedef
NFS: Allow the NFS generic code to pass in a verifier to readdir
NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls
NFS: Improve handling of directory verifiers
NFS: Optimisations for monotonically increasing readdir cookies
NFS: Reduce number of RPC calls when doing uncached readdir
NFS: Do uncached readdir when we're seeking a cookie in an empty page
cache
fs/nfs/client.c | 4 +-
fs/nfs/dir.c | 702 +++++++++++++++++++++++++---------------
fs/nfs/inode.c | 7 -
fs/nfs/internal.h | 6 -
fs/nfs/nfs3proc.c | 35 +-
fs/nfs/nfs4proc.c | 40 +--
fs/nfs/proc.c | 18 +-
include/linux/nfs_fs.h | 9 +-
include/linux/nfs_xdr.h | 17 +-
9 files changed, 508 insertions(+), 330 deletions(-)
--
2.28.0
From: Trond Myklebust <[email protected]>
Remove the contentious inode lock, and instead provide thread safety
using the file->f_lock spinlock.
Signed-off-by: Trond Myklebust <[email protected]>
---
fs/nfs/dir.c | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/fs/nfs/dir.c b/fs/nfs/dir.c
index cb52db9a0cfb..e56b1bd99537 100644
--- a/fs/nfs/dir.c
+++ b/fs/nfs/dir.c
@@ -955,7 +955,6 @@ static int nfs_readdir(struct file *file, struct dir_context *ctx)
static loff_t nfs_llseek_dir(struct file *filp, loff_t offset, int whence)
{
- struct inode *inode = file_inode(filp);
struct nfs_open_dir_context *dir_ctx = filp->private_data;
dfprintk(FILE, "NFS: llseek dir(%pD2, %lld, %d)\n",
@@ -967,15 +966,15 @@ static loff_t nfs_llseek_dir(struct file *filp, loff_t offset, int whence)
case SEEK_SET:
if (offset < 0)
return -EINVAL;
- inode_lock(inode);
+ spin_lock(&filp->f_lock);
break;
case SEEK_CUR:
if (offset == 0)
return filp->f_pos;
- inode_lock(inode);
+ spin_lock(&filp->f_lock);
offset += filp->f_pos;
if (offset < 0) {
- inode_unlock(inode);
+ spin_unlock(&filp->f_lock);
return -EINVAL;
}
}
@@ -987,7 +986,7 @@ static loff_t nfs_llseek_dir(struct file *filp, loff_t offset, int whence)
dir_ctx->dir_cookie = 0;
dir_ctx->duped = 0;
}
- inode_unlock(inode);
+ spin_unlock(&filp->f_lock);
return offset;
}
--
2.28.0
On Sat, Nov 7, 2020 at 9:13 AM <[email protected]> wrote:
>
> From: Trond Myklebust <[email protected]>
>
> The following patch series performs a number of cleanups on the readdir
> code.
> It also adds support for 1MB readdir RPC calls on-the-wire, and modifies
> the caching code to ensure that we cache the entire contents of that
> 1MB call (instead of discarding the data that doesn't fit into a single
> page).
> For filesystems that use ordered readdir cookie schemes (e.g. XFS), it
> optimises searching for cookies in the client's page cache by skipping
> over pages that contain cookie values that are not in the range we are
> searching for.
> Finally, it improves scalability when dealing with very large
> directories by turning off caching when those directories are changing,
> so as to avoid the need for a linear search on the client of the entire
> directory when looking for the first entry pointed to by the current
> file descriptor offset.
>
> v2: Fix the handling of the NFSv3/v4 directory verifier.
> v3: Optimise searching when the readdir cookies are seen to be ordered.
> v4: Optimise performance for large directories that are changing.
> Add in llseek dependency patches.
>
> Trond Myklebust (21):
> NFS: Remove unnecessary inode locking in nfs_llseek_dir()
> NFS: Remove unnecessary inode lock in nfs_fsync_dir()
> NFS: Ensure contents of struct nfs_open_dir_context are consistent
> NFS: Clean up readdir struct nfs_cache_array
> NFS: Clean up nfs_readdir_page_filler()
> NFS: Clean up directory array handling
> NFS: Don't discard readdir results
> NFS: Remove unnecessary kmap in nfs_readdir_xdr_to_array()
> NFS: Replace kmap() with kmap_atomic() in nfs_readdir_search_array()
> NFS: Simplify struct nfs_cache_array_entry
> NFS: Support larger readdir buffers
> NFS: More readdir cleanups
> NFS: nfs_do_filldir() does not return a value
> NFS: Reduce readdir stack usage
> NFS: Cleanup to remove nfs_readdir_descriptor_t typedef
> NFS: Allow the NFS generic code to pass in a verifier to readdir
> NFS: Handle NFS4ERR_NOT_SAME and NFSERR_BADCOOKIE from readdir calls
> NFS: Improve handling of directory verifiers
> NFS: Optimisations for monotonically increasing readdir cookies
> NFS: Reduce number of RPC calls when doing uncached readdir
> NFS: Do uncached readdir when we're seeking a cookie in an empty page
> cache
>
Thanks for these last couple patches to handle changing directories /
uncached scenario better. I'm testing with this set now and should
have results later today.