Hello!
Attempting to understand the implementation of server-side copying of
folders mounted via NFS 4.2.
Currently copying an entire folder (cp -r --reflink=always) produces:
newfstatat(AT_FDCWD, "/mnt/source_file", {st_mode=S_IFREG|0644,
st_size=2466, ...}, AT_SYMLINK_NOFOLLOW) = 0
newfstatat(3, "dest_file", {st_mode=S_IFREG|0644, st_size=2466, ...}, 0) = 0
openat(AT_FDCWD, "/mnt/source_file", O_RDONLY|O_NOFOLLOW) = 4
newfstatat(4, "", {st_mode=S_IFREG|0644, st_size=2466, ...}, AT_EMPTY_PATH) = 0
openat(3, "dest_file", O_WRONLY|O_TRUNC) = 5
ioctl(5, BTRFS_IOC_CLONE or FICLONE, 4) = 0
close(5) = 0
close(4)
sequence of operations for each file within the directory on the
client side. Notably, the actual file copy occurs on the server side
and is instantaneous (BTRFS_IOC_CLONE).
But for TTLs around 50 ms (such as within intra-regional connections
like US West/East), copying a 700MB directory containing 500 files
takes about 4 minutes (while "true" server-side folder copy is almost
instant).
Exploring if there exists a Linux mechanism enabling the copying of an
entire folder without individually accessing each file within the
directory for server-side copy operation.
On 21 Apr 2024, at 13:37, Robert Ayrapetyan wrote:
> Hello!
>
> Attempting to understand the implementation of server-side copying of
> folders mounted via NFS 4.2.
> Currently copying an entire folder (cp -r --reflink=always) produces:
>
> newfstatat(AT_FDCWD, "/mnt/source_file", {st_mode=S_IFREG|0644,
> st_size=2466, ...}, AT_SYMLINK_NOFOLLOW) = 0
> newfstatat(3, "dest_file", {st_mode=S_IFREG|0644, st_size=2466, ...}, 0) = 0
> openat(AT_FDCWD, "/mnt/source_file", O_RDONLY|O_NOFOLLOW) = 4
> newfstatat(4, "", {st_mode=S_IFREG|0644, st_size=2466, ...}, AT_EMPTY_PATH) = 0
> openat(3, "dest_file", O_WRONLY|O_TRUNC) = 5
> ioctl(5, BTRFS_IOC_CLONE or FICLONE, 4) = 0
> close(5) = 0
> close(4)
>
> sequence of operations for each file within the directory on the
> client side. Notably, the actual file copy occurs on the server side
> and is instantaneous (BTRFS_IOC_CLONE).
> But for TTLs around 50 ms (such as within intra-regional connections
> like US West/East), copying a 700MB directory containing 500 files
> takes about 4 minutes (while "true" server-side folder copy is almost
> instant).
Hi Robert,
Looks like `cp` is serialized? Is there a way to send COPY/CLONE in
parallel?
> Exploring if there exists a Linux mechanism enabling the copying of an
> entire folder without individually accessing each file within the
> directory for server-side copy operation.
The NFSv4 protocol doesn't have an operation that can do a server-side
copy/clone of a folder and all its files. It'd be quite an interesting one,
though. The failure and recovery cases would be sprawling.
Ben
> On Apr 21, 2024, at 1:37 PM, Robert Ayrapetyan <[email protected]> wrote:
>
> Hello!
>
> Attempting to understand the implementation of server-side copying of
> folders mounted via NFS 4.2.
> Currently copying an entire folder (cp -r --reflink=always) produces:
>
> newfstatat(AT_FDCWD, "/mnt/source_file", {st_mode=S_IFREG|0644,
> st_size=2466, ...}, AT_SYMLINK_NOFOLLOW) = 0
> newfstatat(3, "dest_file", {st_mode=S_IFREG|0644, st_size=2466, ...}, 0) = 0
> openat(AT_FDCWD, "/mnt/source_file", O_RDONLY|O_NOFOLLOW) = 4
> newfstatat(4, "", {st_mode=S_IFREG|0644, st_size=2466, ...}, AT_EMPTY_PATH) = 0
> openat(3, "dest_file", O_WRONLY|O_TRUNC) = 5
> ioctl(5, BTRFS_IOC_CLONE or FICLONE, 4) = 0
> close(5) = 0
> close(4)
>
> sequence of operations for each file within the directory on the
> client side. Notably, the actual file copy occurs on the server side
> and is instantaneous (BTRFS_IOC_CLONE).
> But for TTLs around 50 ms (such as within intra-regional connections
> like US West/East), copying a 700MB directory containing 500 files
> takes about 4 minutes (while "true" server-side folder copy is almost
> instant).
>
> Exploring if there exists a Linux mechanism enabling the copying of an
> entire folder without individually accessing each file within the
> directory for server-side copy operation.
The Linux FS community has discussed this issue in the
past.
The usual approach is to teach the user space utilities
(tar, rsync, and the like) to kick off file copies in
parallel.
--
Chuck Lever