MIME-Version: 1.0
Reply-To: sedat.dilek@gmail.com
In-Reply-To: <1345045700-9062-1-git-send-email-miklos@szeredi.hu>
References: <1345045700-9062-1-git-send-email-miklos@szeredi.hu>
Date: Wed, 15 Aug 2012 19:14:06 +0200
Message-ID: <CA+icZUXvsFUesuVNVrssLKPDxp4exJB0eTCFoqzh12JToSBCjA@mail.gmail.com>
Subject: Re: [PATCH 00/13] overlay filesystem: request for inclusion (v14)
From: Sedat Dilek <sedat.dilek@gmail.com>
To: Miklos Szeredi <miklos@szeredi.hu>
Cc: viro@zeniv.linux.org.uk, linux-fsdevel@vger.kernel.org,
        linux-kernel@vger.kernel.org, hch@infradead.org,
        torvalds@linux-foundation.org, akpm@linux-foundation.org,
        apw@canonical.com, nbd@openwrt.org, neilb@suse.de, hramrach@centrum.cz,
        jordipujolp@gmail.com, ezk@fsl.cs.sunysb.edu, ricwheeler@gmail.com,
        dhowells@redhat.com, hpj@urpla.net, sedat.dilek@googlemail.com,
        penberg@kernel.org, goran.cetusic@gmail.com, romain@orebokech.com,
        mszeredi@suse.cz, Stephen Rothwell <sfr@canb.auug.org.au>
Content-Type: text/plain; charset=UTF-8
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 11890
Lines: 336

On Wed, Aug 15, 2012 at 5:48 PM, Miklos Szeredi <miklos@szeredi.hu> wrote:
> Here's the latest version of the overlayfs series.
>
> Git tree is here:
>
>   git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.v14
>
> Please consider for 3.7.
>

[ Stephen (linux-next) maintainer ]

Wouldn't it be a good idea to ask Stephen to include your tree into linux-next?
Many people build linux-next manually or with a lot of rand-configs on
a daily base.

With Linux-3.6-rc1 merge-window I have noticed (for the first time) a
lot of merges which got pulled in "blindly" with the remark "matured
in linux-next".
So, chances will be higher that OverlayFS is more and better tested
and might be accepted (hopefully).
My 0.02EUR.

Your patch "vfs: export __inode_permission() to modules" can go
through Al's vfs tree?
When I saw [1] in the first pile of vfs patches for v3.6-rc1 (first
bits of/for union-mounts!) I talked with Andy that the export is
needed when OverlayFS is built as a module.
But I will comment separately on this.

- Sedat -

[1] http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=0bdaea9017b9d2b9996e153a71ee03555969b80e

> Thanks,
> Miklos
>
>
> ---
> Andy Whitcroft (3):
>       overlayfs: add statfs support
>       ovl: switch to __inode_permission()
>       overlayfs: copy up i_uid/i_gid from the underlying inode
>
> Erez Zadok (1):
>       overlayfs: implement show_options
>
> Miklos Szeredi (6):
>       vfs: add i_op->open()
>       vfs: export do_splice_direct() to modules
>       vfs: introduce clone_private_mount()
>       overlay filesystem
>       fs: limit filesystem stacking depth
>       vfs: export __inode_permission() to modules
>
> Neil Brown (1):
>       overlay: overlay filesystem documentation
>
> Robin Dong (2):
>       overlayfs: fix possible leak in ovl_new_inode
>       overlayfs: create new inode in ovl_link
>
> ---
>  Documentation/filesystems/Locking       |    2 +
>  Documentation/filesystems/overlayfs.txt |  199 +++++++++
>  Documentation/filesystems/vfs.txt       |    7 +
>  MAINTAINERS                             |    7 +
>  fs/Kconfig                              |    1 +
>  fs/Makefile                             |    1 +
>  fs/ecryptfs/main.c                      |    7 +
>  fs/internal.h                           |    5 -
>  fs/namei.c                              |   10 +-
>  fs/namespace.c                          |   18 +
>  fs/open.c                               |   23 +-
>  fs/overlayfs/Kconfig                    |    4 +
>  fs/overlayfs/Makefile                   |    7 +
>  fs/overlayfs/copy_up.c                  |  385 ++++++++++++++++++
>  fs/overlayfs/dir.c                      |  604 ++++++++++++++++++++++++++++
>  fs/overlayfs/inode.c                    |  372 +++++++++++++++++
>  fs/overlayfs/overlayfs.h                |   70 ++++
>  fs/overlayfs/readdir.c                  |  566 ++++++++++++++++++++++++++
>  fs/overlayfs/super.c                    |  665 +++++++++++++++++++++++++++++++
>  fs/splice.c                             |    1 +
>  include/linux/fs.h                      |   14 +
>  include/linux/mount.h                   |    3 +
>  22 files changed, 2961 insertions(+), 10 deletions(-)
>  create mode 100644 Documentation/filesystems/overlayfs.txt
>  create mode 100644 fs/overlayfs/Kconfig
>  create mode 100644 fs/overlayfs/Makefile
>  create mode 100644 fs/overlayfs/copy_up.c
>  create mode 100644 fs/overlayfs/dir.c
>  create mode 100644 fs/overlayfs/inode.c
>  create mode 100644 fs/overlayfs/overlayfs.h
>  create mode 100644 fs/overlayfs/readdir.c
>  create mode 100644 fs/overlayfs/super.c
>
> ------------------------------------------------------------------------------
> Changes from v13 to v14
>
> - update to 3.6
>
> - copy i_uid/i_gid from the underlying inode (patch by Andy Whitcroft)
>
> ------------------------------------------------------------------------------
> Changes from v12 to v13
>
> - create new inode in ovl_link (patch by Robin Dong)
>
> - switch to __inode_permission() (patch by Andy Whitcroft)
>
> ------------------------------------------------------------------------------
> Changes from v11 to v12
>
> - update to for-next of vfs tree
>
> - split __dentry_open argument cleanup patch from vfs-add-i_op-open.patch
>
> - change i_op->open and vfs_open so that they take "struct file *"
>
> ------------------------------------------------------------------------------
> Changes from v10 to v11
>
> - fix overlayfs over overlayfs
>
> - improve stack use of lookup and readdir
>
> - add limitations to documentation
>
> - make lower mount read-only
>
> - update permission and fsync to new API
>
> ------------------------------------------------------------------------------
> Changes from v9 to v10
>
> - prevent d_delete() from turning upperdentry negative (reported by
>   Erez Zadok)
>
> - show mount options in /proc/mounts and friends (patch by Erez Zadok)
>
> - fix off-by-one error in readdir (reported by Jordi Pujol)
>
> ------------------------------------------------------------------------------
> Changes from v8 to v9
>
> - support xattr on tmpfs
>
> - fix build after split-up
>
> - fix remove after rename (reported by Jordi Pujol)
>
> - fix rename failure case
>
> ------------------------------------------------------------------------------
> Changes from v7 to v8:
>
> - split overlayfs.c into smaller files
>
> - fix locking for copy up (reported by Al Viro)
>
> - locking analysis of copy up vs. directory rename added as a comment
>
> - tested with lockdep, fixed one lock annotation
>
> - other bug fixes
>
> ------------------------------------------------------------------------------
> Changes from v6 to v7
>
> - added patches from Felix Fietkau to fix deadlocks on jffs2
>
> - optimized directory removal
>
> - properly clean up after copy-up and other failures
>
> ------------------------------------------------------------------------------
> Changes from v5 to v6
>
> - optimize directory merging
>
>   o use rbtree for weeding out duplicates
>
>   o use a cursor for current position within the stream
>
> - instead of f_op->open_other(), implement i_op->open()
>
> - don't share inodes for non-directory dentries - for now.  I hope
>   this can come back once RCU lookup code has settled.
>
> - misc bug fixes
>
> ------------------------------------------------------------------------------
> Changes from v4 to v5
>
> - fix copying up if fs doesn't support xattrs (Andy Whitcroft)
>
> - clone mounts to be used internally to access the underlying
>   filesystems
>
> ------------------------------------------------------------------------------
> Changes from v3 to v4
>
> - export security_inode_permission to allow overlayfs to be modular
>   (Andy Whitcroft)
>
> - add statfs support (Andy Whitcroft)
>
> - change BUG_ON to WARN_ON
>
> - Revert "vfs: add flag to allow rename to same inode", instead
>   introduce s_op->is_same_inode()
>
> - overlayfs: fix rename to self
>
> - fix whiteout after rename
>
> ------------------------------------------------------------------------------
> Changes from v2 to v3
>
>  - Minimal remount support.  As overlayfs reflects the 'readonly'
>    mount status in write-access to the upper filesystem, we must
>    handle remount and either drop or take write access when the ro
>    status changes. (NeilBrown)
>
>  - Use correct seek function for directories.  It is incorrect to call
>    generic_llseek_file on a file from a different filesystem.  For
>    that we must use the seek function that the filesystem defines,
>    which is called by vfs_llseek.  Also, we only want to seek the
>    realfile when is_real is true.  Otherwise we just want to update
>    our own f_pos pointer, so use generic_llseek_file for
>    that. (NeilBrown)
>
>  - Initialise is_real before use.  The previous patch can use
>    od->is_real before it is properly initialised is llseek is called
>    before readdir.  So factor out the initialisation of is_real and
>    call it from both readdir and llseek when f_pos is 0. (NeilBrown)
>
>  - Rename ovl_fill_cache to ovl_dir_read (NeilBrown)
>
>  - Tiny optimisation in open_other handling (NeilBrown)
>
>  - Assorted updates to Documentation/filesystems/overlayfs.txt (NeilBrown)
>
>  - Make copy-up work for >=4G files, make it killable during copy-up.
>    Need to fix recovery after a failed/interrupted copy-up.
>
>  - Store and reference upper/lower dentries in overlay dentries.
>    Store and reference upper/lower vfsmounts in overlay superblock.
>
>  - Add necessary barriers for setting upper dentry in copyup and for
>    retrieving upper dentry locklessly.
>
>  - Make sure the right file is used for directory fsync() after
>    copy-up.
>
>  - Add locking to ovl_dir_llseek() to prevent concurrent call of
>    ovl_dir_reset() with ovl_dir_read().
>
>  - Get rid of ovl_dentry_iput().  The VFS doesn't provide enough
>    locking for this function that the contents of ->d_fsdata could be
>    safely updated.
>
>  - After copying up a non-directory unhash the dentry.  This way the
>    lower dentry ref, which is no longer necessary, can go away.  This
>    revealed a use-after-free bug in truncate handling in
>    fs/namei.c:finish_open().
>
>  - Fix if a copy-up happens between the follow_linka the put_link
>    calls.
>
>  - Replace some WARN_ONs with BUG_ON.  Some things just _really_
>    shouldn't happen.
>
>  - Extract common code from ovl_unlink and ovl_rmdir to a helper
>    function.
>
>  - After unlink and rmdir unhash the dentry.  This will get rid of the
>    lower and upper dentry references after there are no more users of
>    the deleted dentry.  This is a safe replacement for the removed
>    ->d_iput() functionality.
>
>  - Added checks to unlink, rmdir and rename to verify that the
>    parent-child relationship in the upper filesystem matches that of
>    the overlay.  This is necessary to prevent crash and/or corruption
>    if the upper filesystem topology is being modified while part of
>    the overlay.
>
>  - Optimize checking whiteout and opaque attributes.
>
>  - Optimize copy-up on truncate: don't copy up whole file before
>    truncating
>
>  - Misc bug fixes
>
> ------------------------------------------------------------------------------
> Changes from v1 to v2
>
>  - rename "hybrid union filesystem" to "overlay filesystem" or overlayfs
>
>  - added documentation written by Neil
>
>  - correct st_dev for directories (reported by Neil)
>
>  - use getattr() to get attributes from the underlying filesystems,
>    this means that now an overlay filesystem itself can be the lower,
>    read-only layer of another overlay
>
>  - listxattr filters out private extended attributes
>
>  - get write ref on the upper layer on mount unless the overlay
>    itself is mounted read-only
>
>  - raise capabilities for copy up, dealing with whiteouts and opaque
>    directories.  Now the overlay works for non-root users as well
>
>  - "rm -rf" didn't work correctly in all cases if the directory was
>    copied up between opendir and the first readdir, this is now fixed
>    (and the directory operations consolidated)
>
>  - simplified copy up, this broke optimization for truncate and
>    open(O_TRUNC) (now file is copied up to be immediately truncated,
>    will fix)
>
>  - st_nlink for merged directories set to 1, this is an "illegal"
>    value that normal filesystems never have but some use it to
>    indicate that the number of subdirectories is unknown.  Utilities
>    (find, ...) seem to tolerate this well.
>
>  - misc fixes I forgot about
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/