[PULL for 3.18] overlay filesystem v24

[Miklos Szeredi]

I'd like to propose overlayfs for inclusion into 3.18.

Al, would you mind giving it a review?

Git tree is here:

git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.current

Thanks,
Miklos

---
Andy Whitcroft (1):
overlayfs: add statfs support

Erez Zadok (1):
overlayfs: implement show_options

Miklos Szeredi (11):
vfs: add i_op->dentry_open()
vfs: export do_splice_direct() to modules
vfs: export __inode_permission() to modules
vfs: introduce clone_private_mount()
vfs: export check_sticky()
vfs: add whiteout support
vfs: add RENAME_WHITEOUT
ext4: support RENAME_WHITEOUT
shmem: support RENAME_WHITEOUT
overlay filesystem
fs: limit filesystem stacking depth

Neil Brown (1):
overlay: overlay filesystem documentation

Sedat Dilek (1):
vfs: dcache: Export d_ancestor to modules

---
Documentation/filesystems/Locking | 2 +
Documentation/filesystems/overlayfs.txt | 198 +++++++
Documentation/filesystems/vfs.txt | 7 +
MAINTAINERS | 7 +
fs/Kconfig | 1 +
fs/Makefile | 1 +
fs/btrfs/ioctl.c | 20 +-
fs/dcache.c | 1 +
fs/ecryptfs/main.c | 7 +
fs/ext4/namei.c | 95 +++-
fs/internal.h | 7 -
fs/namei.c | 41 +-
fs/namespace.c | 27 +
fs/open.c | 23 +-
fs/overlayfs/Kconfig | 10 +
fs/overlayfs/Makefile | 7 +
fs/overlayfs/copy_up.c | 431 +++++++++++++++
fs/overlayfs/dir.c | 914 ++++++++++++++++++++++++++++++++
fs/overlayfs/inode.c | 408 ++++++++++++++
fs/overlayfs/overlayfs.h | 187 +++++++
fs/overlayfs/readdir.c | 518 ++++++++++++++++++
fs/overlayfs/super.c | 762 ++++++++++++++++++++++++++
fs/splice.c | 1 +
include/linux/fs.h | 37 ++
include/linux/mount.h | 3 +
include/uapi/linux/fs.h | 1 +
mm/shmem.c | 36 +-
27 files changed, 3694 insertions(+), 58 deletions(-)
create mode 100644 Documentation/filesystems/overlayfs.txt
create mode 100644 fs/overlayfs/Kconfig
create mode 100644 fs/overlayfs/Makefile
create mode 100644 fs/overlayfs/copy_up.c
create mode 100644 fs/overlayfs/dir.c
create mode 100644 fs/overlayfs/inode.c
create mode 100644 fs/overlayfs/overlayfs.h
create mode 100644 fs/overlayfs/readdir.c
create mode 100644 fs/overlayfs/super.c
--
To unsubscribe from this list: send the line "unsubscribe linux-unionfs" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Woody Suwalski]

I remember like if it was yesterday... ;-)
Around 3.8 Linus decided to include overlayfs in the kernel, so in 3.10
we were supposed to get it... And we keep waiting...

Woody

Post by Miklos Szeredi
I'd like to propose overlayfs for inclusion into 3.18.
Al, would you mind giving it a review?
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.current
Thanks,
Miklos
---
overlayfs: add statfs support
overlayfs: implement show_options
vfs: add i_op->dentry_open()
vfs: export do_splice_direct() to modules
vfs: export __inode_permission() to modules
vfs: introduce clone_private_mount()
vfs: export check_sticky()
vfs: add whiteout support
vfs: add RENAME_WHITEOUT
ext4: support RENAME_WHITEOUT
shmem: support RENAME_WHITEOUT
overlay filesystem
fs: limit filesystem stacking depth
overlay: overlay filesystem documentation
vfs: dcache: Export d_ancestor to modules
---
Documentation/filesystems/Locking | 2 +
Documentation/filesystems/overlayfs.txt | 198 +++++++
Documentation/filesystems/vfs.txt | 7 +
MAINTAINERS | 7 +
fs/Kconfig | 1 +
fs/Makefile | 1 +
fs/btrfs/ioctl.c | 20 +-
fs/dcache.c | 1 +
fs/ecryptfs/main.c | 7 +
fs/ext4/namei.c | 95 +++-
fs/internal.h | 7 -
fs/namei.c | 41 +-
fs/namespace.c | 27 +
fs/open.c | 23 +-
fs/overlayfs/Kconfig | 10 +
fs/overlayfs/Makefile | 7 +
fs/overlayfs/copy_up.c | 431 +++++++++++++++
fs/overlayfs/dir.c | 914 ++++++++++++++++++++++++++++++++
fs/overlayfs/inode.c | 408 ++++++++++++++
fs/overlayfs/overlayfs.h | 187 +++++++
fs/overlayfs/readdir.c | 518 ++++++++++++++++++
fs/overlayfs/super.c | 762 ++++++++++++++++++++++++++
fs/splice.c | 1 +
include/linux/fs.h | 37 ++
include/linux/mount.h | 3 +
include/uapi/linux/fs.h | 1 +
mm/shmem.c | 36 +-
27 files changed, 3694 insertions(+), 58 deletions(-)
create mode 100644 Documentation/filesystems/overlayfs.txt
create mode 100644 fs/overlayfs/Kconfig
create mode 100644 fs/overlayfs/Makefile
create mode 100644 fs/overlayfs/copy_up.c
create mode 100644 fs/overlayfs/dir.c
create mode 100644 fs/overlayfs/inode.c
create mode 100644 fs/overlayfs/overlayfs.h
create mode 100644 fs/overlayfs/readdir.c
create mode 100644 fs/overlayfs/super.c
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

--
To unsubscribe from this list: send the line "unsubscribe linux-unionfs" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[David Howells]

Post by Miklos Szeredi
I'd like to propose overlayfs for inclusion into 3.18.
Al, would you mind giving it a review?
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.current

Tested-by: David Howells <***@redhat.com>
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[J. R. Okajima]

Post by Miklos Szeredi
I'd like to propose overlayfs for inclusion into 3.18.
Al, would you mind giving it a review?
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.current

Does it mean overlayfs passed all your unionmount-testsuite? And does
the test suite contain tests for "inode-based" union? For example,
- read(2) may get the obsoleted filedata (fstat(2) for metadata too).
- fcntl(F_SETLK) may be broken by copy-up.
- inotify may not work when it refers to the file before being
copied-up.
- unnecessary copy-up may happen, for example mmap(MAP_PRIVATE) after
open(O_RDWR).
- exporting via NFS and fhandle systemcalls will not work.

A few releases ago, OFD file-lock was introduced to improve the
behaviour of POSIX lock. POSIX lock has made users confused and I am
afraid that the similar story will come up because of the "name-based"
union behaviour. Of course the story is not limited to the file-lock.

If I remember correctly, are you the one who consitunes the development
of UnionMount? Is the development totally stopped?
Next paragraph is what I wrote several times.
AUFS is an "inode-based" stackable filesystem and solved them many years
ago. But I have to admit that AUFS is big. Yes it is grown up.
I don't stop including overlayfs into mainline, but if the development
of UnionMount is really stopped, then I'd ask people to consider merging
aufs as well as overlayfs.

http://aufs.sf.net

J. R. Okajima
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Miklos Szeredi]

Post by J. R. Okajima

Post by Miklos Szeredi
I'd like to propose overlayfs for inclusion into 3.18.
Al, would you mind giving it a review?
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.current

Does it mean overlayfs passed all your unionmount-testsuite? And does
the test suite contain tests for "inode-based" union? For example,
- read(2) may get the obsoleted filedata (fstat(2) for metadata too).
- fcntl(F_SETLK) may be broken by copy-up.
- inotify may not work when it refers to the file before being
copied-up.
- unnecessary copy-up may happen, for example mmap(MAP_PRIVATE) after
open(O_RDWR).
- exporting via NFS and fhandle systemcalls will not work.

Most of this is explicitly documented.

Post by J. R. Okajima
A few releases ago, OFD file-lock was introduced to improve the
behaviour of POSIX lock. POSIX lock has made users confused and I am
afraid that the similar story will come up because of the "name-based"
union behaviour. Of course the story is not limited to the file-lock.
If I remember correctly, are you the one who consitunes the development
of UnionMount? Is the development totally stopped?
Next paragraph is what I wrote several times.
AUFS is an "inode-based" stackable filesystem and solved them many years
ago. But I have to admit that AUFS is big. Yes it is grown up.
I don't stop including overlayfs into mainline, but if the development
of UnionMount is really stopped, then I'd ask people to consider merging
aufs as well as overlayfs.

Union-mounts are namespace based as well. There's little semantic
difference between overlayfs and union-mounts. The difference is
mostly in the implementation.

It would be good to have an inode based union as well. I suggest (as
I suggested many times) to try slimming aufs to a bare minimum and
submit that. It may be easier to just start from scratch instead
trying to drop features from the existing codebase. I'd be happy to
review and generally help with such an effort.

Thanks,
Miklos
--
To unsubscribe from this list: send the line "unsubscribe linux-unionfs" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[J. R. Okajima]

Post by Miklos Szeredi
It would be good to have an inode based union as well. I suggest (as
I suggested many times) to try slimming aufs to a bare minimum and
submit that. It may be easier to just start from scratch instead
trying to drop features from the existing codebase. I'd be happy to
review and generally help with such an effort.

??
In 2009, I have posted the "feature-reduced" version of aufs2 as
following your advice. Also a git-branch called "aufs2-tmp-ro" was
created whose size was a half of aufs2's. Have you ever posted your
review comments about them?

It was 2005 when I got an idea of aufs. Its history is almost a decade.
In my current local aufs3 develpment branch, I got about 350
commits. And 680 and 1120 commits for aufs2 and aufs1 individually.

aufs3-linux.git#aufs3.x-rcN/25lktr$ git log1 --no-merges fs/aufs/*.c | wc -l
348
aufs2-2.6.git#aufs2.2-stdalone-38-lktr$ git log1 --no-merges fs/aufs/*.c | wc -l
685
aufs1.git#master$ git log1 --no-merges aufs/fs/aufs/*.c | wc -l
1126

Of cource it won't be 2000 commits to create the patch series you
wrote. I don't know how large the number will be but I am sure this is
incredibly tough work both in commiting and reviewing. Is it still
helpful and really meaningful for you to review? Don't you think it will
be easier to review the resulted code?
For example, this branch was created for linux-3.9 when Al Viro wrote
about the dir mutex lock in copy-up. It has only 20 commits.

aufs3-linux.git#mainline-v3.9-rc8-20130428$ git log1 --no-merges fs/aufs/*.c | wc -l
20


J. R. Okajima
--
To unsubscribe from this list: send the line "unsubscribe linux-unionfs" in
the body of a message to ***@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html

[Sedat Dilek]

Post by Miklos Szeredi
I'd like to propose overlayfs for inclusion into 3.18.
Al, would you mind giving it a review?
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs.git overlayfs.current

Nice to see that the overlayfs "dream" comes true.
overlayfs.current has now got a "review" - will there be a v25
pull-request soonish or is overlayfs.current still in WIP?
Where is that review process happening? On linux-fsdevel I could not
find any informations.

Some updated patches have no review credits - for example the big
overlay filesystem patch which mentions AV's review, etc.

Maintainers file could have a tree item pointing to your personal Git
repository.

Is the documentation up2date?

Some files below fs/overlayfs/ have copyright pointing to year 2011 by
Novell - needs an update?

Thought about a co-maintainer?

Docs could have a pointer to the testsuite by David Howells [1]?

...etc.

- Sedat -

[1] http://git.infradead.org/users/dhowells/unionmount-testsuite.git

Post by Miklos Szeredi
Thanks,
Miklos
---
overlayfs: add statfs support
overlayfs: implement show_options
vfs: add i_op->dentry_open()
vfs: export do_splice_direct() to modules
vfs: export __inode_permission() to modules
vfs: introduce clone_private_mount()
vfs: export check_sticky()
vfs: add whiteout support
vfs: add RENAME_WHITEOUT
ext4: support RENAME_WHITEOUT
shmem: support RENAME_WHITEOUT
overlay filesystem
fs: limit filesystem stacking depth
overlay: overlay filesystem documentation
vfs: dcache: Export d_ancestor to modules
---
Documentation/filesystems/Locking | 2 +
Documentation/filesystems/overlayfs.txt | 198 +++++++
Documentation/filesystems/vfs.txt | 7 +
MAINTAINERS | 7 +
fs/Kconfig | 1 +
fs/Makefile | 1 +
fs/btrfs/ioctl.c | 20 +-
fs/dcache.c | 1 +
fs/ecryptfs/main.c | 7 +
fs/ext4/namei.c | 95 +++-
fs/internal.h | 7 -
fs/namei.c | 41 +-
fs/namespace.c | 27 +
fs/open.c | 23 +-
fs/overlayfs/Kconfig | 10 +
fs/overlayfs/Makefile | 7 +
fs/overlayfs/copy_up.c | 431 +++++++++++++++
fs/overlayfs/dir.c | 914 ++++++++++++++++++++++++++++++++
fs/overlayfs/inode.c | 408 ++++++++++++++
fs/overlayfs/overlayfs.h | 187 +++++++
fs/overlayfs/readdir.c | 518 ++++++++++++++++++
fs/overlayfs/super.c | 762 ++++++++++++++++++++++++++
fs/splice.c | 1 +
include/linux/fs.h | 37 ++
include/linux/mount.h | 3 +
include/uapi/linux/fs.h | 1 +
mm/shmem.c | 36 +-
27 files changed, 3694 insertions(+), 58 deletions(-)
create mode 100644 Documentation/filesystems/overlayfs.txt
create mode 100644 fs/overlayfs/Kconfig
create mode 100644 fs/overlayfs/Makefile
create mode 100644 fs/overlayfs/copy_up.c
create mode 100644 fs/overlayfs/dir.c
create mode 100644 fs/overlayfs/inode.c
create mode 100644 fs/overlayfs/overlayfs.h
create mode 100644 fs/overlayfs/readdir.c
create mode 100644 fs/overlayfs/super.c
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
More majordomo info at http://vger.kernel.org/majordomo-info.html