2005-01-06 19:06:33

by Paul E. McKenney

[permalink] [raw]
Subject: [PATCH] fs: Restore files_lock and set_fs_root exports

Hello, Andrew,

Some export-removal work causes breakage for an out-of-tree filesystem.
Could you please apply the attached patch to restore the exports for
files_lock and set_fs_root?

Thanx, Paul

----- End forwarded message -----

diff -urpN -X ../dontdiff linux-2.5/fs/file_table.c linux-2.5-MVFS/fs/file_table.c
--- linux-2.5/fs/file_table.c Wed Jan 5 13:54:21 2005
+++ linux-2.5-MVFS/fs/file_table.c Wed Jan 5 17:12:53 2005
@@ -26,6 +26,7 @@ EXPORT_SYMBOL(files_stat); /* Needed by

/* public. Not pretty! */
spinlock_t __cacheline_aligned_in_smp files_lock = SPIN_LOCK_UNLOCKED;
+EXPORT_SYMBOL(files_lock);

static spinlock_t filp_count_lock = SPIN_LOCK_UNLOCKED;

diff -urpN -X ../dontdiff linux-2.5/fs/namespace.c linux-2.5-MVFS/fs/namespace.c
--- linux-2.5/fs/namespace.c Wed Jan 5 13:54:22 2005
+++ linux-2.5-MVFS/fs/namespace.c Wed Jan 5 17:12:08 2005
@@ -1207,6 +1207,7 @@ void set_fs_root(struct fs_struct *fs, s
mntput(old_rootmnt);
}
}
+EXPORT_SYMBOL(set_fs_root);

/*
* Replace the fs->{pwdmnt,pwd} with {mnt,dentry}. Put the old values.

----- End forwarded message -----


2005-01-06 19:14:22

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 11:05:38AM -0800, Paul E. McKenney wrote:
> Hello, Andrew,
>
> Some export-removal work causes breakage for an out-of-tree filesystem.
> Could you please apply the attached patch to restore the exports for
> files_lock and set_fs_root?

What out of tree filesystem, and what the heck is it doing?

Without proper explanation it's vetoed.

btw, any reason you put half the world in the Cc list? Al and Andrew I
see, but do the other people on the Cc list have to do with it? And you
forgot the person that killed the export.

2005-01-06 19:17:25

by Al Viro

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 11:05:38AM -0800, Paul E. McKenney wrote:
> Hello, Andrew,
>
> Some export-removal work causes breakage for an out-of-tree filesystem.
> Could you please apply the attached patch to restore the exports for
> files_lock and set_fs_root?

What, in name of everything unholy, is *filesystem* doing with set_fs_root()?

2005-01-06 20:18:12

by Arjan van de Ven

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, 2005-01-06 at 11:05 -0800, Paul E. McKenney wrote:
> Hello, Andrew,
>
> Some export-removal work causes breakage for an out-of-tree filesystem.
> Could you please apply the attached patch to restore the exports for
> files_lock and set_fs_root?


> diff -urpN -X ../dontdiff linux-2.5/fs/namespace.c linux-2.5-MVFS/fs/namespace.c
> --- linux-2.5/fs/namespace.c Wed Jan 5 13:54:22 2005
> +++ linux-2.5-MVFS/fs/namespace.c Wed Jan 5 17:12:08 2005

isn;t clearcase (mvfs) a binary only kernel module, and isn't it so that
we don't export specifically for such (potentially license violating)
modules ?

2005-01-06 20:18:14

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 07:13:55PM +0000, Christoph Hellwig wrote:
> On Thu, Jan 06, 2005 at 11:05:38AM -0800, Paul E. McKenney wrote:
> > Hello, Andrew,
> >
> > Some export-removal work causes breakage for an out-of-tree filesystem.
> > Could you please apply the attached patch to restore the exports for
> > files_lock and set_fs_root?
>
> What out of tree filesystem, and what the heck is it doing?

MVFS, as was correctly guessed from my diff. It is providing a view into
a source-code control system, so that a given process can specify the
version it wishes to see. Yes, different processes then see a different
filesystem tree at the same mount point.

> Without proper explanation it's vetoed.

What additional explanation are you looking for?

> btw, any reason you put half the world in the Cc list? Al and Andrew I
> see, but do the other people on the Cc list have to do with it? And you
> forgot the person that killed the export.

I figured that there was no way that you would miss it, so there was no
point in adding to an already overly long CC line. ;-)

Thanx, Paul

2005-01-06 20:39:43

by Al Viro

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 12:15:31PM -0800, Paul E. McKenney wrote:
> Yep, you win the prize, it is MVFS.
>
> This is the usual port of an existing body of code to the Linux kernel.
> It is not asking for a new export, only restoration of a previously existing
> export.

Sorry, but "our code is badly misdesigned" does not make a valid excuse
when you have been told, repeatedly, by many people, for at least a year
that you needed to sanitize your design.

Request denied. And no, it never had been intended to be exported
or, indeed, used by filesystems.

2005-01-06 20:43:55

by Mike Waychison

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Christoph Hellwig wrote:
> On Thu, Jan 06, 2005 at 11:05:38AM -0800, Paul E. McKenney wrote:
>
>>Hello, Andrew,
>>
>>Some export-removal work causes breakage for an out-of-tree filesystem.
>>Could you please apply the attached patch to restore the exports for
>>files_lock and set_fs_root?
>
>
> What out of tree filesystem, and what the heck is it doing?
>
> Without proper explanation it's vetoed.
>
> btw, any reason you put half the world in the Cc list? Al and Andrew I
> see, but do the other people on the Cc list have to do with it? And you
> forgot the person that killed the export.

Well, autofsng patches (new set forthcoming) use set_fs_root/set_fs_pwd
to pivot a call_usermodehelper process into the triggering process's
namespace.

It has no need however for files_lock.

- --
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE: The opinions expressed in this email are held by me,
and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB3aEPdQs4kOxk3/MRAlvmAJ4sXcg0Cx8+00WrNEkXW4T7Ji3tKwCfVYPO
43IFyNGeDo85sqDJCprLR8I=
=Vbec
-----END PGP SIGNATURE-----

2005-01-06 20:31:31

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 08:20:58PM +0100, Arjan van de Ven wrote:
> On Thu, 2005-01-06 at 11:05 -0800, Paul E. McKenney wrote:
> > Hello, Andrew,
> >
> > Some export-removal work causes breakage for an out-of-tree filesystem.
> > Could you please apply the attached patch to restore the exports for
> > files_lock and set_fs_root?
>
>
> > diff -urpN -X ../dontdiff linux-2.5/fs/namespace.c linux-2.5-MVFS/fs/namespace.c
> > --- linux-2.5/fs/namespace.c Wed Jan 5 13:54:22 2005
> > +++ linux-2.5-MVFS/fs/namespace.c Wed Jan 5 17:12:08 2005
>
> isn;t clearcase (mvfs) a binary only kernel module, and isn't it so that
> we don't export specifically for such (potentially license violating)
> modules ?

Yep, you win the prize, it is MVFS.

This is the usual port of an existing body of code to the Linux kernel.
It is not asking for a new export, only restoration of a previously existing
export.

Thanx, Paul

2005-01-06 20:18:13

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 07:14:41PM +0000, Al Viro wrote:
> On Thu, Jan 06, 2005 at 11:05:38AM -0800, Paul E. McKenney wrote:
> > Hello, Andrew,
> >
> > Some export-removal work causes breakage for an out-of-tree filesystem.
> > Could you please apply the attached patch to restore the exports for
> > files_lock and set_fs_root?
>
> What, in name of everything unholy, is *filesystem* doing with set_fs_root()?

It is using it to set the process's view of the source-code control
system to the desired version. So process A can see version 2.1 while
process B sees version 1.5, and so that either process can change the
version that it sees on the fly.

Thanx, Paul

2005-01-06 20:18:13

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 12:07:38PM -0800, Paul E. McKenney wrote:
> > What out of tree filesystem, and what the heck is it doing?
>
> MVFS, as was correctly guessed from my diff. It is providing a view into
> a source-code control system, so that a given process can specify the
> version it wishes to see. Yes, different processes then see a different
> filesystem tree at the same mount point.

We have that in the VFS as namespace and it has no business in a filesystem
driver. And we have been telling this for more than a year.

> > Without proper explanation it's vetoed.
>
> What additional explanation are you looking for?

The explanation is so good that we can veto that patch with a reason,
as it should really be obvious to you an anyone involved.

2005-01-06 21:13:51

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

> Well, autofsng patches (new set forthcoming) use set_fs_root/set_fs_pwd
> to pivot a call_usermodehelper process into the triggering process's
> namespace.

Once we get anywhere where this is needed we'll find a better interface
for that. Like call_usermodehelper_in_namespace() or something even
better.

2005-01-06 21:13:46

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 08:32:59PM +0000, Al Viro wrote:
> On Thu, Jan 06, 2005 at 12:15:31PM -0800, Paul E. McKenney wrote:
> > Yep, you win the prize, it is MVFS.
> >
> > This is the usual port of an existing body of code to the Linux kernel.
> > It is not asking for a new export, only restoration of a previously existing
> > export.
>
> Sorry, but "our code is badly misdesigned" does not make a valid excuse
> when you have been told, repeatedly, by many people, for at least a year
> that you needed to sanitize your design.

The obvious searches did not find this for me. Any pointers so that
I can bring to the MVFS guys' attention any alternatives that might
have been recommended?

Thanx, Paul

2005-01-06 21:28:44

by Al Viro

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 01:04:08PM -0800, Paul E. McKenney wrote:
> On Thu, Jan 06, 2005 at 08:32:59PM +0000, Al Viro wrote:
> > On Thu, Jan 06, 2005 at 12:15:31PM -0800, Paul E. McKenney wrote:
> > > Yep, you win the prize, it is MVFS.
> > >
> > > This is the usual port of an existing body of code to the Linux kernel.
> > > It is not asking for a new export, only restoration of a previously existing
> > > export.
> >
> > Sorry, but "our code is badly misdesigned" does not make a valid excuse
> > when you have been told, repeatedly, by many people, for at least a year
> > that you needed to sanitize your design.
>
> The obvious searches did not find this for me. Any pointers so that
> I can bring to the MVFS guys' attention any alternatives that might
> have been recommended?

"Use recursive bindings instead of trying to take over the entire mount tree
and mirroring it within your fs code. And do that explicitly from userland".

2005-01-06 21:41:25

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 08:59:31PM +0000, Christoph Hellwig wrote:
> > Well, autofsng patches (new set forthcoming) use set_fs_root/set_fs_pwd
> > to pivot a call_usermodehelper process into the triggering process's
> > namespace.
>
> Once we get anywhere where this is needed we'll find a better interface
> for that. Like call_usermodehelper_in_namespace() or something even
> better.

Ah, a function like that would be nice to help with some issues that I
imagine the virtual-linux (or whatever that patch is called) people will
be having with udev and hotplug. But as they haven't complained yet, I
haven't been very moved to look into it...

thanks,

greg k-h

2005-01-06 23:28:43

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports


Guys, the technical arguments are all of course correct. But the fact
remains that we broke things without any notice. (Yes, perhaps someone did
say something at sometime, but the affected party didn't know, and it's our
job to make sure that they knew). (These exports were removed in October -
the IBM guys need to work out why it took so long to detect the breakage).

I think the exports should be restored. So does Linus ("Not that I like it
all that much, but I don't think we should break existing modules unless we
have a very specific reason to break just those modules.").

Which begs the question "how do we ever get rid of these things when we
have no projected date for Linux-2.8"?

I'd propose:

a) Create Documentation/feature-removal-schedule.txt which describes
things which are going away, when, why, who is involved, etc.

b) Mark things deprecated where appropriate.

c) For module exports it would be nice if we could get the module loader
to print a message "module foo.o uses deprecated symbol files_lock".
Dunno how hard that would be.

Alternatively perhaps we could arrange for the symbol to be marked
deprecated if it's used from a module:

#ifdef MODULE
#define deprecated_if_module __deprecated
#else
#define deprecated_if_module
#endif

Just something to really draw people's attention to the fact that
their code will break at a well-defined date.


2005-01-06 23:58:55

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 03:26:21PM -0800, Andrew Morton wrote:
>
> Guys, the technical arguments are all of course correct. But the fact
> remains that we broke things without any notice. (Yes, perhaps someone did
> say something at sometime, but the affected party didn't know, and it's our
> job to make sure that they knew). (These exports were removed in October -
> the IBM guys need to work out why it took so long to detect the breakage).
>
> I think the exports should be restored. So does Linus ("Not that I like it
> all that much, but I don't think we should break existing modules unless we
> have a very specific reason to break just those modules.").

No. I specificly asked the question how they're using it and they're use is

a) completely buggy
b) poking so deep in the kernel that the user falls under the GPL
derived works clause. As a copyright holder of quite a bit of fs/*.c
I certainly wouldn't give IBM a special exception to use it even if
it was exported.

These exports were only added for intermezzo during 2.4.x and with the
removal of intermezzo they go. They never were a public API, and that they
were needed at all was a managment mistake in how that code was merged.

> Which begs the question "how do we ever get rid of these things when we
> have no projected date for Linux-2.8"?

We've been removing stuff that has

(1) no intree users
(2) doesn't make sense as API

there's simply no guarantee with out of tree users. I've gotten quite a few
mails for the recent removals, and I could tell everyone a better way to
solve it. Just as we did to IBM for this problem (very highlevel as they
don't show sources), but they insist on their completely broken way of doing
it.

2005-01-07 00:04:28

by Greg KH

[permalink] [raw]
Subject: [PATCH] add feature-removal-schedule.txt documentation

On Thu, Jan 06, 2005 at 03:26:21PM -0800, Andrew Morton wrote:
> Which begs the question "how do we ever get rid of these things when we
> have no projected date for Linux-2.8"?
>
> I'd propose:
>
> a) Create Documentation/feature-removal-schedule.txt which describes
> things which are going away, when, why, who is involved, etc.

Ok, I'll bite, here's a patch that does just that. Look good?

thanks,

greg k-h

-----------

Add Documentation/feature-removal-schedule.txt as a way to notify
everyone when and what is going to be removed.

Signed-off-by: Greg Kroah-Hartman <[email protected]>

diff -Nru a/Documentation/feature-removal-schedule.txt b/Documentation/feature-removal-schedule.txt
--- /dev/null Wed Dec 31 16:00:00 196900
+++ b/Documentation/feature-removal-schedule.txt 2005-01-06 15:54:40 -08:00
@@ -0,0 +1,17 @@
+The following is a list of files and features that are going to be
+removed in the kernel source tree. Every entry should contain what
+exactly is going away, why it is happening, and who is going to be doing
+the work. When the feature is removed from the kernel, it should also
+be removed from this file.
+
+---------------------------
+
+What: devfs
+When: July 2005
+Files: fs/devfs/*, include/linux/devfs_fs*.h and assorted devfs
+ function calls throughout the kernel tree
+Why: It has been unmaintained for a number of years, has unfixable
+ races, contains a naming policy within the kernel that is
+ against the LSB, and can be replaced by using udev.
+Who: Greg Kroah-Hartman <[email protected]>
+

2005-01-07 00:24:48

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation

On Thu, Jan 06, 2005 at 03:56:34PM -0800, Greg KH wrote:
> On Thu, Jan 06, 2005 at 03:26:21PM -0800, Andrew Morton wrote:
> > Which begs the question "how do we ever get rid of these things when we
> > have no projected date for Linux-2.8"?
> >
> > I'd propose:
> >
> > a) Create Documentation/feature-removal-schedule.txt which describes
> > things which are going away, when, why, who is involved, etc.
>
> Ok, I'll bite, here's a patch that does just that. Look good?

another item:


What: unused exports that don't make sense as general APIs
When: as soon as noticed
Files: all
Why: Unused functions bloat the kernel and wrong exported functions
will make external driver authors write code that's buggy and
unmaintainable.
Who: Christoph Hellwig <[email protected]> & others.

2005-01-07 00:29:50

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

Christoph Hellwig <[email protected]> wrote:
>
> > I think the exports should be restored. So does Linus ("Not that I like it
> > all that much, but I don't think we should break existing modules unless we
> > have a very specific reason to break just those modules.").
>
> No. I specificly asked the question how they're using it and they're use is
>
> a) completely buggy
> b) poking so deep in the kernel that the user falls under the GPL
> derived works clause. As a copyright holder of quite a bit of fs/*.c
> I certainly wouldn't give IBM a special exception to use it even if
> it was exported.
>
> These exports were only added for intermezzo during 2.4.x and with the
> removal of intermezzo they go. They never were a public API, and that they
> were needed at all was a managment mistake in how that code was merged.

Fine. Completely agree. Sometimes people do need to be forced to make
such changes - I don't think anyone would disagree with that.

What's under discussion here is "how to do it". Do we just remove things
when we notice them, or do we give (say) 12 months notice?

2005-01-07 00:34:33

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation

On Fri, Jan 07, 2005 at 12:23:35AM +0000, Christoph Hellwig wrote:
> On Thu, Jan 06, 2005 at 03:56:34PM -0800, Greg KH wrote:
> > On Thu, Jan 06, 2005 at 03:26:21PM -0800, Andrew Morton wrote:
> > > Which begs the question "how do we ever get rid of these things when we
> > > have no projected date for Linux-2.8"?
> > >
> > > I'd propose:
> > >
> > > a) Create Documentation/feature-removal-schedule.txt which describes
> > > things which are going away, when, why, who is involved, etc.
> >
> > Ok, I'll bite, here's a patch that does just that. Look good?
>
> another item:
>
>
> What: unused exports that don't make sense as general APIs
> When: as soon as noticed
> Files: all
> Why: Unused functions bloat the kernel and wrong exported functions
> will make external driver authors write code that's buggy and
> unmaintainable.
> Who: Christoph Hellwig <[email protected]> & others.

Heh, yeah right, might as well just add the following also if we are in
the CYA mode:

What: Broken function parameters and entire functions
When: As soon as noticed
Files: All
Why: See Documentation/stable_api_nonesense.txt for why this is
necessary.
Who: Greg Kroah-Hartman <[email protected]> and others.


But I think this file is for specific things, the above, and your entry,
shouldn't be needed. Well, I would hope so, but odds are it might be
necessary just to keep from answering the same questions all the time...

thanks,

greg k-h

2005-01-07 00:29:49

by Linus Torvalds

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports



On Thu, 6 Jan 2005, Alan Cox wrote:

> On Iau, 2005-01-06 at 23:26, Andrew Morton wrote:
> > I think the exports should be restored. So does Linus ("Not that I like it
> > all that much, but I don't think we should break existing modules unless we
> > have a very specific reason to break just those modules.").
>
> What happens when the feature is just not (ab)usable in the way proposed ?

At that point there is a specific _reason_ to break it, aka "that function
simply doesn't exist any more".

I'm known for happily breaking binary modules, but I think we should do it
only if we have a reason _other_ than "let's break a module".

Linus

2005-01-07 01:01:32

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 04:24:58PM -0800, Linus Torvalds wrote:
> At that point there is a specific _reason_ to break it, aka "that function
> simply doesn't exist any more".
>
> I'm known for happily breaking binary modules, but I think we should do it
> only if we have a reason _other_ than "let's break a module".

Note that the function that started this thread weren't removed with nwfs
in mind - in fact I doubt anyone here has seen the module or it's
requirements.

Arjan and I have produced lists of unused exported symbols and went through
the list what made sense and what not, and these exports were up high on the
doesn't make sense and any use we could imagine would be harmfull list.

The explained use only seconds that opinion - it has caused constant trouble
for years where IBM has been pushing the big distros to relax VFS sanity
checks and add additioal exports for their totally broken kernel driver.

That no beeing enough I know of at least two propritary Unix variants where
it cause exactly the same trouble.

2005-01-07 01:06:09

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 09:24:17PM +0000, Al Viro wrote:
> On Thu, Jan 06, 2005 at 01:04:08PM -0800, Paul E. McKenney wrote:
> > On Thu, Jan 06, 2005 at 08:32:59PM +0000, Al Viro wrote:
> > > On Thu, Jan 06, 2005 at 12:15:31PM -0800, Paul E. McKenney wrote:
> > > > Yep, you win the prize, it is MVFS.
> > > >
> > > > This is the usual port of an existing body of code to the Linux kernel.
> > > > It is not asking for a new export, only restoration of a previously existing
> > > > export.
> > >
> > > Sorry, but "our code is badly misdesigned" does not make a valid excuse
> > > when you have been told, repeatedly, by many people, for at least a year
> > > that you needed to sanitize your design.
> >
> > The obvious searches did not find this for me. Any pointers so that
> > I can bring to the MVFS guys' attention any alternatives that might
> > have been recommended?
>
> "Use recursive bindings instead of trying to take over the entire mount tree
> and mirroring it within your fs code. And do that explicitly from userland".

Thank you for the pointer! By this, you mean do mount operations in
conjunction with namespaces, right?

I will follow up with more detail as I learn more. The current issue
seems to be with removeable devices. Their users want to be accessing
a particular version, but still see a memory stick that was subsequently
mounted outside of the view. Straightforward use of mounts and namespaces
would prevent the memory stick from being visible to users that were
already in view.

Thanx, Paul

2005-01-07 01:21:30

by Al Viro

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 05:01:19PM -0800, Paul E. McKenney wrote:
> Thank you for the pointer! By this, you mean do mount operations in
> conjunction with namespaces, right?

Namespaces or chroots, not much difference... The point is that you want
to duplicate the mount tree with your stuff added at some point. Instead
of doing duplication on directory tree level (i.e. having your fs code
try and mirror the stuff from other filesystems), you can do that completely
outside of fs code; either by cloning the namespace first or by building
a chroot jail with mount --rbind / <location of jail> and chrooting in
there. Then just mount the stuff that really comes from your data at
whatever place you want.

> I will follow up with more detail as I learn more. The current issue
> seems to be with removeable devices. Their users want to be accessing
> a particular version, but still see a memory stick that was subsequently
> mounted outside of the view. Straightforward use of mounts and namespaces
> would prevent the memory stick from being visible to users that were
> already in view.

There is a way to deal with that and since 2.7 is not going to materialize,
we'd better go and resurrect that project in 2.6... Basically, having
shared and asymmetrically shared subtrees between namespaces/locations
in the same namespace.

I'll try to get the detailed description of that stuff (partial sharing)
written down in a couple of days and post an RFC on l-k and fsdevel.

It's doable, it's not particulary scary, but it sure as hell *far* easier
to do in generic code; there we have access to vfsmount trees and there
lives all code that modifies them, so we don't have to screw with mirroring
directory trees of other filesystems.

Note that even now you can simply go ahead and mount that stick inside
of view explicitly. That will work; the question is how to make it automatic
and do that in a sane way.

2005-01-07 00:48:08

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 04:29:28PM -0800, Andrew Morton wrote:
> Fine. Completely agree. Sometimes people do need to be forced to make
> such changes - I don't think anyone would disagree with that.
>
> What's under discussion here is "how to do it". Do we just remove things
> when we notice them, or do we give (say) 12 months notice?

Remove when we notice with a short (measured in weeks) period where that
removal happens only in -mm. It's a price people have to pay for not
submitting their code upstream.

2005-01-07 02:07:00

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 03:26:21PM -0800, Andrew Morton wrote:
>
> Guys, the technical arguments are all of course correct. But the fact
> remains that we broke things without any notice. (Yes, perhaps someone did
> say something at sometime, but the affected party didn't know, and it's our
> job to make sure that they knew). (These exports were removed in October -
> the IBM guys need to work out why it took so long to detect the breakage).

Indeed!

We are getting lists of symbols used by the various products, as well
as working to get them tested more frequently against kernel.org trees
instead of only against distro releases. IBM being what it is, -finding-
all such products is a challenge, but so it goes.

Thanx, Paul

2005-01-07 02:16:23

by Alan

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Iau, 2005-01-06 at 23:26, Andrew Morton wrote:
> I think the exports should be restored. So does Linus ("Not that I like it
> all that much, but I don't think we should break existing modules unless we
> have a very specific reason to break just those modules.").

What happens when the feature is just not (ab)usable in the way proposed
?


2005-01-07 02:40:22

by Alan

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Gwe, 2005-01-07 at 00:29, Andrew Morton wrote:
> Christoph Hellwig <[email protected]> wrote:
> Fine. Completely agree. Sometimes people do need to be forced to make
> such changes - I don't think anyone would disagree with that.
>
> What's under discussion here is "how to do it". Do we just remove things
> when we notice them, or do we give (say) 12 months notice?

Why should we keep junk around for 12 months that nobody has a legal
reason to be using ? We broke every out of serial tty driver in 2.6.9
and in 2.6.10 for example and we will break them all again in ways we
can't keep the old stuff around. (and Linus broke them all in 2.6.10 8))

There is a difference between a public API like inter_module_get() and
an internal deep reference to something that is fairly private. Zapping
those kind of functions is very different and I agree we should
deprecate them properly.

A lot of this is coming up because old exports for private use were
never really properly marked as _GPL or in other ways so we have a
legacy of assumptions to tidy.

Alan

2005-01-07 03:18:04

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

Alan Cox <[email protected]> wrote:
>
> On Gwe, 2005-01-07 at 00:29, Andrew Morton wrote:
> > Christoph Hellwig <[email protected]> wrote:
> > Fine. Completely agree. Sometimes people do need to be forced to make
> > such changes - I don't think anyone would disagree with that.
> >
> > What's under discussion here is "how to do it". Do we just remove things
> > when we notice them, or do we give (say) 12 months notice?
>
> Why should we keep junk around for 12 months

To give people a reasonable amount of time to stop using these things, of
course.

> that nobody has a legal reason to be using ?

The symbols were exported to non-gpl modules. People used them. Maybe
they shouldn't have. Maybe they were asked not to do so, and maybe or
maybe not they noticed. Certainly we shouldn't have exported these things
in the first place.

We should find a way of repairing things while minimising the amount of
screwing people around.

2005-01-07 03:30:53

by Mike Waychison

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Christoph Hellwig wrote:
> On Thu, Jan 06, 2005 at 04:29:28PM -0800, Andrew Morton wrote:
>
>>Fine. Completely agree. Sometimes people do need to be forced to make
>>such changes - I don't think anyone would disagree with that.
>>
>>What's under discussion here is "how to do it". Do we just remove things
>>when we notice them, or do we give (say) 12 months notice?
>
>
> Remove when we notice with a short (measured in weeks) period where that
> removal happens only in -mm. It's a price people have to pay for not
> submitting their code upstream.

Not everyone has cycles to follow to -mm.

I'd much rather see deprecation warnings in mainline releases for at
least one if not two releases.

- --
Mike Waychison
Sun Microsystems, Inc.
1 (650) 352-5299 voice
1 (416) 202-8336 voice

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
NOTICE: The opinions expressed in this email are held by me,
and may not represent the views of Sun Microsystems, Inc.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB3gJTdQs4kOxk3/MRAq9uAKCRHF7aF/vviLfIQl3fvv4eZSYpCACgh82/
5aTd4a6BqeGISYPZDDUvhSg=
=mhET
-----END PGP SIGNATURE-----

2005-01-07 07:33:48

by Arjan van de Ven

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, 2005-01-06 at 13:04 -0800, Paul E. McKenney wrote:
> On Thu, Jan 06, 2005 at 08:32:59PM +0000, Al Viro wrote:
> > On Thu, Jan 06, 2005 at 12:15:31PM -0800, Paul E. McKenney wrote:
> > > Yep, you win the prize, it is MVFS.
> > >
> > > This is the usual port of an existing body of code to the Linux kernel.
> > > It is not asking for a new export, only restoration of a previously existing
> > > export.
> >
> > Sorry, but "our code is badly misdesigned" does not make a valid excuse
> > when you have been told, repeatedly, by many people, for at least a year
> > that you needed to sanitize your design.
>
> The obvious searches did not find this for me. Any pointers so that
> I can bring to the MVFS guys' attention any alternatives that might
> have been recommended?

eh maybe a weird question, but why are you and not the MVFS guys asking
for this export then? They can probably better explain why they need
it ....

2005-01-07 07:39:31

by Arjan van de Ven

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, 2005-01-06 at 16:24 -0800, Linus Torvalds wrote:
>
> On Thu, 6 Jan 2005, Alan Cox wrote:
>
> > On Iau, 2005-01-06 at 23:26, Andrew Morton wrote:
> > > I think the exports should be restored. So does Linus ("Not that I like it
> > > all that much, but I don't think we should break existing modules unless we
> > > have a very specific reason to break just those modules.").
> >
> > What happens when the feature is just not (ab)usable in the way proposed ?
>
> At that point there is a specific _reason_ to break it, aka "that function
> simply doesn't exist any more".

how about "it can be static, and static functions without external
references can be better optimized by gcc (and are, as of version 3.4)"
(where function pointers are seen as external references, and
EXPORT_SYMBOL does just that)


2005-01-07 08:12:33

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 07:17:35PM -0800, Andrew Morton wrote:
> The symbols were exported to non-gpl modules. People used them. Maybe
> they shouldn't have. Maybe they were asked not to do so, and maybe or
> maybe not they noticed. Certainly we shouldn't have exported these things
> in the first place.

just because it was exported it doesn't mean it ever was legal.

2005-01-07 08:15:44

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Fri, Jan 07, 2005 at 08:33:32AM +0100, Arjan van de Ven wrote:
> eh maybe a weird question, but why are you and not the MVFS guys asking
> for this export then? They can probably better explain why they need
> it ....

Because an person known to the communicity can justify IBM's illegal and
copyright-infringe than someone totally unkown? Paul did the same already
for GPFS and the unmapping_range stuff, and Greg told me has was asked
similar things repeatedly but refused.

It really seems like IBM wants to abuse the positition the it's employees
archived contribution to opesource projects.

2005-01-07 09:00:53

by Ingo Molnar

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports


* Christoph Hellwig <[email protected]> wrote:

> > What's under discussion here is "how to do it". Do we just remove things
> > when we notice them, or do we give (say) 12 months notice?
>
> Remove when we notice with a short (measured in weeks) period where
> that removal happens only in -mm. It's a price people have to pay for
> not submitting their code upstream.

let me chime in as the author/maintainer of Tux, which is an out-of-tree
patch that relies on VFS internals. VFS changes constantly break Tux in
one way or another, but i've not complained once because:

_I simply have no right to complain_

either i submit the code and then i can expect it to be taken into
account, or i dont. It's _the_ basic quid pro quo that keeps technology
moving towards mainline Linux. So if i have to fix up Tux (both for
exports and for other internal details), then that's the price. Often i
have to change Tux to do things differently - and in most cases it's Tux
that was doing things incorrectly. And note that Tux is not a
binary-only module, it's a fully GPL patch.

does this cause more work for me? Sure. Would i prefer to have less
work? Yes, but not in such a shortsighted way. Tux staying external is a
choice i made and i also care about Linux and i very much like the way
the VFS is evolving.

so my strong position is that even asking for any 'warning period' for
changes in VFS internals (including exports/unexports) would be
extremely rude. It would be rude not only towards the authors and
maintainers of mainline VFS code, but also towards other external
trees/drivers who do _not_ ask for any special status and accept the
deal: "follow internals, notice kernel people if they do bad stuff
(extremely rare in my case) and fix/redesign stuff if the external tree
is broken (much more common)".

Ingo

2005-01-07 09:17:20

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Fri, Jan 07, 2005 at 10:00:14AM +0100, Ingo Molnar wrote:
> so my strong position is that even asking for any 'warning period' for
> changes in VFS internals (including exports/unexports) would be
> extremely rude. It would be rude not only towards the authors and
> maintainers of mainline VFS code, but also towards other external
> trees/drivers who do _not_ ask for any special status and accept the
> deal: "follow internals, notice kernel people if they do bad stuff
> (extremely rare in my case) and fix/redesign stuff if the external tree
> is broken (much more common)".

<sarcasm>
<osdl-salespitch>
Unfortunately you don't have the financial and political powers IBM
has, so your opinion doesn't matter as much. Maybe you should become
OSDL member to influence the direction of Linux development.
</osdl-salespitch>
</sarcasm>

2005-01-07 12:14:17

by Antonio Vargas

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Fri, 7 Jan 2005 09:15:42 +0000, Christoph Hellwig <[email protected]> wrote:
> On Fri, Jan 07, 2005 at 10:00:14AM +0100, Ingo Molnar wrote:
> > so my strong position is that even asking for any 'warning period' for
> > changes in VFS internals (including exports/unexports) would be
> > extremely rude. It would be rude not only towards the authors and
> > maintainers of mainline VFS code, but also towards other external
> > trees/drivers who do _not_ ask for any special status and accept the
> > deal: "follow internals, notice kernel people if they do bad stuff
> > (extremely rare in my case) and fix/redesign stuff if the external tree
> > is broken (much more common)".
>
> <sarcasm>
> <osdl-salespitch>
> Unfortunately you don't have the financial and political powers IBM
> has, so your opinion doesn't matter as much. Maybe you should become
> OSDL member to influence the direction of Linux development.
> </osdl-salespitch>
> </sarcasm>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

If a filesystem is binary-only, then it should get the same treatment
as other binary-only things like graphic drivers and
machine-virtualization drivers: either stay on a designated kernel
version or else fix your driver so it work with mainline.

--
Greetz, Antonio Vargas aka winden of network

http://wind.codepixel.com/

Las cosas no son lo que parecen, excepto cuando parecen lo que si son.

2005-01-07 15:12:39

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Fri, Jan 07, 2005 at 08:33:32AM +0100, Arjan van de Ven wrote:
> On Thu, 2005-01-06 at 13:04 -0800, Paul E. McKenney wrote:
> > On Thu, Jan 06, 2005 at 08:32:59PM +0000, Al Viro wrote:
> > > On Thu, Jan 06, 2005 at 12:15:31PM -0800, Paul E. McKenney wrote:
> > > > Yep, you win the prize, it is MVFS.
> > > >
> > > > This is the usual port of an existing body of code to the Linux kernel.
> > > > It is not asking for a new export, only restoration of a previously existing
> > > > export.
> > >
> > > Sorry, but "our code is badly misdesigned" does not make a valid excuse
> > > when you have been told, repeatedly, by many people, for at least a year
> > > that you needed to sanitize your design.
> >
> > The obvious searches did not find this for me. Any pointers so that
> > I can bring to the MVFS guys' attention any alternatives that might
> > have been recommended?
>
> eh maybe a weird question, but why are you and not the MVFS guys asking
> for this export then? They can probably better explain why they need
> it ....

As near as I can tell, they believe that they already did the best they
could to explain their needs and failed to do so.

Thanx, Paul

2005-01-07 15:24:10

by Arjan van de Ven

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports


> > eh maybe a weird question, but why are you and not the MVFS guys asking
> > for this export then? They can probably better explain why they need
> > it ....
>
> As near as I can tell, they believe that they already did the best they
> could to explain their needs and failed to do so.
>

can you try to find even ONE request for these exports from them in the
mailing list archives? Since I find that hard to believe.

2005-01-07 15:36:38

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Fri, Jan 07, 2005 at 04:23:52PM +0100, Arjan van de Ven wrote:
>
> > > eh maybe a weird question, but why are you and not the MVFS guys asking
> > > for this export then? They can probably better explain why they need
> > > it ....
> >
> > As near as I can tell, they believe that they already did the best they
> > could to explain their needs and failed to do so.
> >
>
> can you try to find even ONE request for these exports from them in the
> mailing list archives? Since I find that hard to believe.

I did not find much when I searched yesterday, which is why I had to ask
Al what advice he had given them.

Thanx, Paul

2005-01-07 15:57:04

by Arjan van de Ven

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Fri, 2005-01-07 at 07:34 -0800, Paul E. McKenney wrote:
> On Fri, Jan 07, 2005 at 04:23:52PM +0100, Arjan van de Ven wrote:
> >
> > > > eh maybe a weird question, but why are you and not the MVFS guys asking
> > > > for this export then? They can probably better explain why they need
> > > > it ....
> > >
> > > As near as I can tell, they believe that they already did the best they
> > > could to explain their needs and failed to do so.
> > >
> >
> > can you try to find even ONE request for these exports from them in the
> > mailing list archives? Since I find that hard to believe.
>
> I did not find much when I searched yesterday, which is why I had to ask
> Al what advice he had given them.

as far as I know that was mostly unrelated to the MVFS people asking for
these exports, which you seem to think they did. I think differently
now ;)


2005-01-07 17:45:01

by Randy.Dunlap

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation

Greg KH wrote:
> On Thu, Jan 06, 2005 at 03:26:21PM -0800, Andrew Morton wrote:
>
>>Which begs the question "how do we ever get rid of these things when we
>>have no projected date for Linux-2.8"?
>>
>>I'd propose:
>>
>>a) Create Documentation/feature-removal-schedule.txt which describes
>> things which are going away, when, why, who is involved, etc.
>
> Ok, I'll bite, here's a patch that does just that. Look good?

Brodo, can you add a little more info to this, please?

---
Add 2.4.x cpufreq /proc and sysctl interface removal
to the feature-removal-schedule.

Signed-off-by: Randy Dunlap <[email protected]>

diffstat:=
Documentation/feature-removal-schedule.txt | 9 +++++++++
1 files changed, 9 insertions(+)

---


Attachments:
cpufreq_sched.patch (782.00 B)

2005-01-07 17:56:03

by Linus Torvalds

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation



On Fri, 7 Jan 2005, Randy.Dunlap wrote:
>
> Brodo, can you add a little more info to this, please?

I think for something like this to be really useful, you should not just
say "can be replaced with Xxxx", but have some docs (or pointers to such)
for both users and developers (depending on who is affected) on _how_ to
replace it, or fix it.

Also, I'm not convinced about the single-file format. If we want to do
this (I don't know how much it buys, but hey, I certainly also don't have
any objections), I think it would be much nicer to have a separate
"deprecated" subdirectory, with one file per issue.

(Not that I think it necessarily needs to be just about deprecated or
removed features - again, if we do this, I don't see why it shouldn't
contain the same information about semantic changes, so that when the
locking for an interface changes, you could have a

Documentation/changes/vfs-ioctl-locking.txt

that tells what the new rules are).

Would most of the files be small? Sure hope so. But so what?

Linus

2005-01-07 18:14:20

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation

On Fri, Jan 07, 2005 at 09:54:16AM -0800, Linus Torvalds wrote:
>
>
> On Fri, 7 Jan 2005, Randy.Dunlap wrote:
> >
> > Brodo, can you add a little more info to this, please?
>
> I think for something like this to be really useful, you should not just
> say "can be replaced with Xxxx", but have some docs (or pointers to such)
> for both users and developers (depending on who is affected) on _how_ to
> replace it, or fix it.
>
> Also, I'm not convinced about the single-file format. If we want to do
> this (I don't know how much it buys, but hey, I certainly also don't have
> any objections), I think it would be much nicer to have a separate
> "deprecated" subdirectory, with one file per issue.

Ok, that's fine with me, makes it easier for patches. I'll take my
writeup and Randy's and split them out.

> (Not that I think it necessarily needs to be just about deprecated or
> removed features - again, if we do this, I don't see why it shouldn't
> contain the same information about semantic changes, so that when the
> locking for an interface changes, you could have a
>
> Documentation/changes/vfs-ioctl-locking.txt
>
> that tells what the new rules are).

Hm, but things "change" all the time. "new" rules become "old" rules
over time too. What should probably happen is the proper rules are
documented and kept up to date, like they are today in
Documentation/filesystems/Locking and Documentation/pci.txt as two
examples. Showing what has changed over time in those two files is what
diffs are for :)

Otherwise, the kernel changelogs are good places to dig for changes in
apis, but do we really want to duplicate this in other places too?

thanks,

greg k-h

2005-01-07 22:00:10

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

Christoph Hellwig <[email protected]> wrote:
>
> On Fri, Jan 07, 2005 at 10:00:14AM +0100, Ingo Molnar wrote:
> > so my strong position is that even asking for any 'warning period' for
> > changes in VFS internals (including exports/unexports) would be
> > extremely rude.

No, I'd say that unexports are different. They can clearly break existing
code and so should only be undertaken with caution, and with lengthy notice
if possible.

And it _is_ possible here, because there are no plans to change the
exported functions, and it's only two lines of code, dammit.

The cost to us of maintaining those two lines of code for a year is
basically zero.

The cost to others of us removing those two lines of code without warning
is appreciable.

Obvious solution: don't remove the two lines of code without warning.

The only reason I can see for peremptorily removing those two lines of code
is that there is some benefit to doing so which outweighs the downstream
cost of doing so. Nobody has demonstrated such a benefit.

> <sarcasm>
> <osdl-salespitch>
> Unfortunately you don't have the financial and political powers IBM
> has, so your opinion doesn't matter as much. Maybe you should become
> OSDL member to influence the direction of Linux development.
> </osdl-salespitch>
> </sarcasm>

Christoph, it would be better to constraint yourself to commenting on other
people's actions rather than entering into premature speculation regarding
their motivations, especially when that speculation involves accusations of
corruption.

I have outlined the reasons for delaying the removal of these exports. If
you can demonstrate that those reasons are invalid, and that I had good
reason for also believing that those reasons are invalid then that would be
an appropriate stage at which to start to speculate about ulterior
motivations, thanks very much.

2005-01-07 22:26:02

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Fri, Jan 07, 2005 at 02:00:34PM -0800, Andrew Morton wrote:
> No, I'd say that unexports are different. They can clearly break existing
> code and so should only be undertaken with caution, and with lengthy notice
> if possible.

As mentioned before we only unexported symbols were we were pretty clear
that all users of it are indeep utterly broken. I got about a dozend
replies for this patches, and for more than half of it where the reporter
was either the author or the module was opensource I could help them to
actually fix their code. In this case the code is far more broken than
the others, but we've even been trying to help them fix their code for
more than a year, but IBM folks have been constanly refusing.

> The cost to us of maintaining those two lines of code for a year is
> basically zero.

But as long as IBM people don't fix their %$^%! they'll continue annoying
us and the Distibutors about adding more and more hacks for it, and other
people will write similarly crappy code using it again and making the
removal even more difficult.

> > <sarcasm>
> > <osdl-salespitch>
> > Unfortunately you don't have the financial and political powers IBM
> > has, so your opinion doesn't matter as much. Maybe you should become
> > OSDL member to influence the direction of Linux development.
> > </osdl-salespitch>
> > </sarcasm>
>
> Christoph, it would be better to constraint yourself to commenting on other
> people's actions rather than entering into premature speculation regarding
> their motivations, especially when that speculation involves accusations of
> corruption.

Hey, you got messages in this thread from half a dozen kernel contributors
here and still stand against all of them and for IBM who's violating our
copyrights that don't allow non-propritary derived works from it which
any user of these deeply-internal modules clearly is.

I don't know nor care nor want to speculate over anyones motives, but the
situation is at least strange enough to deserve a little sarcastic anekdote.

2005-01-07 22:56:39

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

Christoph Hellwig <[email protected]> wrote:
>
> On Fri, Jan 07, 2005 at 02:00:34PM -0800, Andrew Morton wrote:
> > No, I'd say that unexports are different. They can clearly break existing
> > code and so should only be undertaken with caution, and with lengthy notice
> > if possible.
>
> As mentioned before we only unexported symbols were we were pretty clear
> that all users of it are indeep utterly broken. I got about a dozend
> replies for this patches, and for more than half of it where the reporter
> was either the author or the module was opensource I could help them to
> actually fix their code. In this case the code is far more broken than
> the others, but we've even been trying to help them fix their code for
> more than a year, but IBM folks have been constanly refusing.

They didn't fix their code because it worked - no reason to do so.

Telling people "this is going away in 12 months" gives them reason to fix
the code. And a reasonable amount of time to do so and to distribute the
new version.

> > The cost to us of maintaining those two lines of code for a year is
> > basically zero.
>
> But as long as IBM people don't fix their %$^%! they'll continue annoying
> us and the Distibutors about adding more and more hacks for it,

Maybe, maybe not. But is it appropriate for us to be trying to control
someone else's internal activities via alterations to the kernel?

> and other
> people will write similarly crappy code using it again and making the
> removal even more difficult.

I doubt if people would be silly enough to use a deprecated export in new
code after the export has been scheduled for removal. If they do then yes,
sorry, we have to draw the line somewhere.

You still have not demonstrated any benefit to any party from not delaying
the removal of these two exports.

2005-01-07 23:36:16

by Adrian Bunk

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Fri, Jan 07, 2005 at 02:00:34PM -0800, Andrew Morton wrote:
>
> No, I'd say that unexports are different. They can clearly break existing
> code and so should only be undertaken with caution, and with lengthy notice
> if possible.
>
> And it _is_ possible here, because there are no plans to change the
> exported functions, and it's only two lines of code, dammit.
>
> The cost to us of maintaining those two lines of code for a year is
> basically zero.
>
> The cost to others of us removing those two lines of code without warning
> is appreciable.
>
> Obvious solution: don't remove the two lines of code without warning.
>
> The only reason I can see for peremptorily removing those two lines of code
> is that there is some benefit to doing so which outweighs the downstream
> cost of doing so. Nobody has demonstrated such a benefit.
>...

I did a bit research using grep, sort, uniq and wc:

between 2.6.9 and 2.6.10:
414 EXPORT_SYMBOL's were removed

since 2.6.10:
90 EXPORT_SYMBOL's were removed in Linus' tree

in 2.6.10-mm2 excluding linus.patch:
71 EXPORT_SYMBOL's are removed

Notes:
- EXPORT_SYMBOL_GPL's weren't counted in any way
- if an EXPORT_SYMBOL was moved, it wasn't counted (using uniq)
- small mistakes in the numbers might be possible since my method to
measure them was't at a scientific level, but after a quick look
it seems the numbers are roughly correct


Resurrecting and documenting all of these recently removed
EXPORT_SYMBOL's because some company might have found some way to
(ab)use one or more of them costs us:
- some extra work
- wasts space for all users of Linux (e.g. some of the recent removals
are "remove EXPORT_SYMBOL'ed but completely unused function" patches
I sent)


cu
Adrian

--

"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed

2005-01-08 00:06:20

by Dominik Brodowski

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation

On Fri, Jan 07, 2005 at 09:02:55AM -0800, Randy.Dunlap wrote:
> Add 2.4.x cpufreq /proc and sysctl interface removal
> to the feature-removal-schedule.

As it is already removed in Davej's cpufreq bitkeeper tree, I don't know if
it's still desired or not, but:

> +What: /proc/sys/cpu and the sysctl interface to cpufreq (2.4.x interfaces)
> +When: January 2005
> +Files: drivers/cpufreq/: cpufreq_userspace.c, proc_intf.c
> + function calls throughout the kernel tree
> +Why: Deprecated, has been replaced/superseded by (what?)....
> +Who: Dominik Brodowski <[email protected]>

What: /proc/sys/cpu/*, sysctl and /proc/cpufreq interfaces to cpufreq (2.4.x interfaces)
When: January 2005
Files: drivers/cpufreq/: cpufreq_userspace.c, proc_intf.c
Why: /proc/sys/cpu/* has been deprecated since inclusion of cpufreq into the
main kernel tree. It bloats /proc/ unnecessarily and doesn't work
well with the "governor"-based design of cpufreq.
/proc/cpufreq/* has also been deprecated for a long time and was only
meant for usage during 2.5. until the new sysfs-based interface became
ready. It has an inconsistent interface which doesn't work well with
userspace setting the frequency. The output from /proc/cpufreq/* can
be emulated using "cpufreq-info --proc" (cpufrequtils).
Both interfaces are superseded by the cpufreq interface in
/sys/devices/system/cpu/cpu%n/cpufreq/.
Who: Dominik Brodowski <[email protected]>

2005-01-07 23:57:56

by Alan

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Gwe, 2005-01-07 at 22:00, Andrew Morton wrote:
> No, I'd say that unexports are different. They can clearly break existing
> code and so should only be undertaken with caution, and with lengthy notice
> if possible.

That was done

> Christoph, it would be better to constraint yourself to commenting on other
> people's actions rather than entering into premature speculation regarding
> their motivations, especially when that speculation involves accusations of
> corruption.

People have been trying to get this stuff fixed for a long time. There
are no sane users of it, there are no apparent legal users of it and the
one remaining problem has shown no sign of wishing to change and will no
doubt try the same again in twelve months.

2005-01-08 00:11:51

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

Alan Cox <[email protected]> wrote:
>
> On Gwe, 2005-01-07 at 22:00, Andrew Morton wrote:
> > No, I'd say that unexports are different. They can clearly break existing
> > code and so should only be undertaken with caution, and with lengthy notice
> > if possible.
>
> That was done

Obviously not very well, because it didn't work. Was a removal date given?

> > Christoph, it would be better to constraint yourself to commenting on other
> > people's actions rather than entering into premature speculation regarding
> > their motivations, especially when that speculation involves accusations of
> > corruption.
>
> People have been trying to get this stuff fixed for a long time. There
> are no sane users of it, there are no apparent legal users of it and the
> one remaining problem has shown no sign of wishing to change and will no
> doubt try the same again in twelve months.

I doubt if that is correct. Paul?

2005-01-08 02:21:02

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Fri, Jan 07, 2005 at 04:12:25PM -0800, Andrew Morton wrote:
> Alan Cox <[email protected]> wrote:
> >
> > People have been trying to get this stuff fixed for a long time. There
> > are no sane users of it, there are no apparent legal users of it and the
> > one remaining problem has shown no sign of wishing to change and will no
> > doubt try the same again in twelve months.
>
> I doubt if that is correct. Paul?

If MVFS tries the same again in twelve months, they will be going through
some representative other than myself. In fact, if MVFS does not have
a credible plan for accommodating a twelve-months-from-now loss of these
exports within 30 days, their management will be hearing from me.

Thanx, Paul

2005-01-08 13:11:01

by Al Viro

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Sat, Jan 08, 2005 at 12:32:47AM +0100, Adrian Bunk wrote:
> - wasts space for all users of Linux (e.g. some of the recent removals
> are "remove EXPORT_SYMBOL'ed but completely unused function" patches
> I sent)

Note that this alone is *NOT* enough for removal - there are groups of
functions that basically are libraries and that are supposed to be
exported, whether we have all of them used in the tree at given time
or not.

2005-01-08 16:49:54

by Alan

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Gwe, 2005-01-07 at 22:58, Andrew Morton wrote:
> You still have not demonstrated any benefit to any party from not delaying
> the removal of these two exports.

Trivial one: it saves memory 8)

2005-01-08 18:32:49

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation

On Thu, Jan 06, 2005 at 03:56:34PM -0800, Greg KH wrote:
> On Thu, Jan 06, 2005 at 03:26:21PM -0800, Andrew Morton wrote:
> > Which begs the question "how do we ever get rid of these things when we
> > have no projected date for Linux-2.8"?
> >
> > I'd propose:
> >
> > a) Create Documentation/feature-removal-schedule.txt which describes
> > things which are going away, when, why, who is involved, etc.
>
> Ok, I'll bite, here's a patch that does just that. Look good?
>
> thanks,
>
> greg k-h
>
> -----------
>
> Add Documentation/feature-removal-schedule.txt as a way to notify
> everyone when and what is going to be removed.
>
> Signed-off-by: Greg Kroah-Hartman <[email protected]>
>
> diff -Nru a/Documentation/feature-removal-schedule.txt b/Documentation/feature-removal-schedule.txt
> --- /dev/null Wed Dec 31 16:00:00 196900
> +++ b/Documentation/feature-removal-schedule.txt 2005-01-06 15:54:40 -08:00
> @@ -0,0 +1,17 @@
> +The following is a list of files and features that are going to be
> +removed in the kernel source tree. Every entry should contain what
> +exactly is going away, why it is happening, and who is going to be doing
> +the work. When the feature is removed from the kernel, it should also
> +be removed from this file.
> +
> +---------------------------
> +
> +What: devfs
> +When: July 2005
> +Files: fs/devfs/*, include/linux/devfs_fs*.h and assorted devfs
> + function calls throughout the kernel tree
> +Why: It has been unmaintained for a number of years, has unfixable
> + races, contains a naming policy within the kernel that is
> + against the LSB, and can be replaced by using udev.
> +Who: Greg Kroah-Hartman <[email protected]>
> +

And another. I would also like to flag the exports themselves as
indicated in the patch below. Thoughts?

Thanx, Paul

What: call_rcu(), call_rcu_bh(), and synchronize_kernel() change from
EXPORT_SYMBOL() to EXPORT_SYMBOL_GPL().
When: January 9, 2006
Files: kernel/rcupdate.c
Why: There are no known environments supporting RCU from which
one could reasonably expect to port a non-GPL kernel module
or driver to Linux.
Who: Paul E. McKenney <[email protected]>


diff -urpN -X ../dontdiff linux-2.5/kernel/rcupdate.c linux-2.5-rcu-export-warn/kernel/rcupdate.c
--- linux-2.5/kernel/rcupdate.c Sat Jan 8 09:25:55 2005
+++ linux-2.5-rcu-export-warn/kernel/rcupdate.c Sat Jan 8 10:21:18 2005
@@ -465,6 +465,8 @@ void synchronize_kernel(void)
}

module_param(maxbatch, int, 0);
+
+/* WARNING: these will become EXPORT_SYMBOL_GPL() in January 2006. */
EXPORT_SYMBOL(call_rcu);
EXPORT_SYMBOL(call_rcu_bh);
EXPORT_SYMBOL(synchronize_kernel);

2005-01-08 22:51:33

by Alan

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation

On Sad, 2005-01-08 at 18:32, Paul E. McKenney wrote:
> What: call_rcu(), call_rcu_bh(), and synchronize_kernel() change from
> EXPORT_SYMBOL() to EXPORT_SYMBOL_GPL().
> When: January 9, 2006
> Files: kernel/rcupdate.c
> Why: There are no known environments supporting RCU from which
> one could reasonably expect to port a non-GPL kernel module
> or driver to Linux.

IBM might want to also note that anyone wanting to do so needs an IBM
patent license for non GPL use ..

2005-01-08 23:04:22

by Arjan van de Ven

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation

On Sat, 2005-01-08 at 21:46 +0000, Alan Cox wrote:
> On Sad, 2005-01-08 at 18:32, Paul E. McKenney wrote:
> > What: call_rcu(), call_rcu_bh(), and synchronize_kernel() change from
> > EXPORT_SYMBOL() to EXPORT_SYMBOL_GPL().
> > When: January 9, 2006
> > Files: kernel/rcupdate.c
> > Why: There are no known environments supporting RCU from which
> > one could reasonably expect to port a non-GPL kernel module
> > or driver to Linux.
>
> IBM might want to also note that anyone wanting to do so needs an IBM
> patent license for non GPL use ..

given this, I actually think it might be better to make it a _GPL export
right away, anything else is setting people up in an entrapment kind of
way.


2005-01-09 06:23:41

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation

On Sun, Jan 09, 2005 at 12:03:45AM +0100, Arjan van de Ven wrote:
> On Sat, 2005-01-08 at 21:46 +0000, Alan Cox wrote:
> > On Sad, 2005-01-08 at 18:32, Paul E. McKenney wrote:
> > > What: call_rcu(), call_rcu_bh(), and synchronize_kernel() change from
> > > EXPORT_SYMBOL() to EXPORT_SYMBOL_GPL().
> > > When: January 9, 2006
> > > Files: kernel/rcupdate.c
> > > Why: There are no known environments supporting RCU from which
> > > one could reasonably expect to port a non-GPL kernel module
> > > or driver to Linux.
> >
> > IBM might want to also note that anyone wanting to do so needs an IBM
> > patent license for non GPL use ..
>
> given this, I actually think it might be better to make it a _GPL export
> right away, anything else is setting people up in an entrapment kind of
> way.

There could be no entrapment unless IBM chose to go after people who used
RCU from a binary module.

Thanx, Paul

2005-01-09 06:27:48

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation

On Sat, Jan 08, 2005 at 09:46:34PM +0000, Alan Cox wrote:
> On Sad, 2005-01-08 at 18:32, Paul E. McKenney wrote:
> > What: call_rcu(), call_rcu_bh(), and synchronize_kernel() change from
> > EXPORT_SYMBOL() to EXPORT_SYMBOL_GPL().
> > When: January 9, 2006
> > Files: kernel/rcupdate.c
> > Why: There are no known environments supporting RCU from which
> > one could reasonably expect to port a non-GPL kernel module
> > or driver to Linux.
>
> IBM might want to also note that anyone wanting to do so needs an IBM
> patent license for non GPL use ..

Last time I checked with IBM's lawyers, they were not interested in doing
so. Nonetheless, you are correct, non-GPL use of RCU would require a
conversation with IBM. And in fact the five relevant patents (one lapsed)
are called out in Documentation/RCU/RTFP.txt. The four non-lapsed patents
are called out as "contributed under GPL", so I think that we are covered.

Thanx, Paul

2005-01-11 12:24:04

by Dominik Brodowski

[permalink] [raw]
Subject: [PATCH] cpufreq 2.4 interface removal schedule [Was: Re: [PATCH] add feature-removal-schedule.txt documentation]

Even though these 2.4. interfaces are already gone in Dave Jones' cpufreq
bitkeeper tree, here's a patch which properly announces it in
Documentation/feature-removal-schedule.txt:


Add meaningful content concerning the removal of deprecated interfaces to
the cpufreq core.

Signed-off-by: Dominik Brodowski <[email protected]>
---

Documentation/feature-removal-schedule.txt | 14 +++++++++++---
1 files changed, 11 insertions(+), 3 deletions(-)

Index: 2.6.10/Documentation/feature-removal-schedule.txt
===================================================================
--- 2.6.10.orig/Documentation/feature-removal-schedule.txt 2005-01-11 13:09:50.000000000 +0100
+++ 2.6.10/Documentation/feature-removal-schedule.txt 2005-01-11 13:15:12.000000000 +0100
@@ -17,10 +17,18 @@

---------------------------

-What: /proc/sys/cpu and the sysctl interface to cpufreq (2.4.x interfaces)
+What: /proc/sys/cpu/*, sysctl and /proc/cpufreq interfaces to cpufreq (2.4.x interfaces)
When: January 2005
Files: drivers/cpufreq/: cpufreq_userspace.c, proc_intf.c
- function calls throughout the kernel tree
-Why: Deprecated, has been replaced/superseded by (what?)....
+Why: /proc/sys/cpu/* has been deprecated since inclusion of cpufreq into
+ the main kernel tree. It bloats /proc/ unnecessarily and doesn't work
+ well with the "governor"-based design of cpufreq.
+ /proc/cpufreq/* has also been deprecated for a long time and was only
+ meant for usage during 2.5. until the new sysfs-based interface became
+ ready. It has an inconsistent interface which doesn't work well with
+ userspace setting the frequency. The output from /proc/cpufreq/* can
+ be emulated using "cpufreq-info --proc" (cpufrequtils).
+ Both interfaces are superseded by the cpufreq interface in
+ /sys/devices/system/cpu/cpu%n/cpufreq/.
Who: Dominik Brodowski <[email protected]>

2005-01-12 18:56:30

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH] add feature-removal-schedule.txt documentation

On Fri, Jan 07, 2005 at 09:02:55AM -0800, Randy.Dunlap wrote:
> Greg KH wrote:
> >On Thu, Jan 06, 2005 at 03:26:21PM -0800, Andrew Morton wrote:
> >
> >>Which begs the question "how do we ever get rid of these things when we
> >>have no projected date for Linux-2.8"?
> >>
> >>I'd propose:
> >>
> >>a) Create Documentation/feature-removal-schedule.txt which describes
> >> things which are going away, when, why, who is involved, etc.
> >
> >Ok, I'll bite, here's a patch that does just that. Look good?
>
> Brodo, can you add a little more info to this, please?
>
> ---
> Add 2.4.x cpufreq /proc and sysctl interface removal
> to the feature-removal-schedule.
>
> Signed-off-by: Randy Dunlap <[email protected]>

Applied, thanks.

greg k-h

2005-01-12 19:53:01

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH] cpufreq 2.4 interface removal schedule [Was: Re: [PATCH] add feature-removal-schedule.txt documentation]

On Tue, Jan 11, 2005 at 01:23:09PM +0100, Dominik Brodowski wrote:
> Even though these 2.4. interfaces are already gone in Dave Jones' cpufreq
> bitkeeper tree, here's a patch which properly announces it in
> Documentation/feature-removal-schedule.txt:
>
>
> Add meaningful content concerning the removal of deprecated interfaces to
> the cpufreq core.
>
> Signed-off-by: Dominik Brodowski <[email protected]>

Applied, thanks.

greg k-h

2005-01-13 02:52:48

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 06, 2005 at 05:01:19PM -0800, Paul E. McKenney wrote:
> On Thu, Jan 06, 2005 at 09:24:17PM +0000, Al Viro wrote:
> > "Use recursive bindings instead of trying to take over the entire mount tree
> > and mirroring it within your fs code. And do that explicitly from userland".
>
> Thank you for the pointer! By this, you mean do mount operations in
> conjunction with namespaces, right?
>
> I will follow up with more detail as I learn more. The current issue
> seems to be with removeable devices. Their users want to be accessing
> a particular version, but still see a memory stick that was subsequently
> mounted outside of the view. Straightforward use of mounts and namespaces
> would prevent the memory stick from being visible to users that were
> already in view.

OK, after much thrashing, here is what the ClearCase users need.
Sorry for the length -- the first few paragraphs gives the flavor of
it and the rest goes into more detail with examples.

Thoughts?

Thanx, Paul

------------------------------------------------------------------------

ClearCase provides a filesystem view of revision-control repositories.
ClearCase users can specify the desired revision directly in the pathname
("view extended naming", for example "/views/v1.1/vob/foo/bar.c")
or they can associate a particular revision with a process and its
descendants ("setview context naming", for example "/vob/foo/bar.c").
Specifying the revision in the pathname is useful for diffing and such,
and associating a revision with a process is useful for builds and
testing using standard tools, sort of like chroot jails.

ClearCase users need the root filesystem to be overlaid on each view,
so that "/etc/passwd" and "/views/v1.1/etc/passwd" reference the same
file.

The current hope is that adding (a) shared and asymmetrically shared
subtrees between namespaces/locations in the same namespace, (b) stackable
LSM modules, and (c) dynamic recursive union mount would enable Linux
to provide this in a technically sound manner. [But this is not clear
to me yet.]

More details on what ClearCase users want to see follow.

1. "View extended naming", where the revision is specified by
the pathname.

a. Users explicitly specify that they want to see a specific
revision (e.g., v1.1) using the ClearCase "startview"
command. This revision is then visible to all processes.

b. The pathname fully specifies the revision and file, e.g.,
"/views/v1.1/vob/foo/bar.c", where the "startview"
command has initialized an MVFS view of the v1.1 version on
"/views/v1.1/vob/foo". ("vob" stands for "versioned object
base".)

c. Note that the entire filesystem is also visible within
the view, so that /etc/passwd" is an synonym for
"/views/v1.1/etc/passwd". This can be thought of
as (currently mythical) "dynamic union mount rbind".
See #4 below for more detail.

2. "Setview context naming", where revision is -not- specified
by the pathname.

a. A particular process designates a particular revision
(e.g., "v1.1") using a ClearCase "setview" command.
This designation applies to both the process and its
future descendants (any descendants already created are
unaffected. The "setview" command does an implicit
"startview" if needed.

b. That process and any of its descendants will then see the
specified revision via "/vob/foo/bar.c", where an
MVFS filesystem has been mounted on "/vob/foo".

c. Changes made via the "setview context naming" paths
are visible in "view extended naming" paths, with
the same restrictions as called out in #1c above.

d. A given process can mix and match "view extended naming"
and "setview context naming" references, so that
both "/views/v1.1/vob/foo/bar.c" and "/vob/foo/bar.c"
may be used to refer to the same file, but in that case
"/views/v1.2/vob/foo/bar.c" might reference a different
file. These synonyms for the same file need to have the
properties of the mythical "dynamic union mount rbind"
described in #4 below.

3. ClearCase users are -not- permitted to mount over a file or
directory within an MVFS filesystem. Therefore, if a user
mounts a memory stick over (say) "/vob/foo/mnt", the results
are undefined.

4. ClearCase users need to be able to access multiple repositories
simultaneously (e.g., for related projects). So there might
be a "/vob/foo" and "/vob/oof" project visible simultaneously.

There are some specialized but important testing situations
that require separate top-level directories, e.g., "/vob"
and "/tmp/vobtest".

5. The ClearCase users need the mythical "dynamic union mount rbind"
to set things up so that any access to a non-MVFS file made from
within a view gives the same results as a direct access to that
file would give if no MVFS filesystems were mounted. For example,
any access to "/views/v1.1/etc/passwd" must give exactly the same
results as the same access to "/etc/passwd" would if there were
no MVFS filesystems mounted.

The "dynamic union mount rbind" therefore needs to have the
following properties:

a. The underlying inodes retain the same link count
no matter how many ClearCase views they are
accessible from. So, if the link count of /etc/passwd
is initially 1, it remains 1 even though it is
accessible via "/etc/passwd", "/views/v1.1/etc/passwd",
"/views/v1.2/etc/passwd", and so on. Similarly,
if the link count of foo/bar.c in the v1.1 revision
is initially 1, it remains 1 even when it is
accessible both via "/vob/foo/bar.c" and
"/views/v1.1/vob/foo/bar.c".

Hard-link tricks break user scripts and programs.

b. The types of the underlying files remain the same
regardless of how many views they are visible in
and regardless of how they are accessed. So, if
/etc/passwd is a normal file, it will appear to be
a normal file when accessed via "/etc/passwd",
"/views/v1.1/etc/passwd", "/views/v1.2/etc/passwd",
and so on. Similarly, if foo/bar.c is a normal file,
is will appear to be a normal file when accessed
via "/vob/foo/bar.c" and "/views/v1.1/vob/foo/bar.c".

Symbolic-link tricks break user scripts and Makefiles.

c. The types of the underlying filesystem remain the
same regardless of which view is used. This is
required by user scripts that use things like "df -k".

d. Any changes to any non-MVFS filesystem are immediately
visible in the view. Some examples:

i. A mount in the root filesystem, e.g.,

mount /dev/cdrom /mnt/cdrom

must result in the CDROM being visible in both
"/mnt/cdrom" and in "/views/v1.1/mnt/cdrom".
Ditto for NFS mounts and automounting.

In this case "immediately" means "before the
mount command's process exits". This need for
immediacy applies both to explicit mounts and
to autofs-induced mounts. Ditto for unmounts.

ii. Dynamic filesystems such as /proc must
exhibit their dynamic nature whether accessed
via "/proc", "/views/v1.1/proc", or
"/views/v1.2/proc".

Again, changes must be immediately visible in
all views, where "immediately" means that there
should be no "time warps". For example, any
monotonic counters must be seen as monotonic,
despite successive reads happening from different
views.

e. Any changes made via "view extended naming" or via
"setview context naming" are visible in the other
view. For example, the file created by

touch /vob/foo/bar.c.new

would also be immediately visible as
"/views/v1.1/vob/foo/bar.c.new" and vice versa.

f. Symbolic links from inside an MVFS filesystem act as
expected, with the expected advice that views use
relative rather than absolute symlinks for targets
within the view.

However, absolute symbolic links from inside an MVFS
filesystem to files that are outside an MVFS filesystem
work as expected.

2005-01-13 07:36:26

by Arjan van de Ven

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Wed, 2005-01-12 at 18:51 -0800, Paul E. McKenney wrote:
> On Thu, Jan 06, 2005 at 05:01:19PM -0800, Paul E. McKenney wrote:
> > On Thu, Jan 06, 2005 at 09:24:17PM +0000, Al Viro wrote:
> > > "Use recursive bindings instead of trying to take over the entire mount tree
> > > and mirroring it within your fs code. And do that explicitly from userland".
> >
> > Thank you for the pointer! By this, you mean do mount operations in
> > conjunction with namespaces, right?
> >
> > I will follow up with more detail as I learn more. The current issue
> > seems to be with removeable devices. Their users want to be accessing
> > a particular version, but still see a memory stick that was subsequently
> > mounted outside of the view. Straightforward use of mounts and namespaces
> > would prevent the memory stick from being visible to users that were
> > already in view.
>
> OK, after much thrashing, here is what the ClearCase users need.
> Sorry for the length -- the first few paragraphs gives the flavor of
> it and the rest goes into more detail with examples.
>
> Thoughts?
>

Interesting; so clearcase mvfs currently extends the linux VFS to do all
this.... It would be very interesting to get these extensions to linux
posted on this mailing list in patch form, since it could well be a nice
generic enhancement of linux. (Assuming Al doesn't go WTF over the code
of course :-)


2005-01-13 17:51:15

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 13, 2005 at 09:07:12AM -0800, Greg KH wrote:
> On Wed, Jan 12, 2005 at 06:51:57PM -0800, Paul E. McKenney wrote:
> >
> > The current hope is that adding (a) shared and asymmetrically shared
> > subtrees between namespaces/locations in the same namespace, (b) stackable
> > LSM modules, and (c) dynamic recursive union mount would enable Linux
> > to provide this in a technically sound manner. [But this is not clear
> > to me yet.]
>
> I don't see how (b) has anything to do with this. Anyone care to
> explain that?

It would allow tracking the processes that are using a given view,
so that state associated with that view could be cleaned up when the
last process exits. One case that motivates this approach:

1. one process creates a view (e.g,. "setview" so that
"/vob/foo/bar.c" references version 1.2, just as
"/views/v1.2/vob/foo/bar.c" would),

2. this process forks off several descendants, then exits, and

3. the descendant processes eventually exit.

The underlying filesystem could use stackable LSM modules to track fork()s
and exit()s, allowing it to work out when all processes using a given
view had terminated.

It is quite possible that there is a better way to do this. Thoughts?

Thanx, Paul

2005-01-13 17:56:20

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 13, 2005 at 08:35:58AM +0100, Arjan van de Ven wrote:
> On Wed, 2005-01-12 at 18:51 -0800, Paul E. McKenney wrote:
> > On Thu, Jan 06, 2005 at 05:01:19PM -0800, Paul E. McKenney wrote:
> > > On Thu, Jan 06, 2005 at 09:24:17PM +0000, Al Viro wrote:
> > > > "Use recursive bindings instead of trying to take over the entire mount tree
> > > > and mirroring it within your fs code. And do that explicitly from userland".
> > >
> > > Thank you for the pointer! By this, you mean do mount operations in
> > > conjunction with namespaces, right?
> > >
> > > I will follow up with more detail as I learn more. The current issue
> > > seems to be with removeable devices. Their users want to be accessing
> > > a particular version, but still see a memory stick that was subsequently
> > > mounted outside of the view. Straightforward use of mounts and namespaces
> > > would prevent the memory stick from being visible to users that were
> > > already in view.
> >
> > OK, after much thrashing, here is what the ClearCase users need.
> > Sorry for the length -- the first few paragraphs gives the flavor of
> > it and the rest goes into more detail with examples.
> >
> > Thoughts?
> >
>
> Interesting; so clearcase mvfs currently extends the linux VFS to do all
> this.... It would be very interesting to get these extensions to linux
> posted on this mailing list in patch form, since it could well be a nice
> generic enhancement of linux. (Assuming Al doesn't go WTF over the code
> of course :-)

Just given the interfaces it makes use of, I would guess that Al would
more than go WTF over the current code. Most would probably say that the
current code does not extend Linux VFS, but rather subverts it. Plus it
was written by people without any history of Linux-kernel development,
so one would anticipate much heartburn over style and usage issues. ;-)

All that aside, if you really really want to see it, I will see what I
can do about getting it to you.

Thanx, Paul

2005-01-13 17:59:17

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 13, 2005 at 09:44:28AM -0800, Paul E. McKenney wrote:
> On Thu, Jan 13, 2005 at 09:07:12AM -0800, Greg KH wrote:
> > On Wed, Jan 12, 2005 at 06:51:57PM -0800, Paul E. McKenney wrote:
> > >
> > > The current hope is that adding (a) shared and asymmetrically shared
> > > subtrees between namespaces/locations in the same namespace, (b) stackable
> > > LSM modules, and (c) dynamic recursive union mount would enable Linux
> > > to provide this in a technically sound manner. [But this is not clear
> > > to me yet.]
> >
> > I don't see how (b) has anything to do with this. Anyone care to
> > explain that?
>
> It would allow tracking the processes that are using a given view,
> so that state associated with that view could be cleaned up when the
> last process exits. One case that motivates this approach:
>
> 1. one process creates a view (e.g,. "setview" so that
> "/vob/foo/bar.c" references version 1.2, just as
> "/views/v1.2/vob/foo/bar.c" would),
>
> 2. this process forks off several descendants, then exits, and
>
> 3. the descendant processes eventually exit.
>
> The underlying filesystem could use stackable LSM modules to track fork()s
> and exit()s, allowing it to work out when all processes using a given
> view had terminated.

But why the "stackable" requirement? Why not just use the LSM interface
and be done with it?

Or do you want to combine this LSM with SELinux or any other LSM
(root-plug, seclvl, etc.)? If so, that's asking for trouble (see the
stackable LSM problems that have been posted multiple times here, and on
the linux-security-modules mailing list.)

thanks,

greg k-h

2005-01-13 17:48:19

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Wed, Jan 12, 2005 at 06:51:57PM -0800, Paul E. McKenney wrote:
>
> The current hope is that adding (a) shared and asymmetrically shared
> subtrees between namespaces/locations in the same namespace, (b) stackable
> LSM modules, and (c) dynamic recursive union mount would enable Linux
> to provide this in a technically sound manner. [But this is not clear
> to me yet.]

I don't see how (b) has anything to do with this. Anyone care to
explain that?

thanks,

greg k-h

2005-01-13 18:37:06

by Paul E. McKenney

[permalink] [raw]
Subject: Re: [PATCH] fs: Restore files_lock and set_fs_root exports

On Thu, Jan 13, 2005 at 09:55:20AM -0800, Greg KH wrote:
> On Thu, Jan 13, 2005 at 09:44:28AM -0800, Paul E. McKenney wrote:
> > On Thu, Jan 13, 2005 at 09:07:12AM -0800, Greg KH wrote:
> > > On Wed, Jan 12, 2005 at 06:51:57PM -0800, Paul E. McKenney wrote:
> > > >
> > > > The current hope is that adding (a) shared and asymmetrically shared
> > > > subtrees between namespaces/locations in the same namespace, (b) stackable
> > > > LSM modules, and (c) dynamic recursive union mount would enable Linux
> > > > to provide this in a technically sound manner. [But this is not clear
> > > > to me yet.]
> > >
> > > I don't see how (b) has anything to do with this. Anyone care to
> > > explain that?
> >
> > It would allow tracking the processes that are using a given view,
> > so that state associated with that view could be cleaned up when the
> > last process exits. One case that motivates this approach:
> >
> > 1. one process creates a view (e.g,. "setview" so that
> > "/vob/foo/bar.c" references version 1.2, just as
> > "/views/v1.2/vob/foo/bar.c" would),
> >
> > 2. this process forks off several descendants, then exits, and
> >
> > 3. the descendant processes eventually exit.
> >
> > The underlying filesystem could use stackable LSM modules to track fork()s
> > and exit()s, allowing it to work out when all processes using a given
> > view had terminated.
>
> But why the "stackable" requirement? Why not just use the LSM interface
> and be done with it?
>
> Or do you want to combine this LSM with SELinux or any other LSM
> (root-plug, seclvl, etc.)? If so, that's asking for trouble (see the
> stackable LSM problems that have been posted multiple times here, and on
> the linux-security-modules mailing list.)

I believe that combining with SELinux is indeed one of the motivations.
And, yes, stackable LSM modules are a bit controversial.

But any way of tracking processes retaining a reference to a view would
meet the need.


Thanx, Paul