Message-ID: <54E87190.6070209@m4x.org>
Date: Sat, 21 Feb 2015 19:52:48 +0800
From: Nicolas Iooss <nicolas.iooss_linux@m4x.org>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0
MIME-Version: 1.0
To: Al Viro <viro@ZenIV.linux.org.uk>
CC: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH] proc: get a reference to the owning module when opening
 file
References: <1423623907-6950-1-git-send-email-nicolas.iooss_linux@m4x.org> <20150217201458.GA12422@ZenIV.linux.org.uk>
In-Reply-To: <20150217201458.GA12422@ZenIV.linux.org.uk>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 7306
Lines: 135

On 02/18/2015 04:14 AM, Al Viro wrote:
> On Wed, Feb 11, 2015 at 11:05:07AM +0800, Nicolas Iooss wrote:
>> Fix this by making proc_reg_open grab a reference to the module owning
>> pde->proc_fops.
> 
> NAK.
> 
> procfs doesn't pin modules while the file is opened, by design.  And
> it's not about to start.  Background:
> 
> * ->owner protects the wrong thing - it protects _code_ in methods, but
> not the data structures.  Preventing rmmod might act as a proxy in some
> cases, but only when objects are _not_ dynamically created/removed.

All right. I've found the relevant documentation for this [1]: what you
say is already well documented and I was wrongly assuming things based
on my experience with debugfs and character devices.  This bad
assumption came partly from the fact that do_dentry_open() takes a
reference to inode->i_fop->owner [2], which is only released in
__fput().  For character devices the file_operation structure is the one
given to register_chrdev() (thanks to a call to replace_fops() [3]) and
for debugfs files this structure is directly the one given to
debugfs_create_file() [4].  As a consequence the semantics of ->owner is
implicitly extended to protect the module which did register_chrdev() or
debugfs_create_file() from being unloaded while the file is used.  If
this extension is a side-effect which must not be generalized, I agree
with your point of view.

> * what we want is to make sure that no new method calls might be issued
> after procfs removal and that procfs removal will wait for method
> calls already in progress to complete.  Again, the only way it's relevant
> to rmmod is that procfs removal is often triggered by module_exit.
> BTW, debugfs is seriously broken in that respect - it does protect the
> module, all right, but have an object removed earlier and you are fucked.

Now I understand why commit 881adb853583 [5] was needed for procfs
instead of using try_module_get()/module_put().  Thank you for having
made this clear.

For the debugfs part, I though that debugfs_remove() did something
similar to unlink() for usual filesystems so that even if the file is
used, the inode is only unlinked from debugfs and the file is not
released.  It is of course not safe to free data which could be used by
the file without the file being released first.  Nevertheless I
considered that the scopes were made so that, while the file was being
used, the module which has registered the file couldn't be removed.
This is currently true for debugfs but not for procfs (by design).

> * mmap is a big can of worms in that respect, mostly in terms of desired
> semantics.  What should be done to existing mappings when object is removed?
> Sure, no new ones should be allowed, that much is obvious.  But what should
> happen to VMAs already there and what should happen to pages covered by
> those?  IMO the effect of truncate() on shared file mappings is the best
> approximation for what we want there, but there's a considerable amount of
> PITA in the vm_operations_struct that needs to be sorted out first.  In
> particular, ->open() and ->close() are often badly racy *and* there are
> all kinds of bogisities with code making assumptions about impossibility of
> getting several VMAs over the same object (fork() isn't the only thing that
> could lead to that; munmap() the middle will do it as well and VM_DONTCOPY
> won't prevent the latter).

I'm not familiar with mm/ code but vma->vm_file holds a reference to the
file which has been used by mmap(), which seems to mean that so long a
mapping exists, the file is guaranteed not to be released.  This
assumption is useful when the lifetime of the object used by mmap() can
be controlled, for example when it simply is allocated memory.  If this
lifetime can't be controlled and a mapped object is removed when
mappings still exist, I hope there is a way to trigger the vm_ops->fault
handler to give the module some control over resulting faults (I haven't
checked the code to see how drivers handle such a situation).  Of
course, this works only if the module providing vma->vm_ops and
vma->vm_ops->fault is still loaded in memory, which is not guaranteed as
the vma does not take a reference to the module.  This is currently not
an issue because modules which implement ->mmap() use filesystems where
"vma->vm_file = get_file(file)" (in mmap_region [6]) also prevents the
module from being unloaded.

Another possibility consists in making a module close every vma which
has a ->vm_ops in it when it is unloaded, which seems quite hard to do
in a safe way and useless for the modules which are currently in the
kernel tree.

> * as it is, mmap() in procfs has _very_ limited use - /proc/bus/pci/*/*
> is all there, it's never modular and it never uses any data structures
> past ->mmap() anyway.  All pages are already mapped after that point,
> ->access() is the only possibly non-trivial method and even that won't
> look at anything other than vma itself (no looking into ->vm_file, etc.).
> 
> Looks like you went and added another ->mmap() user.  I'm not at all sure
> that procfs or debugfs are good places for that, but that depends on the
> details of what you are trying to do.  _IF_ you really need it in procfs,
> welcome to cleaning the vm_operations_struct out.  Won't be fun - I had
> taken stabs at that several times, never got more than a few isolated
> patches out of any of those attempts...  I have notes from the last such
> work and if you are serious about digging through that crap, I can exhumate
> those...

I'm basically developing a project where I created an out-of-tree module
which uses mmap() as a kind of communication channel between a kernel
module and a userspace program (in short, a ring buffer).  The virtual
file which is used for mmap() was initially created in debugfs, but as
this filesystem is not always mounted on production systems, I needed
another one and ended up using procfs.  I did not consider creating a
character or block device in /dev as I wasn't at ease with grabbing a
major number which could potentially conflict with other out-of-tree
drivers.  As using procfs seems not to be fine, I guess I'll anyway use
a block device with a major number in the experimental range (240-254).

In short, the module I'm writing does not have to be in procfs, and I've
got funnier things to work on than cleaning up vm_operations_struct.

Thank you for your detailed reply.

Nicolas


[1]
https://www.kernel.org/doc/htmldocs/kernel-hacking/routines-module-use-counters.html
[2]
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/fs/open.c?id=v3.19#n716
[3]
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/fs/char_dev.c?id=v3.19#n407
[4]
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/fs/debugfs/inode.c?id=v3.19#n52
[5]
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=881adb85358309ea9c6f707394002719982ec607
[6]
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/mm/mmap.c?id=v3.19#n1625


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/