Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S261190AbULACpw (ORCPT ); Tue, 30 Nov 2004 21:45:52 -0500 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S261202AbULACpw (ORCPT ); Tue, 30 Nov 2004 21:45:52 -0500 Received: from wproxy.gmail.com ([64.233.184.194]:47166 "EHLO wproxy.gmail.com") by vger.kernel.org with ESMTP id S261190AbULACpX (ORCPT ); Tue, 30 Nov 2004 21:45:23 -0500 DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:reply-to:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:references; b=IbQnuqB0ratmakd5QS5lsNy03EvPph683ycEbUsB0QzMbvqjsqqTdrL5Gs0bCeJ1IEbtCUF5HOZKmQIYxxF6jAYYMFFHnKQ+WVzrQfrhaVlXWJ5TMPLYyLaXZu5Hr/pqQzOW/4/GqXmatsQx5YsKGojnseYw7+sZLnG/zxgAd1s= Message-ID: Date: Tue, 30 Nov 2004 21:44:52 -0500 From: Scott Young Reply-To: Scott Young To: Jan Engelhardt Subject: Re: file as a directory Cc: Amit Gud , linux-kernel@vger.kernel.org, reisefs-list@namesys.com, Hans Reiser In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit References: <200411292120.iATLKZxE004233@laptop11.inf.utfsm.cl> <1101832103.2885.4.camel@zathras.emsl.pnl.gov> <04113011354200.08643@tabby> <2c59f00304113010262063d219@mail.gmail.com> Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3445 Lines: 71 On Tue, 30 Nov 2004 19:39:15 +0100 (MET), Jan Engelhardt wrote: > >My suggestion is to add a framework, an infrastructure, in the VFS > >wherein a simple plugin can be written to poke into the file as if it > >were a directory. So with that framework in place, I can write a > >plugin for archive support (treating the .tar files as directories), > >Peter could write a plugin for poking into /etc/passwd (treating it as > >a directory), and Jon Doe could write a plugin for sendmail.cf The biggest problem I see with adding the complicated stuff to VFS is the bloat and risk to system stability. However, some things cannot be done in userspace, such as good caching. How is one userspace library supposed to keep a transparent cache of, for example, an index for a tar file, not clutter up the on-disk representation of the cache, effectively manage space utilization, and be able to efficiently detect changes to files in order to invalidate the cache? This would become orders of magnitude easier if a ubiquitous filesystem interface were in use. However, the only ubiquitous filesystem interface is VFS, which shouldn't have to take all the code bloat. Maybe something crazy could work. Let's take some concepts from the Aspect Oriented Programming paradigm. Whenever a program is loaded into memory, calls in the program to the vfs interface are modified to instead call new userspace functions that have all of the desired functionality, and those userspace functions eventually call the real system functions. The kernel wouldn't have to take the bloat, plus it would be able to do things the userspace libraries wouldn't be able to do efficiently. It's the best of both worlds, with a little insanity thrown in (It'd be neat to see the loader bootstrap its own code to weave in the caching of the pre-woven binaries). > That's something I could live with, but how do you want to tag a file being > "tar" so that tar_ops is used instead of the "default file" ops? > > You could not do so without an extra function, and once you use that extra > function to tag a certain file being "tar" -- you know that extensions are > kinda "worthless", and, especially, unrealiable -- you could also have used tar > -tvf. > > Did I mention tar is not the perfect format? It's because it is lacking an > index and letting the kernel wade through a GB-sized tar file just to perform > and readdir (yet imagine reading the last file of it) would be a hell of > skipping. Keeping a non-persistent index in memory may solve the problem, but > hey, I also do not want to spend too much memory just for a single tar file. It would also be nice to have an interface which can build, maintain, and cache on the disk a persistent index into a tar file on the disk, and then be able to delete this index when space is running low. Plus, this index could be generated by streaming the file through memory, so you don't need to consume too much memory for a single file. > >struct file_operations ops = { > > .read = tar_readdir, > > .readdir = tar_readdir, > > ...... > >}; > > > >register_file_type("tar", &ops); > > Jan Engelhardt > -- > ENOSPC - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/