It is great that squashfs is finally in mainline kernel. But the
squashfs tools are tightly bound to the version of the squashfs filesystem.
The latest tools in Debian (and the download page) are limited to
version 3.4 but the upstream kernel version is 4.0!
Do we have a cart with no horse here?
Stephen Hemminger wrote:
> It is great that squashfs is finally in mainline kernel. But the
> squashfs tools are tightly bound to the version of the squashfs filesystem.
> The latest tools in Debian (and the download page) are limited to
> version 3.4 but the upstream kernel version is 4.0!
>
> Do we have a cart with no horse here?
>
The horse is currently in training, or to put it more mundanely, the
squashfs 4.0 tools are currently in Squashfs CVS :-)
I have been working on the 4.0 squashfs tools for the last couple of
months making them (mksquashfs and unsquashfs) ready for a supported
release. Now that Linux 2.6.29 has been released (3 hours ago), I
need to make this release ASAP, which as everything is almost finished,
should be sometime this week.
Phillip
Phillip Lougher wrote:
> Stephen Hemminger wrote:
>> It is great that squashfs is finally in mainline kernel. But the
>> squashfs tools are tightly bound to the version of the squashfs
>> filesystem.
[]
> I have been working on the 4.0 squashfs tools for the last couple of
> months making them (mksquashfs and unsquashfs) ready for a supported
> release. Now that Linux 2.6.29 has been released (3 hours ago), I
> need to make this release ASAP, which as everything is almost finished,
> should be sometime this week.
By the way, with this on-disk format change, is there a way to get
4.0 fs code working with 2.6.27 (long-time-stable) kernel? I tried
to compile it on 2.6.27 but it fails due to some API changes and I'm
not that deep into kernel to fix it...
Thanks!
On Monday 23 March 2009 21:39:02 Phillip Lougher wrote:
> Stephen Hemminger wrote:
> > It is great that squashfs is finally in mainline kernel. But the
> > squashfs tools are tightly bound to the version of the squashfs
> > filesystem. The latest tools in Debian (and the download page) are
> > limited to version 3.4 but the upstream kernel version is 4.0!
> >
> > Do we have a cart with no horse here?
>
> The horse is currently in training, or to put it more mundanely, the
> squashfs 4.0 tools are currently in Squashfs CVS :-)
>
> I have been working on the 4.0 squashfs tools for the last couple of
> months making them (mksquashfs and unsquashfs) ready for a supported
> release. Now that Linux 2.6.29 has been released (3 hours ago), I
> need to make this release ASAP, which as everything is almost finished,
> should be sometime this week.
Any updated ETA on this?
Rob
P.S I notice a squashfs-devel mailing list cc'd here, but there's no link to
it from the squashfs.org page. Buried in the middle of a lot of text is a
link to the sourceforge squashfs project page, which has a link to "forums"
which are not the same as mailing lists.
Since I knew it's called "squashfs-devel" I could google for that and find
http://sourceforge.net/mailarchive/forum.php?forum_name=squashfs-devel
But it might be nice to at least _hint_ about its existence on the web page.
The main page's "contact" section is your personal email and nothing else...
On Sun, Mar 29, 2009 at 00:27, Rob Landley <[email protected]> wrote:
> P.S I notice a squashfs-devel mailing list cc'd here, but there's no link to
> it from the squashfs.org page. Buried in the middle of a lot of text is a
> link to the sourceforge squashfs project page, which has a link to "forums"
> which are not the same as mailing lists.
There should be a link to it somewhere on the SF project page...
> But it might be nice to at least _hint_ about its existence on the web page.
> The main page's "contact" section is your personal email and nothing else...
Yep. Fortunately we also have MAINTAINERS ;-)
| SQUASHFS FILE SYSTEM
| P: Phillip Lougher
| M: [email protected]
| L: [email protected] (subscribers-only)
| W: http://squashfs.org.uk
| S: Maintained
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
On Sun, 29 Mar 2009 11:02:19 +0200
Geert Uytterhoeven <[email protected]> wrote:
> On Sun, Mar 29, 2009 at 00:27, Rob Landley <[email protected]> wrote:
> > P.S I notice a squashfs-devel mailing list cc'd here, but there's no link to
> > it from the squashfs.org page. Buried in the middle of a lot of text is a
> > link to the sourceforge squashfs project page, which has a link to "forums"
> > which are not the same as mailing lists.
>
> There should be a link to it somewhere on the SF project page...
>
> > But it might be nice to at least _hint_ about its existence on the web page.
> > The main page's "contact" section is your personal email and nothing else...
>
> Yep. Fortunately we also have MAINTAINERS ;-)
>
> | SQUASHFS FILE SYSTEM
> | P: Phillip Lougher
> | M: [email protected]
> | L: [email protected] (subscribers-only)
> | W: http://squashfs.org.uk
> | S: Maintained
>
> Gr{oetje,eeting}s,
>
> Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- [email protected]
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like that.
> -- Linus Torvalds
I built it from cvs (imported into git), and it works okay.
Stephen Hemminger wrote:
> On Sun, 29 Mar 2009 11:02:19 +0200
> Geert Uytterhoeven <[email protected]> wrote:
>
>> On Sun, Mar 29, 2009 at 00:27, Rob Landley <[email protected]> wrote:
>>> P.S I notice a squashfs-devel mailing list cc'd here, but there's no link to
>>> it from the squashfs.org page. Buried in the middle of a lot of text is a
>>> link to the sourceforge squashfs project page, which has a link to "forums"
>>> which are not the same as mailing lists.
>> There should be a link to it somewhere on the SF project page...
>>
>>> But it might be nice to at least _hint_ about its existence on the web page.
>>> The main page's "contact" section is your personal email and nothing else...
Squashfs is a sourceforge project, and it has a link to the main sourceforge
page from the webpage. I've always thought that sufficient because users
can get to the mailing lists, web forum, bug tracker etc. from that, and then
choose what they think is the most appropriate way to report issues.
Missing information is simply that, and I rely on feedback from users to
say if I've forgotten to add important or relevant information anywhere.
It would be helpful if that feedback didn't include an implied
criticism, that way we'll all get along better.
>
> I built it from cvs (imported into git), and it works okay.
>
Good, this is what I suggested you do :-) CVS contains everything I've
been working on for the last couple of months.
I intended to do a release last week, but, release testing found a number
of bugs and other issues which had to be addressed, and which consequently
delayed the release.
If you grabbed the tools from CVS more than a day ago you'll find
there's problems with Unsquashfs and big-endian systems. The last
bug fix (a deadlock issue in mksquashfs undiscovered since 3.4)
went in just now, and so CVS should be what's in the release.
BTW Stephen, it would be helpful if you subscribed to the
squashfs-devel mailing list, at the moment I've having to
authorise all your posts which is a bit of a pain.
Phillip
Michael Tokarev wrote:
> Phillip Lougher wrote:
>> Stephen Hemminger wrote:
>>> It is great that squashfs is finally in mainline kernel. But the
>>> squashfs tools are tightly bound to the version of the squashfs
>>> filesystem.
> []
>> I have been working on the 4.0 squashfs tools for the last couple of
>> months making them (mksquashfs and unsquashfs) ready for a supported
>> release. Now that Linux 2.6.29 has been released (3 hours ago), I
>> need to make this release ASAP, which as everything is almost finished,
>> should be sometime this week.
>
> By the way, with this on-disk format change, is there a way to get
> 4.0 fs code working with 2.6.27 (long-time-stable) kernel?
I tried to compile it on 2.6.27 but it fails due to some API changes and I'm
> not that deep into kernel to fix it...
>
Yes I know. The 4.0 patches only work on 2.6.28 or later AFAIK.
When I get time and if there's demand (you're the first person to ask
and so there's obviously some demand), I'll backport the patches
to earlier kernels.
Phillip
On Monday 30 March 2009 19:55:24 Stephen Hemminger wrote:
> I built it from cvs (imported into git), and it works okay.
I grabbed the cvs snapshot gentoo's been using, but I don't see any way to
feed in a device list text file so I can create the /dev/console node root
filesystems need without needing root access on the host to mknod.
genext2fs has the -D option, which takes:
/dev d 755 0 0 - - - - -
/dev/console c 640 0 0 5 1 0 0 -
gen_init_cpio takes:
dir /dev 755 0 0
nod /dev/console 660 0 0 c 5 1
Is there a squashfs option I'm missing...?
Rob
--
GPLv3 is to GPLv2 what Attack of the Clones is to The Empire Strikes Back.
Rob Landley wrote:
> On Monday 30 March 2009 19:55:24 Stephen Hemminger wrote:
>> I built it from cvs (imported into git), and it works okay.
>
> I grabbed the cvs snapshot gentoo's been using, but I don't see any way to
> feed in a device list text file so I can create the /dev/console node root
> filesystems need without needing root access on the host to mknod.
>
> genext2fs has the -D option, which takes:
>
> /dev d 755 0 0 - - - - -
> /dev/console c 640 0 0 5 1 0 0 -
>
> gen_init_cpio takes:
>
> dir /dev 755 0 0
> nod /dev/console 660 0 0 c 5 1
>
> Is there a squashfs option I'm missing...?
>
No you're unfortunately not missing an option :-) That option
doesn't exist.
But thanks for sending the email, a couple of people over the
years have asked for something like this, and you've reminded
me that this option doesn't exist in Mksquashfs...
I have thought about adding this option a couple of times, but
it has always been dropped through lack of time. Efficiently
adding the option isn't as easy as it sounds because the "fake"
device nodes have to somehow be made available to the
recursive directory scan routine that scans the source
directories for files. Adding a list of fake device
nodes which is scanned on every directory for potential inclusion
in that directory is easy, but not terribly efficient.
I'll delay the release of the 4.0 tools and see if I can
implement this option in the next day or two.
Phillip
>>>>> "Phillip" == Phillip Lougher <[email protected]> writes:
Hi,
Phillip> I have thought about adding this option a couple of times,
Phillip> but it has always been dropped through lack of time.
Phillip> Efficiently adding the option isn't as easy as it sounds
Phillip> because the "fake" device nodes have to somehow be made
Phillip> available to the recursive directory scan routine that scans
Phillip> the source directories for files. Adding a list of fake
Phillip> device nodes which is scanned on every directory for
Phillip> potential inclusion in that directory is easy, but not
Phillip> terribly efficient.
The normal solution for this is fakeroot, which is also what we're
using for mksquashfs in buildroot:
http://git.buildroot.net/buildroot/tree/target/squashfs/squashfsroot.mk
Phillip> I'll delay the release of the 4.0 tools and see if I can
Phillip> implement this option in the next day or two.
I would say rather get 4.0 out of the door as soon as possible and
wait with such nice-to-have stuff for later.
--
Bye, Peter Korsgaard
Phillip Lougher wrote:
> Rob Landley wrote:
>> On Monday 30 March 2009 19:55:24 Stephen Hemminger wrote:
>>> I built it from cvs (imported into git), and it works okay.
>>
>> I grabbed the cvs snapshot gentoo's been using, but I don't see any
>> way to feed in a device list text file so I can create the
>> /dev/console node root filesystems need without needing root access on
>> the host to mknod.
>>
>> genext2fs has the -D option, which takes:
>>
>> /dev d 755 0 0 - - - - -
>> /dev/console c 640 0 0 5 1 0 0 -
>>
>> gen_init_cpio takes:
>>
>> dir /dev 755 0 0
>> nod /dev/console 660 0 0 c 5 1
>>
>> Is there a squashfs option I'm missing...?
>>
> I'll delay the release of the 4.0 tools and see if I can
> implement this option in the next day or two.
The Squashfs 4.0 tools have been released on sourceforge
(http://www.squashfs.org), and mksquashfs now includes support for
fake devices and directories.
From README-4.0
Mksquashfs now supports pseudo files, these allow fake directories, character
and block devices to be specified and added to the Squashfs filesystem being
built, rather than requiring them to be present in the source directories.
This, for example, allows device nodes to be added to the filesystem without
requiring root access.
Two options are supported, -p allows one pseudo file to be specified on the
command line, and -pf allows a pseudo file to be specified containing a
list of pseudo definitions, one per line.
Pseudo device nodes are specified using 7 arguments
Filename type mode uid gid major minor
Where type is either
b - for block devices, and
c - for character devices
mode is the octal mode specifier, similar to that expected by chmod.
Uid and gid can be either specified as a decimal number, or by name.
For example:
/dev/chr_dev c 666 root root 100 1
/dev/blk_dev b 444 0 0 200 200
Directories are specified using 5 arguments
Filename type mode uid gid
Where type is d.
To ensure the pseudo definition is parsed as one argument and to
ensure it isn't interpreted by the shell, the definition should be enclosed
in quotes when using the -p option, i.e.
mksquashfs test test.sqsh -p "/dev/chrdev c 666 root 100 1"
If a directory given in a pseudo file path exists in the real source
directories it doesn't have to be specified as a pseudo directory, if it
doesn't exist in the source directories, it must be specified as a pseudo
directory (otherwise any pseudo devices under the directory will be ignored).
For example
/dev d 777 root root
/dev/chr_dev c 666 root root 100 1
Phillip
On Sunday 05 April 2009 18:27:49 Phillip Lougher wrote:
> Phillip Lougher wrote:
> > Rob Landley wrote:
> >> On Monday 30 March 2009 19:55:24 Stephen Hemminger wrote:
> >>> I built it from cvs (imported into git), and it works okay.
> >>
> >> I grabbed the cvs snapshot gentoo's been using, but I don't see any
> >> way to feed in a device list text file so I can create the
> >> /dev/console node root filesystems need without needing root access on
> >> the host to mknod.
> >>
> >> genext2fs has the -D option, which takes:
> >>
> >> /dev d 755 0 0 - - - - -
> >> /dev/console c 640 0 0 5 1 0 0 -
> >>
> >> gen_init_cpio takes:
> >>
> >> dir /dev 755 0 0
> >> nod /dev/console 660 0 0 c 5 1
> >>
> >> Is there a squashfs option I'm missing...?
> >
> > I'll delay the release of the 4.0 tools and see if I can
> > implement this option in the next day or two.
>
> The Squashfs 4.0 tools have been released on sourceforge
> (http://www.squashfs.org), and mksquashfs now includes support for
> fake devices and directories.
>
> From README-4.0
Some suggested wordsmithing on the text you posted:
---
Mksquashfs now supports pseudo-files, which may be specified via the -p
and -pf command line options.
Pseudo-files allow fake directories, character devices, and block devices to
be added to the Squashfs filesystem being built without requiring them to be
present in the source filesystem. This allows a non-root user to create a
squashfs filesystem containing device nodes and directories (but files) with
arbitrary ownership/permissions.
The -p option specifies an individual pseudo-file, and takes a single argument
such as -p "/dev d 755 root root". The -p option may be repeated as
necessary.
The -pf option specifies a file containing a list of pseudo-file definitions,
one per line.
Pseudo-file definitions look like this:
/dev d 755 root root
/dev/chr_dev c 666 root root 100 1
/dev/blk_dev b 444 0 0 200 200
The first field is the complete path to the new entry in the squashfs
filesystem. (This path cannot currently contain any whitespace space
characters.) The second field is a single letter indicating the entry type.
Currently supported types are d (directory), b (block device), and c (char
device). The third, fourth, and fifth fields are (respectively) octal mode,
user id, and group ID. UID and GID can be specified either by decimal number
or using a name valid on the host filesystem.
Directory pseudo-file definitions only contain five fields. Device
pseudo-file definitions include major and minor numbers in the sixth and
seventh fields.
To ensure the pseudo-file definition is parsed as one argument, and to
ensure it isn't interpreted by the shell, the definition should be enclosed
in quotes when using the -p option, i.e.
mksquashfs test test.sqsh -p "/dev/chrdev c 666 root 100 1"
If a directory given in a pseudo file path exists in the real source
directories it doesn't have to be specified as a pseudo directory, if it
doesn't exist in the source directories, it must first be specified as a
pseudo directory (otherwise any pseudo devices under the directory will be
ignored).
For example
/dev d 755 root root
/dev/chr_dev c 666 root root 100 1
---
Thanks, looks good. I should be able to test it out in the morning.
Rob
--
GPLv3 is to GPLv2 what Attack of the Clones is to The Empire Strikes Back.