Hi Phillip,
I was wondering what's the status of mainlining squashfs?
Is there anything I can do to help?
Thanks!
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453
Fax: +32 (0)2 700 8622
E-mail: [email protected]
Internet: http://www.sony-europe.com/
Sony Technology and Software Centre Europe
A division of Sony Service Centre (Europe) N.V.
Registered office: Technologielaan 7 · B-1840 Londerzeel · Belgium
VAT BE 0413.825.160 · RPR Brussels
Fortis 293-0376800-10 GEBA-BE-BB
Geert Uytterhoeven wrote:
> Hi Phillip,
>
> I was wondering what's the status of mainlining squashfs?
> Is there anything I can do to help?
>
I am currently working on this. A re-write and merge of the Squashfs
metadata and fragment cache code has been finished (it is in the
Squashfs CVS tree). The original implementation had a number of
performance issues which I wanted to address before resubmission.
The major work of course is the move to a little endian layout without
any bit fields in the structures to make it easier to swap member fields
on big-endian architectures. I'm currently working on this on a non-CVS
branch. Most of the work till now has been without any attempt to
retain backwards compatibility with the current 3.3 little-endian
filesystem layout. After your private email regarding retaining
backwards compatibility I will look again at this issue, it may well be
that I will abandon my current layout changes that I have so far made.
I have taken time off work to work on the necessary mainline changes
full time (unpaid unfortunately, so I need to look for another job
soon). Hopefully this will mean I will make better progress than I have
been able to in the last couple of years.
Phillip
>>>>> "Phillip" == Phillip Lougher <[email protected]> writes:
Hi,
Phillip> I am currently working on this. A re-write and merge of the
Phillip> Squashfs metadata and fragment cache code has been finished
Phillip> (it is in the Squashfs CVS tree). The original
Phillip> implementation had a number of performance issues which I
Phillip> wanted to address before resubmission.
Nice to hear! Are you aiming for 2.6.27 or later?
Phillip> The major work of course is the move to a little endian
Phillip> layout without any bit fields in the structures to make it
Phillip> easier to swap member fields on big-endian architectures.
Phillip> I'm currently working on this on a non-CVS branch.
Phillip> Most of the work till now has been without any attempt to
Phillip> retain backwards compatibility with the current 3.3
Phillip> little-endian filesystem layout. After your private email
Phillip> regarding retaining backwards compatibility I will look
Phillip> again at this issue, it may well be that I will abandon my
Phillip> current layout changes that I have so far made.
What private mail? Anything of interest to the rest of the group? As I
said earlier, I'm strongly in faviour of cleaning up the layout now
and get rid of all the backwards compatibility code. Doing it once
it's in mainline will be really hard.
Phillip> I have taken time off work to work on the necessary mainline
Phillip> changes full time (unpaid unfortunately, so I need to look
Phillip> for another job soon). Hopefully this will mean I will make
Phillip> better progress than I have been able to in the last couple
Phillip> of years.
;) Good luck with it. Is there anything I can do to help?
--
Bye, Peter Korsgaard
Hi Peter,
On Wed, 4 Jun 2008, Peter Korsgaard wrote:
> >>>>> "Phillip" == Phillip Lougher <[email protected]> writes:
> Phillip> The major work of course is the move to a little endian
> Phillip> layout without any bit fields in the structures to make it
> Phillip> easier to swap member fields on big-endian architectures.
> Phillip> I'm currently working on this on a non-CVS branch.
> Phillip> Most of the work till now has been without any attempt to
> Phillip> retain backwards compatibility with the current 3.3
> Phillip> little-endian filesystem layout. After your private email
> Phillip> regarding retaining backwards compatibility I will look
> Phillip> again at this issue, it may well be that I will abandon my
> Phillip> current layout changes that I have so far made.
>
> What private mail? Anything of interest to the rest of the group? As I
Oops, I've just returned from holidays, and realize I never actually responded
to this question. So here is the gist of it:
| I had a closer look at the structures and bitfields, and I still think it's
| possible to stay compatible with the current little endian layout:
| 1. About half of the bitfields are actually bytes or 16-bit words. Hence
| these can easily be converted to __u8 or __le16.
| 2. Most of the remaining bitfields sit in an 8, 16, or 32 bit integer.
| Hence these can be converted to such an integer (or an array of bytes,
| cfr. 3 below), and accessed using get_unaligned_le{16,32}() and a
| shift/mask operation.
| 3. There are a few bitfields that sit in an `odd' number of bytes. These
| can be converted to an array of bytes, and accessed using
| get_unaligned_le{16,32}() and a shift/mask operation.
| 4. The integral fields in the structs should be accessed using
| get_unaligned_le{8,16,32}(), too.
I noticed the squashfs CVS repository is getting a new (incompatible) 4.0
layout now?
What do people think?
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453
Fax: +32 (0)2 700 8622
E-mail: [email protected]
Internet: http://www.sony-europe.com/
A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis 293-0376800-10 GEBA-BE-BB
On Thu, Jul 17, 2008 at 4:35 PM, Geert Uytterhoeven
<[email protected]> wrote:
> Hi Peter,
>
> On Wed, 4 Jun 2008, Peter Korsgaard wrote:
>> >>>>> "Phillip" == Phillip Lougher <[email protected]> writes:
>> Phillip> The major work of course is the move to a little endian
>> Phillip> layout without any bit fields in the structures to make it
>> Phillip> easier to swap member fields on big-endian architectures.
>> Phillip> I'm currently working on this on a non-CVS branch.
>> Phillip> Most of the work till now has been without any attempt to
>> Phillip> retain backwards compatibility with the current 3.3
>> Phillip> little-endian filesystem layout. After your private email
>> Phillip> regarding retaining backwards compatibility I will look
>> Phillip> again at this issue, it may well be that I will abandon my
>> Phillip> current layout changes that I have so far made.
>>
>> What private mail? Anything of interest to the rest of the group? As I
>
> Oops, I've just returned from holidays, and realize I never actually responded
> to this question. So here is the gist of it:
>
> | I had a closer look at the structures and bitfields, and I still think it's
> | possible to stay compatible with the current little endian layout:
> | 1. About half of the bitfields are actually bytes or 16-bit words. Hence
> | these can easily be converted to __u8 or __le16.
> | 2. Most of the remaining bitfields sit in an 8, 16, or 32 bit integer.
> | Hence these can be converted to such an integer (or an array of bytes,
> | cfr. 3 below), and accessed using get_unaligned_le{16,32}() and a
> | shift/mask operation.
> | 3. There are a few bitfields that sit in an `odd' number of bytes. These
> | can be converted to an array of bytes, and accessed using
> | get_unaligned_le{16,32}() and a shift/mask operation.
> | 4. The integral fields in the structs should be accessed using
> | get_unaligned_le{8,16,32}(), too.
>
> I noticed the squashfs CVS repository is getting a new (incompatible) 4.0
> layout now?
>
> What do people think?
>
Last time I looked you were not the maintainer of squashfs. Phillip
spent a long time examining the possibility of maintaining
compatibility but decided asthat there were sufficient advantages in a
new layout to warrant the change.
Rob.
> With kind regards,
>
> Geert Uytterhoeven
> Software Architect
>
> Sony Techsoft Centre Europe
> The Corporate Village ? Da Vincilaan 7-D1 ? B-1935 Zaventem ? Belgium
>
> Phone: +32 (0)2 700 8453
> Fax: +32 (0)2 700 8622
> E-mail: [email protected]
> Internet: http://www.sony-europe.com/
>
> A division of Sony Europe (Belgium) N.V.
> VAT BE 0413.825.160 ? RPR Brussels
> Fortis 293-0376800-10 GEBA-BE-BB
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Squashfs-devel mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/squashfs-devel
>
>
Hi Robert,
On Thu, 17 Jul 2008, Robert Lougher wrote:
> On Thu, Jul 17, 2008 at 4:35 PM, Geert Uytterhoeven
> <[email protected]> wrote:
> > On Wed, 4 Jun 2008, Peter Korsgaard wrote:
> >> >>>>> "Phillip" == Phillip Lougher <[email protected]> writes:
> >> Phillip> The major work of course is the move to a little endian
> >> Phillip> layout without any bit fields in the structures to make it
> >> Phillip> easier to swap member fields on big-endian architectures.
> >> Phillip> I'm currently working on this on a non-CVS branch.
> >> Phillip> Most of the work till now has been without any attempt to
> >> Phillip> retain backwards compatibility with the current 3.3
> >> Phillip> little-endian filesystem layout. After your private email
> >> Phillip> regarding retaining backwards compatibility I will look
> >> Phillip> again at this issue, it may well be that I will abandon my
> >> Phillip> current layout changes that I have so far made.
> >>
> >> What private mail? Anything of interest to the rest of the group? As I
> >
> > Oops, I've just returned from holidays, and realize I never actually responded
> > to this question. So here is the gist of it:
> >
> > | I had a closer look at the structures and bitfields, and I still think it's
> > | possible to stay compatible with the current little endian layout:
> > | 1. About half of the bitfields are actually bytes or 16-bit words. Hence
> > | these can easily be converted to __u8 or __le16.
> > | 2. Most of the remaining bitfields sit in an 8, 16, or 32 bit integer.
> > | Hence these can be converted to such an integer (or an array of bytes,
> > | cfr. 3 below), and accessed using get_unaligned_le{16,32}() and a
> > | shift/mask operation.
> > | 3. There are a few bitfields that sit in an `odd' number of bytes. These
> > | can be converted to an array of bytes, and accessed using
> > | get_unaligned_le{16,32}() and a shift/mask operation.
> > | 4. The integral fields in the structs should be accessed using
> > | get_unaligned_le{8,16,32}(), too.
> >
> > I noticed the squashfs CVS repository is getting a new (incompatible) 4.0
> > layout now?
> >
> > What do people think?
>
> Last time I looked you were not the maintainer of squashfs. Phillip
Sorry, I don't want to offend anyone. Nor do I intent to become the maintainer.
I just answered Peter's question, because I think people should know the
answer.
> spent a long time examining the possibility of maintaining
> compatibility but decided asthat there were sufficient advantages in a
> new layout to warrant the change.
If there are good reasons for breaking compatibility, we're more than happy to
accept that. Don't you think it would be a good idea to explain them to the
readers of the squashfs-devel mailing list?
Phillip, can you please elaborate? Many thanks in advance!
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre Europe
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453
Fax: +32 (0)2 700 8622
E-mail: [email protected]
Internet: http://www.sony-europe.com/
A division of Sony Europe (Belgium) N.V.
VAT BE 0413.825.160 · RPR Brussels
Fortis 293-0376800-10 GEBA-BE-BB
Hi,
On Fri, Jul 18, 2008 at 7:14 AM, Geert Uytterhoeven
<[email protected]> wrote:
> If there are good reasons for breaking compatibility, we're more than happy to
> accept that. Don't you think it would be a good idea to explain them to the
> readers of the squashfs-devel mailing list?
>
> Phillip, can you please elaborate? Many thanks in advance!
>
We need to wait for Phillip to reply for the details. But my
understanding is that Phillip is using this as an opportunity to clean
up the layout and to remove limitations. This may give Squashfs an
easier ride in mainlining reviews, and once Squashfs is mainlined
layout changes may become much more difficult, so it's wise to do it
now.
Rob.
> With kind regards,
>
> Geert Uytterhoeven
> Software Architect
>
> Sony Techsoft Centre Europe
> The Corporate Village ? Da Vincilaan 7-D1 ? B-1935 Zaventem ? Belgium
>
> Phone: +32 (0)2 700 8453
> Fax: +32 (0)2 700 8622
> E-mail: [email protected]
> Internet: http://www.sony-europe.com/
>
> A division of Sony Europe (Belgium) N.V.
> VAT BE 0413.825.160 ? RPR Brussels
> Fortis 293-0376800-10 GEBA-BE-BB