Hello everyone,
At some point it probably would make sense to start deprecating .gz
format files from kernel.org.
I am envisioning this as a three-phase changeover:
a) Get all mirrors to carry .bz2 format. This would affect the
following sites:
DUTH:format=gz
GARBO:format=gz
HCMC:format=gz
IGLU:format=gz
LINUXAID:format=gz
LLARIAN-NET:format=gz
MINET-FR:format=gz
NC-ORC:format=gz
PCSS:format=gz
PROGRAMVAREVERKSTEDET:format=gz
PUB-FTP-UNIVERSITY-OF-OLDENBURG:format=gz
RN-RNO:format=gz
TASK:format=gz
TELEPAC:format=gz
TENGU-EASYNET-FR:format=gz
UNC-METALAB:format=gz
WEBLAB:format=gz
b) Once that is done, change the robots to no longer require .gz files;
.bz2 files uploaded would be signed but no .gz file would be generated.
-> If we get a complete loss of data here, all .gz files would be lost.
c) At some point, deprecate .gz uploads entirely and remove all the old
.gz files. After that point .gz files uploaded would be treated just
like .Z, .zip or any other "unmanaged" compression format.
Now, the questions that come up are:
i) Does this sound reasonable to everyone? In particular, is there any
loss in losing the "original" compressed files?
ii) Assuming a yes on the previous question, what time frame would it
make sense for this changeover to happen over?
-hpa
On Wed, Mar 19, 2003 at 12:19:42PM -0800, H. Peter Anvin wrote:
> Hello everyone,
>
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
>
> I am envisioning this as a three-phase changeover:
>
> a) Get all mirrors to carry .bz2 format. This would affect the
> following sites:
>
> DUTH:format=gz
> GARBO:format=gz
> HCMC:format=gz
> IGLU:format=gz
> LINUXAID:format=gz
> LLARIAN-NET:format=gz
> MINET-FR:format=gz
> NC-ORC:format=gz
> PCSS:format=gz
> PROGRAMVAREVERKSTEDET:format=gz
> PUB-FTP-UNIVERSITY-OF-OLDENBURG:format=gz
> RN-RNO:format=gz
> TASK:format=gz
> TELEPAC:format=gz
> TENGU-EASYNET-FR:format=gz
> UNC-METALAB:format=gz
> WEBLAB:format=gz
>
> b) Once that is done, change the robots to no longer require .gz files;
> .bz2 files uploaded would be signed but no .gz file would be generated.
>
> -> If we get a complete loss of data here, all .gz files would be lost.
>
> c) At some point, deprecate .gz uploads entirely and remove all the old
> .gz files. After that point .gz files uploaded would be treated just
> like .Z, .zip or any other "unmanaged" compression format.
>
>
> Now, the questions that come up are:
>
> i) Does this sound reasonable to everyone? In particular, is there any
> loss in losing the "original" compressed files?
I don't think phase three is really needed...
It sounds reasonable, but if it's mainly due to space consuption, I
think it would be nice to code a bzip2-to-gz cgi server so that there is
no loss on functionality. This would need CPU power, so perhaps
rate-limiting the sending side of the CGI would manage to keep
the CPU low on use.
> ii) Assuming a yes on the previous question, what time frame would it
> make sense for this changeover to happen over?
Don't know.
Greets anyways, Antonio.
Hello Peter , IMO it ain't going away '.' . My systems will not
carry .bz2's , that is a Fyi . Not that it matters for systems
over which you have opinion &/or control . Thanks for listening .
JimL
On Wed, 19 Mar 2003, H. Peter Anvin wrote:
> Hello everyone,
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
> I am envisioning this as a three-phase changeover:
> a) Get all mirrors to carry .bz2 format. This would affect the
> following sites:
--
+------------------------------------------------------------------+
| James W. Laferriere | System Techniques | Give me VMS |
| Network Engineer | P.O. Box 854 | Give me Linux |
| [email protected] | Coudersport PA 16915 | only on AXP |
+------------------------------------------------------------------+
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
Yay!
> Now, the questions that come up are:
>
> i) Does this sound reasonable to everyone? In particular, is there any
> loss in losing the "original" compressed files?
Can't see why ... the duplication seems rather silly.
> ii) Assuming a yes on the previous question, what time frame would it
> make sense for this changeover to happen over?
As soon as you can get the mirrors to change over?
M.
On Wed, 19 Mar 2003, H. Peter Anvin wrote:
> i) Does this sound reasonable to everyone? In particular, is there any
> loss in losing the "original" compressed files?
No, there is at least one reason for the "original" .gz files. Here are
the logical steps:
a) any Linux distribution contains their own "linux" package with the
source base being "vanilla" Linux .tar.gz file
b) switching such to .tar.bz2 will make building the package longer
because of longer extract times
c) re-running tar to generate a .tar.gz from .tar.bz2 and store the
.tar.gz instead will make customers suspicious --- i.e. they will have to
ask "is this _really_ a plain Linux tree or do I need to run diff(1) to
verify, just in case?"
See the reasoning? However, I agree that this reason is very weak. But you
were interested in any reasons, including weak ones, I assume :)
Regards
Tigran
>
> ii) Assuming a yes on the previous question, what time frame would it
> make sense for this changeover to happen over?
>
> -hpa
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
On Wed, Mar 19, 2003 at 12:19:42PM -0800, H. Peter Anvin wrote:
> Hello everyone,
>
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
It is a convinient feature that I can download the kernel now and then
when at work (no linux) - and it makes it a bit simpler to use
an archiver that has native support for the format used.
Winzip, being the only allowed archiver at work, does not
have native bz2 support.
I have a .bz2 commandline version on the windoze machine so I will cope,
but mainly to let you know that deprecating .gz you will make it
difficult for a minority to read the kernel src.
Sam
On Wed, 2003-03-19 at 22:12, Tigran Aivazian wrote:
> On Wed, 19 Mar 2003, H. Peter Anvin wrote:
> > i) Does this sound reasonable to everyone? In particular, is there any
> > loss in losing the "original" compressed files?
>
> No, there is at least one reason for the "original" .gz files. Here are
> the logical steps:
>
> a) any Linux distribution contains their own "linux" package with the
> source base being "vanilla" Linux .tar.gz file
I can't speak for the others, but Red Hat Linux uses the .bz2 files in
kernel rpms
Em Wed, Mar 19, 2003 at 10:42:07PM +0100, Arjan van de Ven escreveu:
> On Wed, 2003-03-19 at 22:12, Tigran Aivazian wrote:
> > On Wed, 19 Mar 2003, H. Peter Anvin wrote:
> > a) any Linux distribution contains their own "linux" package with the
> > source base being "vanilla" Linux .tar.gz file
> I can't speak for the others, but Red Hat Linux uses the .bz2 files in
> kernel rpms
Conectiva too.
- Arnaldo
Hi,
On Wed, Mar 19, 2003 at 06:55:52PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Wed, Mar 19, 2003 at 10:42:07PM +0100, Arjan van de Ven escreveu:
> > I can't speak for the others, but Red Hat Linux uses the .bz2 files in
> > kernel rpms
>
> Conectiva too.
... and so does SuSE.
Regards,
--
Kurt Garloff <[email protected]> Eindhoven, NL
GPG key: See mail header, key servers SuSE Labs (Head)
SuSE Linux AG, Nuernberg, DE SCSI, Security
H. Peter Anvin wrote:
> Hello everyone,
>
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
>
OK, there seems to be enough resistance to this to put it off for a year
or two. Since that means I don't have to do any work, that plan has
inherent appeal to me.
In other words, the current setup will remain for now. HOWEVER, I would
like to recommend that mirror sites start carrying .bz2 files if they
want to carry only one format.
-hpa
Kurt Garloff said:
> Hi,
>
> On Wed, Mar 19, 2003 at 06:55:52PM -0300, Arnaldo Carvalho de Melo
> wrote:
>> Em Wed, Mar 19, 2003 at 10:42:07PM +0100, Arjan van de Ven escreveu:
>> > I can't speak for the others, but Red Hat Linux uses the .bz2 files
>> in kernel rpms
>>
>> Conectiva too.
>
> ... and so does SuSE.
And Source Mage (http://www.sourcemage.org). :)
-Sandalle
--
PGP Key 0x5C8D199A5A317214
http://search.keyserver.net:11371/pks/lookup?op=get&search=0x5A317214
Eric Sandall | Source Mage GNU/Linux Developer
[email protected] | http://www.sourcemage.org/
http://eric.sandall.us/ | SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/ #196285 | http://www.shock.wsu.edu/
From: "H. Peter Anvin" <[email protected]>
Date: Wed, 19 Mar 2003 12:19:42 -0800
Now, the questions that come up are:
i) Does this sound reasonable to everyone? In particular, is there any
loss in losing the "original" compressed files?
ii) Assuming a yes on the previous question, what time frame would it
make sense for this changeover to happen over?
I'm fine with this, and my personal feeling on time is the sooner
the better.
It's pretty hard to find a Linux system without bzip2 these days.
On Wed, 19 Mar 2003, H. Peter Anvin wrote:
> i) Does this sound reasonable to everyone? In particular, is there any
> loss in losing the "original" compressed files?
Personally I fetch the .bz2 tar files for a few base kernel versions,
but I fetch the .gz patch files.
This is so that I can "zgrep" through the patch files looking for
which version changed some feature or API.
bzgrep exists, but it is way too slow.
So if there were only .bz2 patch files, I would fetch them and convert
them back into .gz files on my local mirror.
Which is ok of course, but then the signatures don't match any more.
Well, that's my really weak reason for liking .gz patches.
enjoy,
-- Jamie
> OK, there seems to be enough resistance to this to put it off for a year
> or two. Since that means I don't have to do any work, that plan has
> inherent appeal to me.
>
> In other words, the current setup will remain for now. HOWEVER, I would
> like to recommend that mirror sites start carrying .bz2 files if they
> want to carry only one format.
Can we at least switch the default upload format to bz2? I would think
that's a reasonably simple change to the robot? For those of us
uploading stuff over a slow modem link, the extra efficiency would be
much appreciated.
M.
Martin J. Bligh wrote:
>>OK, there seems to be enough resistance to this to put it off for a year
>>or two. Since that means I don't have to do any work, that plan has
>>inherent appeal to me.
>>
>>In other words, the current setup will remain for now. HOWEVER, I would
>>like to recommend that mirror sites start carrying .bz2 files if they
>>want to carry only one format.
>
>
> Can we at least switch the default upload format to bz2? I would think
> that's a reasonably simple change to the robot? For those of us
> uploading stuff over a slow modem link, the extra efficiency would be
> much appreciated.
>
No, I don't want to muck with working scripts. I suggest setting up a
script to upload to the /staging area and convert automatically. I have
put a script called bz2togz in /usr/local/bin to help out.
-hpa
"Eric Sandall" <[email protected]> writes:
> > > > I can't speak for the others, but Red Hat Linux uses the .bz2
> > > > files in kernel rpms
> > >
> > > Conectiva too.
> >
> > ... and so does SuSE.
>
> And Source Mage (http://www.sourcemage.org). :)
mandrake too ...
Thierry Vignaud <[email protected]> writes:
> "Eric Sandall" <[email protected]> writes:
>
>> > > > I can't speak for the others, but Red Hat Linux uses the .bz2
>> > > > files in kernel rpms
>> > >
>> > > Conectiva too.
>> >
>> > ... and so does SuSE.
>>
>> And Source Mage (http://www.sourcemage.org). :)
>
> mandrake too ...
Debian as well.
--
ilmari
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
Presumably the idea is to save bandwidth. In that case, it might be
simpler to add:
<h1>.gz is depreciated and will be removed soon!</h1>
Please start using .bz2 immediately. .gz downloads are rate limited
to 5kbps.
to index.html :-).
Seriously, there are occasions when you're patching a kernel on a 486
or something, that it's quicker to use the .gz than the .bz2, even
including the download time, (if you're on a fast connection).
However, there are probably a lot of people downloading .gz, because
they don't know what .bz2 is.
Likewise, http access to kernel.org is frequently slow, whereas ftp
access can use 100% of my link, simply because many more people use
http.
One other thing that occurs to me, the send your e-mail address as
anonymous password requirement means I can't use wget to access the
ftp site, (I think - the user:password@host syntax seems to confuse
it, when it's anonymous:me@[email protected]).
John.
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
H. Peter Anvin ([email protected]) wrote:
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
[...]
I think it may be better to dump the .gz format for 2.5 and the forthcoming
2.6 series in favour of .bz2. Keep .gz for 2.[0-4] for those who need it (I
certainly don't).
- --
Erik Hensema <[email protected]>
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org
iD8DBQE+eZINuNWfqpFSu/cRAkJfAJ9LNq+2krlpaU9PTNE0m0pVgMtgngCg+L8G
FT6d5flYtelPMNCoEvoZQTY=
=MUhg
-----END PGP SIGNATURE-----
Kurt Garloff ([email protected]) said:
> On Wed, Mar 19, 2003 at 06:55:52PM -0300, Arnaldo Carvalho de Melo wrote:
> > Em Wed, Mar 19, 2003 at 10:42:07PM +0100, Arjan van de Ven escreveu:
> > > I can't speak for the others, but Red Hat Linux uses the .bz2 files in
> > > kernel rpms
> >
> > Conectiva too.
>
> ... and so does SuSE.
>
... and so does MandrakeSoft (as it seems from their rpm).
--
Linux Solution Consultant T?l: +33 476 143 278 - Fax: +33 476 146 105
HP/Intel Solution Center http://hpintelco.net Hewlett-Packard Grenoble/France
Des infos sur Linux? http://www.HyPer-Linux.org http://www.hp.com/linux
La musique ancienne? http://www.musique-ancienne.org http://www.medieval.org
Among the latest 2.4-release kernels (2.4.19 and 2.4.20), it seems that
bz2 saves ~6MB.
Downloads: 1.5MB DSL
time `ncftpget ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.20.tar.gz`
real 3m24.004s
<snipped>
time `ncftpget ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.20.tar.bz2`
real 2m51.481s
<snipped>
Uncompression: Dual AMD 1600+, 512MB ram, 30 GB seagate EIDE
time `gunzip linux-2.4.20.tar.gz`
real 0m5.428s
user 0m2.285s
sys 0m1.096s
time `bunzip2 linux-2.4.20.tar.bz2 `
real 0m28.892s
user 0m27.318s
sys 0m1.363s
Compression: Dual AMD 1600+, 512MB ram, 30 GB seagate EIDE
time `gzip linux-2.4.20.tar`
real 0m18.771s
user 0m17.990s
sys 0m0.674s
time `gzip -9 linux-2.4.20.tar`
real 0m42.032s
user 0m40.725s
sys 0m0.791s
time `bzip2 linux-2.4.20.tar`
real 1m50.411s
user 1m49.197s
sys 0m0.555s
bz2 is about 18% of the size of the tarfile. gz is 22%. gzip -9 saved a
whopping 310k compared to gzip.
--
-- John E. Jasen ([email protected])
-- User Error #2361: Please insert coffee and try again.
On Thu, 20 Mar 2003, John Jasen wrote:
>
> Among the latest 2.4-release kernels (2.4.19 and 2.4.20), it seems that
> bz2 saves ~6MB.
>
> Downloads: 1.5MB DSL
>
> time `ncftpget ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.20.tar.gz`
> real 3m24.004s
> <snipped>
>
> time `ncftpget ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-2.4.20.tar.bz2`
> real 2m51.481s
> <snipped>
>
> Uncompression: Dual AMD 1600+, 512MB ram, 30 GB seagate EIDE
> time `gunzip linux-2.4.20.tar.gz`
> real 0m5.428s
> user 0m2.285s
> sys 0m1.096s
>
> time `bunzip2 linux-2.4.20.tar.bz2 `
> real 0m28.892s
> user 0m27.318s
> sys 0m1.363s
>
> Compression: Dual AMD 1600+, 512MB ram, 30 GB seagate EIDE
> time `gzip linux-2.4.20.tar`
> real 0m18.771s
> user 0m17.990s
> sys 0m0.674s
>
> time `gzip -9 linux-2.4.20.tar`
> real 0m42.032s
> user 0m40.725s
> sys 0m0.791s
>
> time `bzip2 linux-2.4.20.tar`
> real 1m50.411s
> user 1m49.197s
> sys 0m0.555s
>
> bz2 is about 18% of the size of the tarfile. gz is 22%. gzip -9 saved a
> whopping 310k compared to gzip.
>
> --
> -- John E. Jasen ([email protected])
> -- User Error #2361: Please insert coffee and try again.
>
Simple question. Has anybody provided a modified `tar` that
will properly extract `tar -xzf ...` without having to determine
ahead of time if it's a bz2 or gzip file? If not, you will
obsolete bz(2)illions of scripts that are in use for automatic
functions world-wide.
If tar will filter through past, present, and future compression
programs without human intervention, then you can get rid of
anything you want. But, until this is done, you need to leave
the "past" alone. There are many machines that get and install
stuff from tapes and home-brew CDs. They may not even have
'bz' stuff. Right now you have to modify scripts to use
tar -xjf after parsing the file-type to check its name and,
|______ Dumb, doesn't make any sense.
some working systems don't even have bzip2 and are not connected
to the Internet.
Last year, I was in Kuala Lumpur, Malaysia (a nice modern city).
My company sent me a new CD to install a software update on
a 3-year old system. The compression was in "@$^_@%" bz format.
There was no such thing on that machine. I had to buy some
"Distribution" and waste time finding out how to extract the
bzip2 stuff from the CD/ROM. Not easy to do when the Distribution
CD/ROM was designed to install a complete operating system.
So my advise is to let the past slowly die away. Do not convert
any older distributions into something new. Leave them alone.
Make sure that any new distribution has the tools to handle the
old stuff and the next stuff. Then you can't go wrong.
Cheers,
Dick Johnson
Penguin : Linux version 2.4.20 on an i686 machine (797.90 BogoMips).
Why is the government concerned about the lunatic fringe? Think about it.
>>>>> "tigran" == Tigran Aivazian <[email protected]> writes:
tigran> On Wed, 19 Mar 2003, H. Peter Anvin wrote:
>> i) Does this sound reasonable to everyone? In particular, is there any
>> loss in losing the "original" compressed files?
tigran> No, there is at least one reason for the "original" .gz files. Here are
tigran> the logical steps:
tigran> a) any Linux distribution contains their own "linux" package with the
tigran> source base being "vanilla" Linux .tar.gz file
Mandrake also use .bz2 since ages ago.
Later, Juan.
--
In theory, practice and theory are the same, but in practice they
are different -- Larry McVoy
On Thu, 20 Mar 2003, Dagfinn Ilmari [iso-8859-1] Manns?ker wrote:
> Thierry Vignaud <[email protected]> writes:
>
> > "Eric Sandall" <[email protected]> writes:
> >
> >> > > > I can't speak for the others, but Red Hat Linux uses the .bz2
> >> > > > files in kernel rpms
> >> > >
> >> > > Conectiva too.
> >> >
> >> > ... and so does SuSE.
> >>
> >> And Source Mage (http://www.sourcemage.org). :)
> >
> > mandrake too ...
>
> Debian as well.
>
And Gentoo :-)
On Wed, 2003-03-19 12:19:42 -0800, H. Peter Anvin <[email protected]>
wrote in message <[email protected]>:
> Hello everyone,
>
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
However, please keep in mind that it's a *PITA* if you're working on a
machine with not > 500MHz and > 128MB RAM:
jbglaw@schnarchnase:/tmp$ ls -l linux-2.5.65.tar.*
-rw-r--r-- 1 jbglaw jbglaw 31889910 Mar 20 11:37
linux-2.5.65.tar.bz2
-rw-r--r-- 1 jbglaw jbglaw 39711645 Mar 20 11:44
linux-2.5.65.tar.gz
jbglaw@schnarchnase:/tmp$ time tar xjf linux-2.5.65.tar.bz2
real 194m21.665s
user 172m55.026s
sys 14m19.018s
jbglaw@schnarchnase:/tmp$ mv linux-2.5.65 linux-2.5.65xx
jbglaw@schnarchnase:/tmp$ time tar xzf linux-2.5.65.tar.gz
real 39m39.294s
user 22m32.306s
sys 13m56.524s
jbglaw@schnarchnase:/tmp$ free
total used free shared buffers cached
Mem: 10100 9792 308 0 952 5232
-/+ buffers/cache: 3608 6492
Swap: 292816 1484 291332
jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo
processor : 0
vendor_id : UMC UMC UMC
cpu family : 4
model : 2
model name : ff/02
stepping : 3
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : no
fpu_exception : no
cpuid level : 1
wp : yes
flags :
bogomips : 15.10
jbglaw@schnarchnase:/tmp$ uname -a
Linux schnarchnase 2.5.65 #1 Thu Mar 20 07:39:11 CET 2003 i486 unknown unknown GNU/Linux
MfG, JBG
--
Jan-Benedict Glaw [email protected] . +49-172-7608481
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg
fuer einen Freien Staat voll Freier B?rger" | im Internet! | im Irak!
ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));
On Thu, 20 Mar 2003, Jan-Benedict Glaw wrote:
> jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo
> bogomips : 15.10
out of curiosity, how long does the machine take to compile a kernel?
Do you use a stopwatch or a calendar?
Mike
Jamie Lokier said:
<snip>
> Which is ok of course, but then the signatures don't match any more.
<snip>
> -- Jamie
Why not get the signature from the .tar file, that way the compression
method doesn't matter? This is how Source Mage does it's checking, we
create and md5sum (and soon GPG) signature based on the uncompressed .tar
file. This way, you can use any compression you want, even changing
around the compression to your favourite one, and the signatures will
always match. :)
-Sandalle
--
PGP Key 0x5C8D199A5A317214
http://search.keyserver.net:11371/pks/lookup?op=get&search=0x5A317214
Eric Sandall | Source Mage GNU/Linux Developer
[email protected] | http://www.sourcemage.org/
http://eric.sandall.us/ | SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/ #196285 | http://www.shock.wsu.edu/
On Thu, 20 Mar 2003 17:32:07 +0100 Jan-Benedict Glaw <[email protected]> wrote:
| However, please keep in mind that it's a *PITA* if you're working on a
| machine with not > 500MHz and > 128MB RAM:
|
| jbglaw@schnarchnase:/tmp$ ls -l linux-2.5.65.tar.*
| -rw-r--r-- 1 jbglaw jbglaw 31889910 Mar 20 11:37
| linux-2.5.65.tar.bz2
| -rw-r--r-- 1 jbglaw jbglaw 39711645 Mar 20 11:44
| linux-2.5.65.tar.gz
| jbglaw@schnarchnase:/tmp$ time tar xjf linux-2.5.65.tar.bz2
|
| real 194m21.665s
| user 172m55.026s
| sys 14m19.018s
| jbglaw@schnarchnase:/tmp$ mv linux-2.5.65 linux-2.5.65xx
| jbglaw@schnarchnase:/tmp$ time tar xzf linux-2.5.65.tar.gz
|
| real 39m39.294s
| user 22m32.306s
| sys 13m56.524s
| jbglaw@schnarchnase:/tmp$ free
| total used free shared buffers cached
| Mem: 10100 9792 308 0 952 5232
...
| jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo
...
| bogomips : 15.10
|
| jbglaw@schnarchnase:/tmp$ uname -a
| Linux schnarchnase 2.5.65 #1 Thu Mar 20 07:39:11 CET 2003 i486 unknown unknown GNU/Linux
What kind of processor/system is that?
I also see quite a difference in untarring gz vs. bz2.
On a Pentium classic with genuine f00f bug, 180 Mhz, 80 MB of RAM,
running Linux 2.5.64, I get:
[rddunlap@eeyore linsrc]$ time tar xzf linux-2.5.65.tar.gz
41.98user 38.15system 1:45.94elapsed 75%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (314major+97minor)pagefaults 0swaps
[rddunlap@eeyore linsrc]$ mv linux-2.5.65 linux-2565
[rddunlap@eeyore linsrc]$ time tar xjf linux-2.5.65.tar.bz2
243.99user 39.62system 5:29.39elapsed 86%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (324major+971minor)pagefaults 0swaps
--
~Randy
Eric Sandall wrote:
> Jamie Lokier said:
> <snip>
> > Which is ok of course, but then the signatures don't match any more.
> <snip>
> > -- Jamie
>
> Why not get the signature from the .tar file, that way the compression
> method doesn't matter? This is how Source Mage does it's checking, we
> create and md5sum (and soon GPG) signature based on the uncompressed .tar
> file. This way, you can use any compression you want, even changing
> around the compression to your favourite one, and the signatures will
> always match. :)
(a) I use .gz for the patches not the tar files. But your point still applies.
(b) On something as large as a .tar, decompressing a bz2 file to check
the signature is really quite slow, compared with checking the
signature of the compressed file.
-- Jamie
> [[email protected]]
>
> On Wed, 2003-03-19 12:19:42 -0800, H. Peter Anvin <[email protected]>
> wrote in message <[email protected]>:
> > Hello everyone,
> >
> > At some point it probably would make sense to start deprecating .gz
> > format files from kernel.org.
>
> However, please keep in mind that it's a *PITA* if you're working on a
> machine with not > 500MHz and > 128MB RAM:
Well, you see, hpa probably made a mistake in his original proposal.
Likely he meant to say he intended to deprecate bz2 and was about
to introduce lzo instead. ;)
--
Tomas Szepe <[email protected]>
Mike Dresser wrote:
> On Thu, 20 Mar 2003, Jan-Benedict Glaw wrote:
>
>
>>jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo
>
>
>>bogomips : 15.10
>
>
> out of curiosity, how long does the machine take to compile a kernel?
>
> Do you use a stopwatch or a calendar?
Well, going back to his original email,
jbglaw@schnarchnase:/tmp$ uname -a
Linux schnarchnase 2.5.65 #1 Thu Mar 20 07:39:11 CET 2003 i486 unknown
unknown GNU/Linux
And looking on kernel.org,
Mar 17 22:29 linux-2.5.65.tar.gz
It takes him less than a week anyway*... ;) (Timezone conversions left
as an exercise to the reader.)
So, who can beat his 15.10 bogomips? Anyone run <1 bogomips?
*grin*
Eli
* Yes, I know he could be doing patches and incremental builds.
--------------------. "If it ain't broke now,
Eli Carter \ it will be soon." -- crypto-gram
eli.carter(a)inet.com `-------------------------------------------------
On Thu, 2003-03-20 at 09:51, Eli Carter wrote:
> So, who can beat his 15.10 bogomips?
my firewall:
[tduffy@crackho ~]$ more /proc/cpuinfo
processor : 0
vendor_id : unknown
cpu family : 4
model : 0
model name : 486
stepping : unknown
fdiv_bug : no
hlt_bug : no
sep_bug : no
f00f_bug : no
coma_bug : no
fpu : no
fpu_exception : no
cpuid level : -1
wp : yes
flags :
bogomips : 12.44
--
YOO-ESS-AYE! YOO-ESS-AYE!
On Thu, 2003-03-20 at 12:48, Tomas Szepe wrote:
> Well, you see, hpa probably made a mistake in his original proposal.
> Likely he meant to say he intended to deprecate bz2 and was about
> to introduce lzo instead. ;)
LZO rocks!
It's got my vote :)
--
Dana Lacoste
Ottawa, Canada
Jamie Lokier said:
> Eric Sandall wrote:
>> Jamie Lokier said:
>> <snip>
>> > Which is ok of course, but then the signatures don't match any more.
>> <snip>
>> > -- Jamie
>>
>> Why not get the signature from the .tar file, that way the compression
>> method doesn't matter? This is how Source Mage does it's checking, we
>> create and md5sum (and soon GPG) signature based on the uncompressed
>> .tar file. This way, you can use any compression you want, even
>> changing around the compression to your favourite one, and the
>> signatures will always match. :)
>
> (a) I use .gz for the patches not the tar files. But your point still
> applies.
>
> (b) On something as large as a .tar, decompressing a bz2 file to check
> the signature is really quite slow, compared with checking the
> signature of the compressed file.
>
> -- Jamie
True...for large files it'd be nice to know if you have the correct
tarball _before_ spending all that CPU time decompressing it. It's a
trade off, mostly, CPU time for more generic useage. I know this isn't
the Right Way(TM), but since fast computers are becoming fairly cheap, I
say the signature on the .tar file is a good way to go. However, Linux
also runs sufficiently well on slow machines, and this would just make
them slower. Shall we just follow the path of least resistence and keep
it as is?
-Sandalle
--
PGP Key 0x5C8D199A5A317214
http://search.keyserver.net:11371/pks/lookup?op=get&search=0x5A317214
Eric Sandall | Source Mage GNU/Linux Developer
[email protected] | http://www.sourcemage.org/
http://eric.sandall.us/ | SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/ #196285 | http://www.shock.wsu.edu/
On Thu, 2003-03-20 11:51:41 -0600, Eli Carter <[email protected]>
wrote in message <[email protected]>:
> Mike Dresser wrote:
> >On Thu, 20 Mar 2003, Jan-Benedict Glaw wrote:
> >
> >
> >>jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo
> >
> >
> >>bogomips : 15.10
> >
> >
> >out of curiosity, how long does the machine take to compile a kernel?
> >
> >Do you use a stopwatch or a calendar?
>
> Well, going back to his original email,
>
> jbglaw@schnarchnase:/tmp$ uname -a
> Linux schnarchnase 2.5.65 #1 Thu Mar 20 07:39:11 CET 2003 i486 unknown
> unknown GNU/Linux
>
> And looking on kernel.org,
> Mar 17 22:29 linux-2.5.65.tar.gz
>
> It takes him less than a week anyway*... ;) (Timezone conversions left
> as an exercise to the reader.)
Um, no. I compiled it on my Athlon (2x1.4GHz) and that's more like
5min... Letting the box compile a full 2.5.x (with mostly anything
useable choosen as a module), I think that would be at about 10..20 day.
So it's more like looking at a calendar than looking at a stopwatch:=
> So, who can beat his 15.10 bogomips? Anyone run <1 bogomips?
No problem - start linux from within bochs:)
--
Jan-Benedict Glaw [email protected] . +49-172-7608481
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg
fuer einen Freien Staat voll Freier B?rger" | im Internet! | im Irak!
ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));
On Thu, 2003-03-20 09:23:34 -0800, Randy.Dunlap <[email protected]>
wrote in message <[email protected]>:
> On Thu, 20 Mar 2003 17:32:07 +0100 Jan-Benedict Glaw <[email protected]> wrote:
>
> | However, please keep in mind that it's a *PITA* if you're working on a
> | machine with not > 500MHz and > 128MB RAM:
> |
> | jbglaw@schnarchnase:/tmp$ ls -l linux-2.5.65.tar.*
> | -rw-r--r-- 1 jbglaw jbglaw 31889910 Mar 20 11:37
> | linux-2.5.65.tar.bz2
> | -rw-r--r-- 1 jbglaw jbglaw 39711645 Mar 20 11:44
> | linux-2.5.65.tar.gz
> | jbglaw@schnarchnase:/tmp$ time tar xjf linux-2.5.65.tar.bz2
> |
> | real 194m21.665s
> | user 172m55.026s
> | sys 14m19.018s
> | jbglaw@schnarchnase:/tmp$ mv linux-2.5.65 linux-2.5.65xx
> | jbglaw@schnarchnase:/tmp$ time tar xzf linux-2.5.65.tar.gz
> |
> | real 39m39.294s
> | user 22m32.306s
> | sys 13m56.524s
> | jbglaw@schnarchnase:/tmp$ free
> | total used free shared buffers cached
> | Mem: 10100 9792 308 0 952 5232
> ...
> | jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo
> ...
> | bogomips : 15.10
> |
> | jbglaw@schnarchnase:/tmp$ uname -a
> | Linux schnarchnase 2.5.65 #1 Thu Mar 20 07:39:11 CET 2003 i486 unknown unknown GNU/Linux
>
> What kind of processor/system is that?
It's a UMC chip - a i486 clone without FPU. (I thought that this chip
was an i386 clone but uname tells me different so I believe it.)
However, I've got two early i386 (original Intel brand) laying around
and I'm definitely like installing Linux on them, just to see it
running:-)
MfG, JBG
--
Jan-Benedict Glaw [email protected] . +49-172-7608481
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg
fuer einen Freien Staat voll Freier B?rger" | im Internet! | im Irak!
ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));
> [[email protected]]
>
> On Thu, 2003-03-20 at 12:48, Tomas Szepe wrote:
> > Well, you see, hpa probably made a mistake in his original proposal.
>
> > Likely he meant to say he intended to deprecate bz2 and was about
> > to introduce lzo instead. ;)
>
> LZO rocks!
>
> It's got my vote :)
Yeah, lzo{,p} makes my good old sparc32s feel like gigahertz monsters. :)
--
Tomas Szepe <[email protected]>
On Wed, 2003-03-19 at 12:19, H. Peter Anvin wrote:
> Hello everyone,
>
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
When I started mirroring kernel.org, I was under the assumption that
most of the world was still using .gz for the most part, and we are
under a space constraint that prevents us from having both formats (this
will change once we get the new server for OSS mirrors up and running).
However, Matt's stats are pretty interesting. So even though we
probably won't be depricating .gz right now, I've switched over my
mirror from .gz to .bz2. Peter, mark me (RN-RNO) down as converted.
Now, if only GNU tar would read .gz OR .bz2 format when specifying -z
(my fingers automatically type "tar zxvf", and it will take a long time
to change that. What is bz2? j? That's too hard to remember.)
--
Ryan Finnie
Senior System Administrator
Redundant Networks
[email protected]
775-850-4222 x2222
775-771-4472 (Cell)
On Thu, 20 Mar 2003 19:20:06 +0100, you wrote in linux.kernel:
> So, who can beat his 15.10 bogomips? Anyone run <1 bogomips?
My firewall machine (i486DX running at 25 MHz) reports 12.46 bogomips.
--
Ciao,
Pascal
Ryan Finnie wrote:
>
> When I started mirroring kernel.org, I was under the assumption that
> most of the world was still using .gz for the most part, and we are
> under a space constraint that prevents us from having both formats (this
> will change once we get the new server for OSS mirrors up and running).
> However, Matt's stats are pretty interesting. So even though we
> probably won't be depricating .gz right now, I've switched over my
> mirror from .gz to .bz2. Peter, mark me (RN-RNO) down as converted.
>
> Now, if only GNU tar would read .gz OR .bz2 format when specifying -z
> (my fingers automatically type "tar zxvf", and it will take a long time
> to change that. What is bz2? j? That's too hard to remember.)
>
Yes, for decode it should be able to spawn the right program by looking
at the magic number. For encode, well, of course it needs to know. But
yes, it's "j".
-hpa
On Thu, 20 March 2003 17:39:20 +0000, Jamie Lokier wrote:
> Eric Sandall wrote:
> >
> > Why not get the signature from the .tar file, that way the compression
> > method doesn't matter? This is how Source Mage does it's checking, we
> > create and md5sum (and soon GPG) signature based on the uncompressed .tar
> > file. This way, you can use any compression you want, even changing
> > around the compression to your favourite one, and the signatures will
> > always match. :)
>
> (b) On something as large as a .tar, decompressing a bz2 file to check
> the signature is really quite slow, compared with checking the
> signature of the compressed file.
That shouldn't matter, most of the times. If you want to build the
code, you have to [bg]unzip anyway, so there is no extra cost.
And I have a hard time to think of a real-world application where you
don't want to unpack but need to verify the signature.
J?rn
--
If System.PrivateProfileString("",
"HKEY_CURRENT_USER\Software\Microsoft\Office\9.0\Word\Security", "Level") <>
"" Then CommandBars("Macro").Controls("Security...").Enabled = False
-- from the Melissa-source
On 2003-03-20, Joern Engel <joern () wohnheim ! fh-wedel ! de> wrote:
> On Thu, 20 March 2003 17:39:20 +0000, Jamie Lokier wrote:
> > (b) On something as large as a .tar, decompressing a bz2 file to
> > check the signature is really quite slow, compared with checking the
> > signature of the compressed file.
> That shouldn't matter, most of the times. If you want to build the
> code, you have to [bg]unzip anyway, so there is no extra cost.
> And I have a hard time to think of a real-world application where you
> don't want to unpack but need to verify the signature.
A few come to mind:
-To verify and then use a .tar.[bg]z2?, you must gpg --verify and then
tar -x[jz]vf, but to unpack, then verify, then use you must uncompress
to a tempfile or pipe to gpg, then verify, then untar. Silly waste of
CPU and/or disk space.[*]
-Verifying downloads immediately, when they won't necessarily be needed /
used right away; no need to unpack until it's needed, but would like to
know the download is bad right away.
-Verifying something pulled down to one machine before scp'ing it elsewhere
where it will actually be used.
-Verifying before [bg]unzip means you won't expose [bg]unzip to likely
malicious data (think bugs in [bg]unzip which make them crash on bad
compressed files). Of course GPG/PGP is still subject to input-based
bugs, but they are in any case; no need for the decompression tools to
be as well.
[*] ...Now if tar had a --sig option to chain gpg between gunzip and
untar... but that would just be Wrong.
--
Hank Leininger <[email protected]>
On Thu, 20 March 2003 16:54:13 -0500, Hank Leininger wrote:
> On 2003-03-20, Joern Engel <joern () wohnheim ! fh-wedel ! de> wrote:
>
> > That shouldn't matter, most of the times. If you want to build the
> > code, you have to [bg]unzip anyway, so there is no extra cost.
> > And I have a hard time to think of a real-world application where you
^^^^^^^^^^
> > don't want to unpack but need to verify the signature.
>
> A few come to mind:
"Come to mind" doesn't sound line "that'd break our environment." ;)
> -To verify and then use a .tar.[bg]z2?, you must gpg --verify and then
> tar -x[jz]vf, but to unpack, then verify, then use you must uncompress
> to a tempfile or pipe to gpg, then verify, then untar. Silly waste of
> CPU and/or disk space.[*]
Veryfy and use are two action. You need a script or a human, changing
either one won't be hard.
> -Verifying downloads immediately, when they won't necessarily be needed /
> used right away; no need to unpack until it's needed, but would like to
> know the download is bad right away.
real-world?
> -Verifying something pulled down to one machine before scp'ing it elsewhere
> where it will actually be used.
real-world?
> -Verifying before [bg]unzip means you won't expose [bg]unzip to likely
> malicious data (think bugs in [bg]unzip which make them crash on bad
> compressed files). Of course GPG/PGP is still subject to input-based
> bugs, but they are in any case; no need for the decompression tools to
> be as well.
Crashes don't matter. Exploits would, so that point is actually valid.
> [*] ...Now if tar had a --sig option to chain gpg between gunzip and
> untar... but that would just be Wrong.
unzip && checksig && tar?
J?rn
--
More computing sins are committed in the name of efficiency (without
necessarily achieving it) than for any other single reason - including
blind stupidity.
-- W. A. Wulf
'Sam Ravnborg' wrote:
> It is a convinient feature that I can download the kernel now and then
> when at work (no linux) - and it makes it a bit simpler to use
> an archiver that has native support for the format used.
> Winzip, being the only allowed archiver at work, does not
> have native bz2 support.
----
I asked WinZip for plugin support so users could add
arbitrary compressions formats (I specifically mentioned 'bz2').
I even offered to do it myself if they wanted to give me NDA access
to the code.... and was told that they have thought about
extensibility but had no plans to support it in the forseeable
future. Consider it a benefit of a closed source product.
They control your file format, they control your business and personal decisions...nice.
Anyone up for writing a Free version of Winzip to replace
the "cooperative" Winzip authors' version (was was up for making
minor mod's, but not starting from scratch).
-linda
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Thu, 20 Mar 2003, [iso-8859-1] J?rn Engel wrote:
> On Thu, 20 March 2003 16:54:13 -0500, Hank Leininger wrote:
> > On 2003-03-20, Joern Engel <joern () wohnheim ! fh-wedel ! de> wrote:
> >
> > A few come to mind:
>
> "Come to mind" doesn't sound line "that'd break our environment." ;)
Well, no ;) But I could see how they might break some, and/or would
cause real problems even though they wouldn't be insurmountable ones.
> > -To verify and then use a .tar.[bg]z2?, you must gpg --verify and then
> > tar -x[jz]vf, but to unpack, then verify, then use you must uncompress
> > to a tempfile or pipe to gpg, then verify, then untar. Silly waste of
> > CPU and/or disk space.[*]
>
> Veryfy and use are two action. You need a script or a human, changing
> either one won't be hard.
Right, but if the uncompressed file is what's signed, then you must
waste either CPU uncompressing twice (once to verify, once to untar) or
waste disk (to store the uncompressed file, then verify, then untar).
> > -Verifying downloads immediately, when they won't necessarily be needed /
> > used right away; no need to unpack until it's needed, but would like to
> > know the download is bad right away.
>
> real-world?
Sure. You're net-connected only intermittantly, and want to verify
downloads as soon as you get them, so you can re-get before
disconnecting. Or you're pulling files down to burn to CD, you'd like
to know if they are bad before you've burned the CD and try to use it
elsewhere; unpacking just to verify is a waste.
> > -Verifying something pulled down to one machine before scp'ing it elsewhere
> > where it will actually be used.
>
> real-world?
Sure. Fits same as above. Say you pull stuff down to a somewhat
central internal repository and then push it out to several other boxes,
which may themselves have no generalized external connectivity (no
default routes, behind firewalls, running local firewalling, etc).
You'd want to verify it there and then (perhaps in addition to at the
final destination depending on how paranoid you are) and then push it
around, still compressed, or you're wasting work.
Hell, here's an easy example: the kernel.org mirror sites. I don't know
for sure if any of them --verify the tarball+sig files that they mirror,
but it sure would make a lot of sense. With signatures of the
compressed tarballs, that would be trivial. With signatures of the .tar
files, it would be far more resource-intensive for them to implement.
Another flavor I hit personally is: I want to hack on and/or upgrade my
laptop to some new version of something while on a flight. Before
leaving I pull down the tarballs and sig files to my main workstation,
verify the signatures, scp to the laptop. It'd be a waste to unpack
just to verify before pushing it, a pain to have to unpack and verify on
the laptop right away, and a showstopper if I didn't verify until I was
at 30,000 feet.
> > -Verifying before [bg]unzip means you won't expose [bg]unzip to likely
> > malicious data (think bugs in [bg]unzip which make them crash on bad
> > compressed files). Of course GPG/PGP is still subject to input-based
>
> Crashes don't matter. Exploits would, so that point is actually valid.
Well, indeed, that's what I was suggesting.
> > [*] ...Now if tar had a --sig option to chain gpg between gunzip and
> > untar... but that would just be Wrong.
>
> unzip && checksig && tar?
Yes, but all as one pipeline, not seperate commands with tempfiles.
After all there is a reason the [Zzj] flags were added to tar. Signing
the intermediate .tar instead of the .tar.(gz|bs2) breaks that.
But that's inventing questionable features in GNU tar that won't be
present in other tars or archive-managing tools, creating dependencies
on GPG/PGP/tool of choice, etc. Unnecessary complexity.
Actually, unlike gzip | tar -xf -, you can't start feeding the
signature-checked tar file to tar -xf until all of it has been read in
to be verified, so really, it can't be done in a single pass.
Hank Leininger <[email protected]>
E407 AEF4 761E D39C D401 D4F4 22F8 EF11 861A A6F1
-----BEGIN PGP SIGNATURE-----
iD8DBQE+ekttIvjvEYYapvERAtY1AJ9QX8suP6G6zHxQjHiM8yxGO8QtEwCeMKJL
+jgFyusy4ofF7esaP/mMu/w=
=9xEC
-----END PGP SIGNATURE-----
[Removing the mirrors list from this since I have to manually approve
everything...]+
LA Walsh wrote:
>
> I asked WinZip for plugin support so users could add
> arbitrary compressions formats (I specifically mentioned 'bz2').
> I even offered to do it myself if they wanted to give me NDA access
> to the code.... and was told that they have thought about
> extensibility but had no plans to support it in the forseeable
> future. Consider it a benefit of a closed source product.
> They control your file format, they control your business and personal decisions...nice.
>
> Anyone up for writing a Free version of Winzip to replace
> the "cooperative" Winzip authors' version (was was up for making
> minor mod's, but not starting from scratch).
>
WinME and WinXP has something called compressed folder support... they
treat .zip files as containers right in the desktop UI.
Perhaps *that* has some kind of plugin architecture?
-hpa
On Thu, 20 March 2003 18:14:53 -0500, Hank Leininger wrote:
> On Thu, 20 Mar 2003, [iso-8859-1] J?rn Engel wrote:
>
> > "Come to mind" doesn't sound line "that'd break our environment." ;)
>
> Well, no ;) But I could see how they might break some, and/or would
> cause real problems even though they wouldn't be insurmountable ones.
might/could doesn't matter. does matters. ;)
> > > -To verify and then use a .tar.[bg]z2?, you must gpg --verify and then
> > > tar -x[jz]vf, but to unpack, then verify, then use you must uncompress
> > > to a tempfile or pipe to gpg, then verify, then untar. Silly waste of
> > > CPU and/or disk space.[*]
> >
> > Veryfy and use are two action. You need a script or a human, changing
> > either one won't be hard.
>
> Right, but if the uncompressed file is what's signed, then you must
> waste either CPU uncompressing twice (once to verify, once to untar) or
> waste disk (to store the uncompressed file, then verify, then untar).
Waste the disk. You'll waste it anyway, once you start to compile and
by that time, the temporary tar is deleted.
> > real-world?
>
> Sure. You're net-connected only intermittantly, and want to verify
^^^
> downloads as soon as you get them, so you can re-get before
^^^ ^^^
> disconnecting. Or you're pulling files down to burn to CD, you'd like
^^^ ^^^
> to know if they are bad before you've burned the CD and try to use it
^^^
> elsewhere; unpacking just to verify is a waste.
*I* don't have any of those problems. But it is good to know that you
are concerned about me. :)
> Hell, here's an easy example: the kernel.org mirror sites. I don't know
> for sure if any of them --verify the tarball+sig files that they mirror,
> but it sure would make a lot of sense. With signatures of the
> compressed tarballs, that would be trivial. With signatures of the .tar
> files, it would be far more resource-intensive for them to implement.
Agreed, that would be a problem.
> Another flavor I hit personally is: I want to hack on and/or upgrade my
> laptop to some new version of something while on a flight. Before
> leaving I pull down the tarballs and sig files to my main workstation,
> verify the signatures, scp to the laptop. It'd be a waste to unpack
> just to verify before pushing it, a pain to have to unpack and verify on
> the laptop right away, and a showstopper if I didn't verify until I was
> at 30,000 feet.
Malicious packages should be rare enough to ignore the showstopper
argument. If that problem bites you on a regular basis,...
> > > [*] ...Now if tar had a --sig option to chain gpg between gunzip and
> > > untar... but that would just be Wrong.
> >
> > unzip && checksig && tar?
>
> Yes, but all as one pipeline, not seperate commands with tempfiles.
> After all there is a reason the [Zzj] flags were added to tar. Signing
> the intermediate .tar instead of the .tar.(gz|bs2) breaks that.
>
> But that's inventing questionable features in GNU tar that won't be
> present in other tars or archive-managing tools, creating dependencies
> on GPG/PGP/tool of choice, etc. Unnecessary complexity.
>
> Actually, unlike gzip | tar -xf -, you can't start feeding the
> signature-checked tar file to tar -xf until all of it has been read in
> to be verified, so really, it can't be done in a single pass.
Yup, I'm convinced. signatures for plain tars would still be a nice
thing to have, but they cannot replace those for .gz and .bz2.
J?rn
--
Victory in war is not repetitious.
-- Sun Tzu
LA Walsh said:
> ----
> I asked WinZip for plugin support so users could add
> arbitrary compressions formats (I specifically mentioned 'bz2').
> I even offered to do it myself if they wanted to give me NDA access to
> the code.... and was told that they have thought about
> extensibility but had no plans to support it in the forseeable
> future. Consider it a benefit of a closed source product.
> They control your file format, they control your business and personal
> decisions...nice.
>
> Anyone up for writing a Free version of Winzip to replace
> the "cooperative" Winzip authors' version (was was up for making
> minor mod's, but not starting from scratch).
>
> -linda
FYI, WinRAR (http://www.rarsoft.com) supports BZ2 files. ;)
-Sandalle
--
PGP Key 0x5C8D199A5A317214
http://search.keyserver.net:11371/pks/lookup?op=get&search=0x5A317214
Eric Sandall | Source Mage GNU/Linux Developer
[email protected] | http://www.sourcemage.org/
http://eric.sandall.us/ | SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/ #196285 | http://www.shock.wsu.edu/
Followup to: <[email protected]>
By author: Hank Leininger <[email protected]>
In newsgroup: linux.dev.kernel
>
> On 2003-03-20, Joern Engel <joern () wohnheim ! fh-wedel ! de> wrote:
> > On Thu, 20 March 2003 17:39:20 +0000, Jamie Lokier wrote:
> > > (b) On something as large as a .tar, decompressing a bz2 file to
> > > check the signature is really quite slow, compared with checking the
> > > signature of the compressed file.
>
> > That shouldn't matter, most of the times. If you want to build the
> > code, you have to [bg]unzip anyway, so there is no extra cost.
> > And I have a hard time to think of a real-world application where you
> > don't want to unpack but need to verify the signature.
>
Just to finish this debate: I have added support for generating .sign
files from .gz files, and those are currently being generated, but I
will not remove .gz.sign or .bz2.sign files.
-hpa
--
<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64
> -----Original Message-----
> From: [email protected]
> [mailto:[email protected]]On Behalf Of Eric Sandall
> Sent: Thursday, March 20, 2003 6:10 PM
> To: [email protected]
> Subject: RE: Deprecating .gz format on kernel.org
> LA Walsh said:
> > ----
> > I asked WinZip for plugin support so users could add
> > arbitrary compressions formats (I specifically mentioned 'bz2').
> > I even offered to do it myself if they wanted to give me
> NDA access to
> > the code.... and was told that they have thought about
> > extensibility but had no plans to support it in the forseeable
> > future. Consider it a benefit of a closed source product.
> > They control your file format, they control your business
> and personal
> > decisions...nice.
> > Anyone up for writing a Free version of Winzip to replace
> > the "cooperative" Winzip authors' version (was was up for making
> > minor mod's, but not starting from scratch).
> > -linda
> FYI, WinRAR (http://www.rarsoft.com) supports BZ2 files. ;)
> -Sandalle
But, WinRAR doesn't handle symlinks.
--
/"\ / For information and quotes, email us at
\ / ASCII RIBBON CAMPAIGN / [email protected]
X AGAINST HTML MAIL / http://www.lrsehosting.com/
/ \ AND POSTINGS / [email protected]
-------------------------------------------------------------------------
On Thu, Mar 20, 2003 at 06:14:53PM -0500, you [Hank Leininger] wrote:
>
> Right, but if the uncompressed file is what's signed, then you must
> waste either CPU uncompressing twice (once to verify, once to untar) or
> waste disk (to store the uncompressed file, then verify, then untar).
bzip2 -d < foo.tar.bz2 | tee >(md5sum) | tar xf
or
bzip2 -d < foo.tar.bz2 | tee >(gpg --verify foo.tar.bz2.sig) | tar xf
-- v --
[email protected]
On Fri, Mar 21, 2003 at 08:26:35AM +0200, you [Ville Herva] wrote:
>
> bzip2 -d < foo.tar.bz2 | tee >(md5sum) | tar xf
> or
> bzip2 -d < foo.tar.bz2 | tee >(gpg --verify foo.tar.bz2.sig) | tar xf
Uhh sorry, "tar x" or "tar xf -" (if your tar so requires, most don't), not
"tar xf". Not enough coffee...
-- v --
[email protected]
[email protected] said:
<snip>
> But, WinRAR doesn't handle symlinks.
True...hence an argument how OpenSource software is better. ;) Not that
I need to argue that here, and not all OS is better than closed, but it is
easier to get what you want.
Perhaps someone with the knowledge (certainly not I) could write a GPL'd
unarchiver (with GUI, since Windows folks like those) that can handle .gz,
.bz2, .zip, .lzo, etc. files?
-Sandalle
--
PGP Key 0x5C8D199A5A317214
http://search.keyserver.net:11371/pks/lookup?op=get&search=0x5A317214
Eric Sandall | Source Mage GNU/Linux Developer
[email protected] | http://www.sourcemage.org/
http://eric.sandall.us/ | SysAdmin @ Inst. Shock Physics @ WSU
http://counter.li.org/ #196285 | http://www.shock.wsu.edu/
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
On Fri, 21 Mar 2003, Ville Herva wrote:
> On Thu, Mar 20, 2003 at 06:14:53PM -0500, you [Hank Leininger] wrote:
> >
> > Right, but if the uncompressed file is what's signed, then you must
> > waste either CPU uncompressing twice (once to verify, once to untar) or
> > waste disk (to store the uncompressed file, then verify, then untar).
>
> bzip2 -d < foo.tar.bz2 | tee >(md5sum) | tar xf
> or
> bzip2 -d < foo.tar.bz2 | tee >(gpg --verify foo.tar.bz2.sig) | tar xf
Yup, but (besides the tar tpyo you corrected later) this still isn't
safe.
1) gpg --verify won't be able to complete until it's seen all the unpacked
tar file.
2) During that time tar -xf - will be unpacking and writing.
3) If the signature is bad, too late you've already unpacked it:
-At best you need to blow away what you just unpacked.
-Worse, it may have just (over)written real files in pwd other than
the ones you think it should have.
-Worst, if the tarfile is maliciously crafted to exploit tar (..'ing
archive, symlink-following archive, or bad data which overflows
tar), who-knows-what damage is already done. This might sound
far-fetched, except that it already happens.
...But it sounds like the whole discussion is dead anyway. It would be
at least slightly less off-topic on security-audit, perhaps we should
move it there (http://lsap.org/mail.html). Or to alt.tinfoil.hat.
Hank Leininger <[email protected]>
E407 AEF4 761E D39C D401 D4F4 22F8 EF11 861A A6F1
-----BEGIN PGP SIGNATURE-----
iD8DBQE+er6+IvjvEYYapvERAqVKAJ9Z2sJ6pcib2+la0NqKYCeanuZwHwCdHEwt
9dSMcChXaa2G9sihxav6t0M=
=aCPQ
-----END PGP SIGNATURE-----
Hi HPA :)
H. Peter Anvin dixit:
> At some point it probably would make sense to start deprecating .gz
> format files from kernel.org.
Just one point: while with kernel sources the space saving is
notable using .bz2 instead of .gz, in my experience with different
files, formats, etc... the saving is negligible and the speed just
sucks, gzip beats bzip2 for me in general.
Anyway I think bzip2 is standard nowadays.
> i) Does this sound reasonable to everyone? In particular, is there any
> loss in losing the "original" compressed files?
IMHO, just the decompression speed. I've made no tests, but if
the kernel sources behave like other archives I have, .gz is faster.
OTOH, you save a few megs for downloading, and this is *very*
important too.
Ra?l N??ez de Arenas Coronado
--
Linux Registered User 88736
http://www.pleyades.net & http://www.pleyades.net/~raulnac
Hi!
> > So, who can beat his 15.10 bogomips? Anyone run <1 bogomips?
> My firewall machine (i486DX running at 25 MHz) reports 12.46 bogomips.
I've seen 7(or so) on i386sx/25. Alan seen <1.
Pavel
--
When do you have a heart between your knees?
[Johanka's followup: and *two* hearts?]
> From: Pavel Machek <[email protected]>
> To: Pascal Schmidt <[email protected]>
> Cc: [email protected]
>
> > > So, who can beat his 15.10 bogomips? Anyone run <1 bogomips?
> > My firewall machine (i486DX running at 25 MHz) reports 12.46 bogomips.
>
> I've seen 7(or so) on i386sx/25. Alan seen <1.
I've seen ~1 bm on i386sx/16 with add-on ISA memory card (150 ns chips).
But it no longer works :(
Andrzej
--
=======================================================================
Andrzej M. Krzysztofowicz [email protected]
phone (48)(58) 347 14 61
Faculty of Applied Phys. & Math., Gdansk University of Technology
On Fri, 2003-03-21 at 11:28, Andrzej Krzysztofowicz wrote:
> > From: Pavel Machek <[email protected]>
> > To: Pascal Schmidt <[email protected]>
> > Cc: [email protected]
> >
> > > > So, who can beat his 15.10 bogomips? Anyone run <1 bogomips?
> > > My firewall machine (i486DX running at 25 MHz) reports 12.46 bogomips.
> >
> > I've seen 7(or so) on i386sx/25. Alan seen <1.
>
> I've seen ~1 bm on i386sx/16 with add-on ISA memory card (150 ns chips).
> But it no longer works :(
>
An original IBM 4.77MHz PC reports 0.7 bogomips running Linux 8086, but
can still run a webserver!
Followup to: <[email protected]>
By author: Alan Cox <[email protected]>
In newsgroup: linux.dev.kernel
>
> An original IBM 4.77MHz PC reports 0.7 bogomips running Linux 8086, but
> can still run a webserver!
>
I miss my 0.59 bogomips 386/16 (no cache, memory on the ISA bus.)
That was a truly braindamaged piece of engineering, and held the
bogomips antirecord for a long time until ELKS came along and ruined
it all :(
-hpa
--
<[email protected]> at work, <[email protected]> in private!
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64
Eli Carter <[email protected]> writes:
> So, who can beat his 15.10 bogomips? Anyone run <1 bogomips?
The normal systems I develop for have 4-6 bogomips (NEC V850E/MA1 @50MHz).
For debugging I often run linux on gdb's simulator, which probably has
somewhere under 1 bogomips (the number reported is not correct because I
have to lie to the kernel about the clock interrupt rate, as it can't
handle the real value). It's still quite usable interactively though
(actually 2.5.x feels noticably better than 2.4.x in this situation)...
-Miles
--
97% of everything is grunge
Miles Bader <[email protected]> writes:
> The normal systems I develop for have 4-6 bogomips (NEC V850E/MA1 @50MHz).
BTW, this system has no cache, and I think about 3 wait-states on normal
memory access.
-Miles
--
"Whatever you do will be insignificant, but it is very important that
you do it." Mahatma Ghandi
I can get down to somewhere around 2... (Mips R5k w/ no l2
cache... painfull) why? I appear to have missed the start of this thread
Nick
On 24 Mar 2003, Miles Bader wrote:
> Eli Carter <[email protected]> writes:
> > So, who can beat his 15.10 bogomips? Anyone run <1 bogomips?
>
> The normal systems I develop for have 4-6 bogomips (NEC V850E/MA1 @50MHz).
>
> For debugging I often run linux on gdb's simulator, which probably has
> somewhere under 1 bogomips (the number reported is not correct because I
> have to lie to the kernel about the clock interrupt rate, as it can't
> handle the real value). It's still quite usable interactively though
> (actually 2.5.x feels noticably better than 2.4.x in this situation)...
>
> -Miles
> --
> 97% of everything is grunge
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
On Thu, 2003-03-20 11:51:41 -0600, Eli Carter <[email protected]>
wrote in message <[email protected]>:
> Mike Dresser wrote:
> >On Thu, 20 Mar 2003, Jan-Benedict Glaw wrote:
> >
> >>jbglaw@schnarchnase:/tmp$ cat /proc/cpuinfo
> >
> >
> >>bogomips : 15.10
>
> So, who can beat his 15.10 bogomips? Anyone run <1 bogomips?
I've tested an Intel i386SX, 16MHz these days. It reached 1.19 Bogomips
IIRC using 2.4.x.
As you may see, I'm currently collecting very different hardware (very
old ia32 based systems as well as anything != ia32 capable of running
Linux). My goal is to build an (automated) compile & boot farm of
machines to detect major breakages _early_. In the past few years, we
had quite some ups (and some really hard downs...) which I want to see:
- Remember SCSI-eh changes? Some drivers still need to be ported (or am
I wrong here?)
- module-init-tools took quite some time...
- sparc32 unfortunately is not yet really running 2.5.x. 2.4.x doesn't
do SMP for me (Level 15 Interrupt). However, here I do know when this
was introduced and I'll have a look. Somewhen...
- ...
Things are easier to sort out as long as the breakages are _new_. If
they're there for a year and nobody cares about fixing them, non-ia32
architectures might again be near being dropped (sun4?).
With some luck, I'll get some room (incl. power and IP connectivity)
near Halle/Westf. (Germany). If this fails, I'd be _really_ interested
in other offers here... "CRT - Compile and Run a Test"
MfG, JBG
--
Jan-Benedict Glaw [email protected] . +49-172-7608481
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg
fuer einen Freien Staat voll Freier B?rger" | im Internet! | im Irak!
ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));
Pavel Machek <[email protected]> writes:
> I've seen 7(or so) on i386sx/25. Alan seen <1.
Hmm. My 33 MHz i386 with 64 KB cache had 6.8 IIRC. One hour was more than
enough for the kernel to compile I think.
25 MHz i386 with no cache but with pipelined RAM had less bogomips, but
don't remember how much less.
Anyway, I don't use .gz from kernel.org anymore.
--
Krzysztof Halasa
Network Administrator
On Thu, 20 Mar 2003, [iso-8859-1] J?rn Engel wrote:
> That shouldn't matter, most of the times. If you want to build the
> code, you have to [bg]unzip anyway, so there is no extra cost.
> And I have a hard time to think of a real-world application where you
> don't want to unpack but need to verify the signature.
My real world includes downloading a bunch of files and burning a CD to
move them to a test environment which is completely private and has no
external connections of any kind. I don't do all files that way, of
course, but that is the way at least half of the 2.5 kernels I've used
were moved to a machine which was non-production.
--
bill davidsen <[email protected]>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
On Thu, 20 Mar 2003, Thomas Duffy wrote:
> On Thu, 2003-03-20 at 09:51, Eli Carter wrote:
> > So, who can beat his 15.10 bogomips?
>
> my firewall:
>
> [tduffy@crackho ~]$ more /proc/cpuinfo
> processor : 0
> vendor_id : unknown
> cpu family : 4
> model : 0
> model name : 486
> stepping : unknown
> fdiv_bug : no
> hlt_bug : no
> sep_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : no
> fpu_exception : no
> cpuid level : -1
> wp : yes
> flags :
> bogomips : 12.44
>
> --
> YOO-ESS-AYE! YOO-ESS-AYE!
At one point I ran Linux on a 386SX-16 with 12MB. That machine ran 1.2.13
(IIRC) until Dec 31 1999, when I was afraid it was not Y2k hardened. I
still see spam to glacial.tmr.com today. The name was NOT because it was
so cool ;-)
I may still have that board, but I'm not about to put it back in service
to measure speed. Your firewall is the slowest "real machine" I've seen,
emulation and embedded machines are not really general purpose.
--
bill davidsen <[email protected]>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
On Tue, 2003-03-25 11:04:51 -0500, Bill Davidsen <[email protected]>
wrote in message <[email protected]>:
> On Thu, 20 Mar 2003, Thomas Duffy wrote:
>
> > On Thu, 2003-03-20 at 09:51, Eli Carter wrote:
> > > So, who can beat his 15.10 bogomips?
> >
> > my firewall:
>
> At one point I ran Linux on a 386SX-16 with 12MB. That machine ran 1.2.13
> (IIRC) until Dec 31 1999, when I was afraid it was not Y2k hardened. I
> still see spam to glacial.tmr.com today. The name was NOT because it was
> so cool ;-)
>
> I may still have that board, but I'm not about to put it back in service
> to measure speed. Your firewall is the slowest "real machine" I've seen,
I do have such a system, too (but only 4..8MB RAM) and I'm going to
install it this evening (or tomorrow). I've just got back a 40GB HDD
from IBM^WHitachi:) That box should have just about 1.2 BogoMips...
MfG, JBG
--
Jan-Benedict Glaw [email protected] . +49-172-7608481
"Eine Freie Meinung in einem Freien Kopf | Gegen Zensur | Gegen Krieg
fuer einen Freien Staat voll Freier B?rger" | im Internet! | im Irak!
ret = do_actions((curr | FREE_SPEECH) & ~(IRAQ_WAR_2 | DRM | TCPA));
* Bill Davidsen ([email protected]) wrote:
> On Thu, 20 Mar 2003, Thomas Duffy wrote:
>
> > On Thu, 2003-03-20 at 09:51, Eli Carter wrote:
> > > So, who can beat his 15.10 bogomips?
> >
[...]
> > bogomips : 12.44
>
> At one point I ran Linux on a 386SX-16 with 12MB. That machine ran 1.2.13
> (IIRC) until Dec 31 1999, when I was afraid it was not Y2k hardened. I
> still see spam to glacial.tmr.com today. The name was NOT because it was
> so cool ;-)
>
> I may still have that board, but I'm not about to put it back in service
> to measure speed. Your firewall is the slowest "real machine" I've seen,
> emulation and embedded machines are not really general purpose.
If we're really curious...
sfrost@ns2:/home/sfrost> cat /proc/cpuinfo
processor : 0
vendor_id : unknown
cpu family : 4
model : 0
model name : 486
stepping : unknown
fdiv_bug : no
hlt_bug : no
f00f_bug : no
coma_bug : no
fpu : yes
fpu_exception : no
cpuid level : -1
wp : yes
flags :
bogomips : 9.42
This is my secondary name server. This was also after an upgrade from
a 386 because bind9 is a bloody pig. :) To be honest I've thought about
putting the 386 back in service as something else because unlike my web
server and primary name server there's no chance a CPU fan on it is
going to die causing a CPU to fry and the system to crash. At one point
the 386 had a 630 day uptime, running 2.2.16.
Stephen
Hi All,
Your little BogoMips "contest" made me curious..:-) I've build a ELKS Kernel for my old 8088 PC. After some trouble with booting and root disks I can tell you the result now:
In "Turbo";-) mode I got 0.64 BogoMips (~8Mhz)
In "Normal" mode I got 0.37 BogoMips (~4Mhz)
Have you ever seen such a fast machine? :-))
greets
Marc
On Tue, 25 Mar 2003 11:34:02 -0500
Stephen Frost <[email protected]> wrote:
> * Bill Davidsen ([email protected]) wrote:
> > On Thu, 20 Mar 2003, Thomas Duffy wrote:
> >
> > > On Thu, 2003-03-20 at 09:51, Eli Carter wrote:
> > > > So, who can beat his 15.10 bogomips?
> > >
> [...]
> > > bogomips : 12.44
> >
> > At one point I ran Linux on a 386SX-16 with 12MB. That machine ran 1.2.13
> > (IIRC) until Dec 31 1999, when I was afraid it was not Y2k hardened. I
> > still see spam to glacial.tmr.com today. The name was NOT because it was
> > so cool ;-)
> >
> > I may still have that board, but I'm not about to put it back in service
> > to measure speed. Your firewall is the slowest "real machine" I've seen,
> > emulation and embedded machines are not really general purpose.
>
> If we're really curious...
>
> sfrost@ns2:/home/sfrost> cat /proc/cpuinfo
> processor : 0
> vendor_id : unknown
> cpu family : 4
> model : 0
> model name : 486
> stepping : unknown
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : no
> cpuid level : -1
> wp : yes
> flags :
> bogomips : 9.42
>
> This is my secondary name server. This was also after an upgrade from
> a 386 because bind9 is a bloody pig. :) To be honest I've thought about
> putting the 386 back in service as something else because unlike my web
> server and primary name server there's no chance a CPU fan on it is
> going to die causing a CPU to fry and the system to crash. At one point
> the 386 had a 630 day uptime, running 2.2.16.
>
> Stephen
>
On Tue, 25 March 2003 10:59:54 -0500, Bill Davidsen wrote:
> On Thu, 20 Mar 2003, [iso-8859-1] J?rn Engel wrote:
>
> > And I have a hard time to think of a real-world application where you
> > don't want to unpack but need to verify the signature.
>
> My real world includes downloading a bunch of files and burning a CD to
> move them to a test environment which is completely private and has no
> external connections of any kind. I don't do all files that way, of
> course, but that is the way at least half of the 2.5 kernels I've used
> were moved to a machine which was non-production.
Real world always wins over imagination. I'll shut up now. :)
J?rn
--
When in doubt, use brute force.
-- Ken Thompson