LinuxLists.cc - [RFC] Linux Kernel Subversion Howto

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Thu, Feb 03, 2005 at 12:29:00AM +0000, H. Peter Anvin wrote:

> > And now a question to Larry and whoever else is involved in the
> > bkcvs mirror on kernel.org: what is the periodicity of the CVS
> > repository update ?
> >
>
> Currently it's nightly. Larry has offered to run it more often if
> someone can provide a dedicated fast machine to run it on. (Larry: is
> it a matter of memory or of CPU or both? If nothing else we should
> have the old kernel.org server, dual P3/1133 with 6 GB RAM, coming
> free soon.)

Incremental updates should not be that resource consuming... Having the
repository updated every a few hours would be nice...

> Please let me know if there is something that should be put on
> kernel.org; we can host repositories there of course.

Maybe we could put the converted SVN repository (together with the
conversion tool) on kernel.org, and let those who are interested just
mirror it (either with svm or rsync) instead of generating it on
their own machine (which takes several hours).

Is somebody interested ?

Stelian.
--
Stelian Pop <[email protected]>

2005-02-03 10:33:39

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, Feb 02, 2005 at 05:15:16PM +0100, Lethalman wrote:

> (first sorry for my poor English)
> Very nice howto. It's useful for generic use of svn too.
> The notations about converting bk to svn is really interesting... nice job!
>
> Just a little error:
> How to I ignore temporary build files ? <- to should be do
>
> I would add this rule as a personal cross-development tecnique:
> Before any kind of changes and commits, it would be good to update the
> repository to prevent incompatibilities in the code if previous
> changes was made by other developers.

Thanks for the suggestions, I've updated the howto on the webpage.

Stelian.
> >
> > I've put it also on my web page along with the necessary scripts:
> > http://popies.net/svn-kernel/

Stelian.
--
Stelian Pop <[email protected]>

2005-02-03 10:48:36

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, Feb 02, 2005 at 10:47:58PM +0100, Daniele Venzano wrote:

> Il giorno 02/feb/05, alle 16:54, Stelian Pop ha scritto:
>
> >Hi,
> >
> >I've played lately a bit with Subversion and used it for managing
> >the kernel sources, using Larry McVoy's bk2cvs bridge and Ben Collins'
> >bkcvs2svn conversion script.
>
> Really useful, thanks !
> I'm using svn to manage a very small part of the kernel tree (2 files).
> It is difficult to keep in sync with mainstream development without
> having to fetch and keep huge amounts of data (this regardless of the
> version control system used).

Indeed. As I put it in the howto, the SVN repository is about 900 MB
(but I think it can be less when using svn's fsfs storage backend,
I must test this...), plus 600 MB per working copy.

There is not much anybody can do about this.

> For now I'm keeping the latest stable 2.6 release of the files I need
> in the svn repo, then when I need to sync with the rest of the world, I
> get the latest -bk patch and see if there are some related changes. If
> so, I create a new branch, apply the -bk patch (only the interesting
> part) and then apply my modifications on top of that.

With the 'full' svn solution you lose some storage space but you
gain in time. The above steps are automatic and you don't have to
bother looking at the changes, decide if it matters or not, etc.

> That still means I have to download usually a >1MB compressed file for
> ~60KB of interesting (uncompressed) data, but that is still much better
> than the gigabytes of network traffic needed for a full kernel tree and
> up to date working copies.

I think you end up with pretty much the same network traffic in both
cases (svn or linux.tar.gz + bk patches). The only difference is the
storage cost.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-03 16:45:08

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

(Thanks for the forward, Peter, I would have missed this).

Intel has very kindly donated one of their high end boxes and that's
what is running bkbits.net these days. We could run the exporter there
pretty much as often as you want. Send some love to Intel, this box is
way more stable than the previous bkbits.net. Len Brown made it happen.

As Peter said, we do exports from Linus' tree every 24 hours. I can
think of two things that we could do which might be useful to the non BK
users: export more frequently (pretty questionable in my mind but it's
no big deal to bump it up to twice or whatever) and/or export other trees
(far more interesting in my mind).

We can always use more hardware if someone is looking to donate but
we can also afford to buy what we need. Our biggest ongoing cost in
supporting you these days is our internet connection, that's running
about $11K/year for the portion that is exclusively yours. Fear not,
we get marketing value from providing that so it's no problem, we'll
keep doing it. A question though, is anyone feeling like we need to
allocate more bandwidth or is that enough? You guys currently keep a
T1 line pretty much filled 24x7, we can add another one or bump it up
to a T3 if really need to do so.

We're always looking for more ways to help you (or more properly said:
more ways to be perceived as helping) so let us know what you would like.
In Bk or out of it. Is BK/Web good enough? Need something? Let us
know.

Cheers,

--lm

On Wed, Feb 02, 2005 at 04:28:23PM -0800, H. Peter Anvin wrote:
> Followup to: <[email protected]>
> By author: Stelian Pop <[email protected]>
> In newsgroup: linux.dev.kernel
> >
> > Hi,
> >
> > I've played lately a bit with Subversion and used it for managing
> > the kernel sources, using Larry McVoy's bk2cvs bridge and Ben Collins'
> > bkcvs2svn conversion script.
> >
> > Since there is little information on the web on how to properly
> > set up a SVN repository and use it for tracking the latest kernel
> > tree, I wrote a small howto (modeled after the bk kernel howto)
> > in case it can be useful for other people too.
> >
> > Feel free to comment on it (but let's not start a new BK flamewar
> > or SVN bashing session please). If there is enough interest I'll
> > submit a patch to include this in the kernel Documentation/
> > directory.
> >
> > I've put it also on my web page along with the necessary scripts:
> > http://popies.net/svn-kernel/
> >
> > And now a question to Larry and whoever else is involved in the
> > bkcvs mirror on kernel.org: what is the periodicity of the CVS
> > repository update ?
> >
>
> Currently it's nightly. Larry has offered to run it more often if
> someone can provide a dedicated fast machine to run it on. (Larry: is
> it a matter of memory or of CPU or both? If nothing else we should
> have the old kernel.org server, dual P3/1133 with 6 GB RAM, coming
> free soon.)
>
> Please let me know if there is something that should be put on
> kernel.org; we can host repositories there of course.
>
> -hpa

--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-03 16:49:20

by H. Peter Anvin

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Larry McVoy wrote:
>
> As Peter said, we do exports from Linus' tree every 24 hours. I can
> think of two things that we could do which might be useful to the non BK
> users: export more frequently (pretty questionable in my mind but it's
> no big deal to bump it up to twice or whatever) and/or export other trees
> (far more interesting in my mind).
>

I think exporting every 6 hours might be a good place. That avoids the
"can't get it until tomorrow" effect.

As far as exporting other trees, I'll leave it up to whomever wants it
to say which, but we certainly can host the uplink result.

-hpa

2005-02-03 19:36:53

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, Feb 02, 2005 at 07:34:59PM -0800, Larry McVoy wrote:

> (Thanks for the forward, Peter, I would have missed this).

Sorry, I should have cc'ed you directly but I thought you would
never miss a subject containing $(random scm tool) :)

> As Peter said, we do exports from Linus' tree every 24 hours. I can
> think of two things that we could do which might be useful to the non BK
> users: export more frequently (pretty questionable in my mind but it's
> no big deal to bump it up to twice or whatever)

As Peter said, once every 6 hours is fine. Or even more often, what
the heck, as I said in a previous post I don't think an incremental
export is that much costly. It could be done at the same time as
the -bkX patches...

> and/or export other trees
> (far more interesting in my mind).

Why not if there is some interest. As far as I am concerned, I
have the impression that most kernel BK trees in bkbits.net are
just temporary trees 'Linus please pull from here' type. I don't
know if it's useful for somebody else to track those down.
>
> We're always looking for more ways to help you (or more properly said:
> more ways to be perceived as helping) so let us know what you would like.
> In Bk or out of it. Is BK/Web good enough? Need something? Let us
> know.

Speaking from the out-BK point of view, what would really be nice
is better granularity in the CVS export (a 1-1 changeset to CVS commit
mapping). I know this involves playing with CVS branches and could
be a bit tricky but should be doable.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-03 20:22:10

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

> As Peter said, once every 6 hours is fine. Or even more often, what
> the heck, as I said in a previous post I don't think an incremental
> export is that much costly. It could be done at the same time as
> the -bkX patches...

I'll see what I can do.

> Speaking from the out-BK point of view, what would really be nice
> is better granularity in the CVS export (a 1-1 changeset to CVS commit
> mapping). I know this involves playing with CVS branches and could
> be a bit tricky but should be doable.

I have two problems with this request:

- The idea that the granularity in CVS is unreasonable is pure
nonesense. Here's the data as of this email:

CVS BitKeeper [*]
Deltas 235,956 280,212

- It is not at all an easy thing to do in CVS, we looked at it and
guessed it is about 3 man months of work.

So let's see what's reasonable. In order for you to get the last 16%
of the granularity, which you need because you want to compete with us,
you'd like us to do another 3 man months of work. What would you say if
you were me in this situation?

--lm

[*] Commands used to generate the above data:

BK:
bk -r prs -hnd:I: | wc -l

CVS:
#!/usr/bin/perl -w

open(F, "find linux-2.5 -name '*,v' |");
$files = $revs = 0;
while (<F>) {
chop;
open(F2, $_);
$head = <F2>;
close(F2);
unless ($head =~ /head\s+1\.(\d+)/) {
warn "\n$_ is junk\n";
next;
}
$files++;
$revs += $1;
print STDERR "files=$files revs=$revs\r";
}
print "\n\nfiles=$files revs=$revs\n";

2005-02-03 22:36:19

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

> I really don't want to start a new BK flamewar. You asked what could
> you do and I said what would be nice to have. End of story.
>
> > - The idea that the granularity in CVS is unreasonable is pure
>
> I didn't say it was unreasonable, I said it could be better.

Sure, everything can always be better. Reasonable is reasonable. If
you want more than reasonable then use BK.

> > CVS BitKeeper [*]
> > Deltas 235,956 280,212
>
> Indeed, for now the differences are rather small. But with more and
> more BK trees and more merges between them the proportion will raise.

Actually that's not been the case to date, it's held pretty constant
and in fact the ratio has gotten better. The last time we visited
these numbers it wasn't as good as it is today in CVS>

> If Andrew were to start using BK today we could immediately lose
> (on the CVS side) a big part of the history.

"A big part"? What big part? You are fixated on something that doesn't
have any value. You're complaining about losing information that CVS
wouldn't have recorded if you were using CVS. The CVS export tree has
MORE information than you would have if all the development had been
done under CVS. Explain to me why this information is suddenly so
valuable to you? Explain to me why all the people using all the CVS
maintained projects in the world aren't whining about all the lost
information.

> It is a bit difficult to get it right wrt renames, deletes etc, and
> it can take quite a while to execute, but 3 man month work is a bit
> extreme.

I stand by the 3 month number.

> I thought the competition was between the tools not the data inside...
> Why is that every time someone wants the full history of the kernel
> you think it wants to compete with you ? That history is not even a
> secret, everybody can get it from BK/web or by running the free
> edition of BK (if allowed).

Right but people agree to not use BK to compete with us. It's a form
of payment for the product, we give it to you for no money and you agree
not to copy the product and take away our business.

I realize you hate this, it curtails your freedom, there are a thousand
reasons not to like it, I wouldn't like it if I were in your shoes.
I get it, it's a miserable arrangement. However, we took a huge risk
handing you and everyone else our product for free. It works better, you
can see that, and you are just the sort to try and reverse engineer it.
As a business strategy it was foolish. But it wasn't a business decision,
it was a choice that I made because I wanted to help Linus.

And it worked. That ought to have some value in your eyes. Maybe
enough to respect our terms.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-03 22:18:45

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Thu, Feb 03, 2005 at 12:20:49PM -0800, Larry McVoy wrote:

> > As Peter said, once every 6 hours is fine. Or even more often, what
> > the heck, as I said in a previous post I don't think an incremental
> > export is that much costly. It could be done at the same time as
> > the -bkX patches...
>
> I'll see what I can do.

Thanks.

> > Speaking from the out-BK point of view, what would really be nice
> > is better granularity in the CVS export (a 1-1 changeset to CVS commit
> > mapping). I know this involves playing with CVS branches and could
> > be a bit tricky but should be doable.
>
> I have two problems with this request:

I really don't want to start a new BK flamewar. You asked what could
you do and I said what would be nice to have. End of story.

> - The idea that the granularity in CVS is unreasonable is pure

I didn't say it was unreasonable, I said it could be better.

> nonesense. Here's the data as of this email:
>
> CVS BitKeeper [*]
> Deltas 235,956 280,212

Indeed, for now the differences are rather small. But with more and
more BK trees and more merges between them the proportion will raise.

If Andrew were to start using BK today we could immediately lose
(on the CVS side) a big part of the history.

> - It is not at all an easy thing to do in CVS, we looked at it and
> guessed it is about 3 man months of work.

I may be stupid, but I did write several months ago a bitkeeper to
prcs conversion tool, and this followed BK branches. It's a 269 lines
python script including documentation. I can send you that if you like.

The script does basically:
for each bk changeset {
get cset parent, merge parent, tags, comment, date
prcs checkout -r bktoprcsbranch(cset) parent
for each bk changeset rename {
prcs rename
}
bk export changeset
if merge parent {
prcs merge with merge parent
}
if cset ends in .1 {
prcs create branch
}
prcs checkin bktoprcsbranch(cset)
if bk has tag {
prcs tag
}
}

with bktoprcsbranch(cset) returning the bk 'branch',
1 for the 'trunk' and x.y.z for a x.y.z.t revision.

It is a bit difficult to get it right wrt renames, deletes etc, and
it can take quite a while to execute, but 3 man month work is a bit
extreme.

> So let's see what's reasonable. In order for you to get the last 16%
> of the granularity, which you need because you want to compete with us,
> you'd like us to do another 3 man months of work. What would you say i> you
> were me in this situation?

I thought the competition was between the tools not the data inside...
Why is that every time someone wants the full history of the kernel
you think it wants to compete with you ? That history is not even a
secret, everybody can get it from BK/web or by running the free
edition of BK (if allowed).

Stelian.
--
Stelian Pop <[email protected]>

2005-02-04 10:17:14

by Michael S. Tsirkin

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi, Stelian!
One thing everyone creating kernel patches with subversion
must be aware of, is the fact that the subversion built-in diff command does
not understand the gnu diff -p flag (or indeed, any gnu diff flags at all,
with the exception of -u, which is the default anyway).

Thus you must use an external diff command to create kernel patches,
passing the -up flags to gnu diff, otherwise the patch is much less readable.
There are two good ways to do this that I know of:

1. Remember to always generate patches with:
svn diff --diff-cmd=/usr/bin/diff -x -up

2. Make subversion use gnu diff -up by defult to generate patches.

To do this, create a script that runs diff -up, passing any additional
parameters exactly as they are:

cat > /usr/bin/svndiff << EOF
#!/usr/bin/perl

exec("diff","-up",@ARGV);
EOF

chmod +x /usr/bin/svndiff

And set it as the default diff command in subversion:

cat >> ~/.subversion/config << EOF

[helpers]
diff-cmd = /usr/bin/svndiff

EOF

Tested with subversion 1.1.3.

--
MST - Michael S. Tsirkin

2005-02-04 10:59:47

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Fri, Feb 04, 2005 at 12:18:27PM +0200, Michael S. Tsirkin wrote:

>
> Hi, Stelian!
> One thing everyone creating kernel patches with subversion
> must be aware of, is the fact that the subversion built-in diff command does
> not understand the gnu diff -p flag (or indeed, any gnu diff flags at all,
> with the exception of -u, which is the default anyway).

There is a section called "How do I generate 'proper' diffs ?" dealing
with this.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-04 11:07:03

by Michael S. Tsirkin

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Quoting r. Stelian Pop <[email protected]>:
> Subject: Re: [RFC] Linux Kernel Subversion Howto
>
> On Fri, Feb 04, 2005 at 12:18:27PM +0200, Michael S. Tsirkin wrote:
>
> >
> > Hi, Stelian!
> > One thing everyone creating kernel patches with subversion
> > must be aware of, is the fact that the subversion built-in diff command does
> > not understand the gnu diff -p flag (or indeed, any gnu diff flags at all,
> > with the exception of -u, which is the default anyway).
>
> There is a section called "How do I generate 'proper' diffs ?" dealing
> with this.
>
> Stelian.
> --
> Stelian Pop <[email protected]>
>

Yep but the trick with --diff-cmd has the advantage of not changing the
default diff for the current user.

--
MST - Michael S. Tsirkin

2005-02-04 11:20:32

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Fri, Feb 04, 2005 at 01:08:35PM +0200, Michael S. Tsirkin wrote:

> > There is a section called "How do I generate 'proper' diffs ?" dealing
> > with this.
> >
> > Stelian.
> > --
> > Stelian Pop <[email protected]>
> >
>
> Yep but the trick with --diff-cmd has the advantage of not changing the
> default diff for the current user.

Makes sense, I've updated the web page.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-04 13:00:27

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Thu, Feb 03, 2005 at 02:28:54PM -0800, Larry McVoy wrote:

> > > CVS BitKeeper [*]
> > > Deltas 235,956 280,212
> >
> > Indeed, for now the differences are rather small. But with more and
> > more BK trees and more merges between them the proportion will raise.
>
> Actually that's not been the case to date, it's held pretty constant
> and in fact the ratio has gotten better. The last time we visited
> these numbers it wasn't as good as it is today in CVS>

In fact I am looking at the number of *changesets*, not the sum of all
file revisions.

CVS BitKeeper
Changesets: 26419 59220 (minus 6994 empty merges)

This isn't 16%, its more or less 50%, and this is the loss I was
complaining about.

It is true that 34% of the changesets could be recreated by analysing
the 'per file' commit logs, but the 16% are not recreatable (those
16% correspond to the case when someone makes several changes to a same
file without pushing to Linus between the two).

>> It is a bit difficult to get it right wrt renames, deletes etc, and
>> it can take quite a while to execute, but 3 man month work is a bit
>> extreme.

> I stand by the 3 month number.

If all that bothers you is the amount of work implied, I have an idea.

I could write a script which does use BK and exports the metadata into
CVS (or SVN directly), without losing any changeset information. I'll
do this for free.

But you may consider this contrary to the 'non competitor' clause of
the BKL (clause 3d), that's why I need some authorization from you
before starting.

Also, I won't work on that if I cannot distribute the result of my work.
The distribution of the metadata is contrary to the 3 (g) clause in bkl,
so I need your authorization to distribute the resulting CVS (or SVN)
repository.

Note that for the distribution it doesn't need to be me, it could be a
third party who would run the script and distribute the result
(kernel.org, osdl, Linus himself etc), or it could even be you if you
want to be in control.

How is that ?

> As a business strategy it was foolish. But it wasn't a business decision,
> it was a choice that I made because I wanted to help Linus.
>
> And it worked. That ought to have some value in your eyes.

It has and I don't question the use of BitKeeper for the kernel
development, as long as it is useful for it (and it is !) and as
long as the data (with the metadata) is also available outside of BK.

> Maybe
> enough to respect our terms.

If I didn't respect your terms I wouldn't ask for permission to do
what I want to.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-04 16:07:01

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Fri, Feb 04, 2005 at 02:01:27PM +0100, Stelian Pop wrote:
> On Thu, Feb 03, 2005 at 02:28:54PM -0800, Larry McVoy wrote:
>
> > > > CVS BitKeeper [*]
> > > > Deltas 235,956 280,212
> > >
> > > Indeed, for now the differences are rather small. But with more and
> > > more BK trees and more merges between them the proportion will raise.
> >
> > Actually that's not been the case to date, it's held pretty constant
> > and in fact the ratio has gotten better. The last time we visited
> > these numbers it wasn't as good as it is today in CVS>
>
> In fact I am looking at the number of *changesets*, not the sum of all
> file revisions.
>
> CVS BitKeeper
> Changesets: 26419 59220 (minus 6994 empty merges)
>
> This isn't 16%, its more or less 50%, and this is the loss I was
> complaining about.
>
> It is true that 34% of the changesets could be recreated by analysing
> the 'per file' commit logs, but the 16% are not recreatable (those
> 16% correspond to the case when someone makes several changes to a same
> file without pushing to Linus between the two).

You need to rethink your math, you are way off. I'll explain it so that
the rest of the people can see this is just pure FUD.

To make sure this was apples to apples I went to the BK and CVS trees
which are in lock step and reran the numbers:

CVS BitKeeper % in CVS
file deltas 210,609 218,742 96%
changsets 26,603 59,220 44%

You are not factoring out the ChangeSet deltas, which are BK metadata,
they aren't part of the source tree. If you remove those then the
difference between CVS and BK revision history is 209K deltas vs
221K deltas.

In other words, the CVS tree is missing no more than 4% of the deltas
to the source files.

READ THAT AGAIN, PLEASE.

The CVS tree has 96% of all the deltas to all your source files. 96%.

My good friend Stelian would have you believe that you are missing 50%
of your data when in fact you are missing NONE of your data, you have
ALL of your data in an almost the identical form.

What Stelian is complaining about is the patch set which is easily
extractable from CVS is at a coarser granularity than the patch set
extractable from BK. That's true but so what? Before BK you only
a handful of patches between releases, now you have thousands.

I suppose what we could do is stick the BK changeset key into the
delta history so that if you really wanted to get the BK level
granularity you could.

> [describes violating the license]
> But you may consider this contrary to the 'non competitor' clause of
> the BKL (clause 3d), that's why I need some authorization from you
> before starting.

And you most certainly do not have it, as you are aware it is a
violation of the license. Asking for an exception is about as polite as
me asking for an exception from the GPL's requirements.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-04 16:27:57

by Roland Dreier

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Larry> You need to rethink your math, you are way off. I'll
Larry> explain it so that the rest of the people can see this is
Larry> just pure FUD.

[...]

Larry> The CVS tree has 96% of all the deltas to all your source
Larry> files. 96%.

Larry> My good friend Stelian would have you believe that you are
Larry> missing 50% of your data when in fact you are missing NONE
Larry> of your data, you have ALL of your data in an almost the
Larry> identical form.

Larry> What Stelian is complaining about is the patch set which is
Larry> easily extractable from CVS is at a coarser granularity
Larry> than the patch set extractable from BK. That's true but so
Larry> what? Before BK you only a handful of patches between
Larry> releases, now you have thousands.

I don't pretend to understand all the tools or where information is
being lost, but I did use Stelian's HOWTO to make a subversion kernel
tree last night. I'm including log information for what happen to be
the two most recent subversion changesets in my tree. I think we can
all agree that quite a bit of useful information is lost to users of
the subversion tree.

------------------------------------------------------------------------
r26549 | torvalds | 2005-01-31 07:47:51 -0800 (Mon, 31 Jan 2005) | 208 lines

Merge bk://linux-acpi.bkbits.net/to-linus
into ppc970.osdl.org:/home/torvalds/v2.6/linux

2005/01/31 01:37:39-05:00 len.brown
Merge intel.com:/home/lenb/src/26-stable-dev
into intel.com:/home/lenb/src/26-latest-dev

2005/01/31 01:35:48-05:00 len.brown
[ACPI] tell parse_cmdline_early() that "pnpacpi=off" != "acpi=off"

Signed-off-by: Len Brown <[email protected]>

2005/01/31 00:15:20-05:00 len.brown
Merge intel.com:/home/lenb/src/26-stable-dev
into intel.com:/home/lenb/src/26-latest-dev

2005/01/31 00:12:40-05:00 len.brown
[ACPI] add "pnpacpi=off"

Signed-off-by: David Shaohua Li <[email protected]>
Signed-off-by: Len Brown <[email protected]>

2005/01/30 23:51:03-05:00 len.brown
Merge intel.com:/home/lenb/bk/26-latest-ref
into intel.com:/home/lenb/src/26-latest-dev

2005/01/27 18:16:15-05:00 len.brown
[ACPI] ACPI_FUTURE_USAGE for acpi_ut_create_pkg_state_and_push()

Signed-off-by: Adrian Bunk <[email protected]>
Signed-off-by: Len Brown <[email protected]>

2005/01/27 18:15:47-05:00 len.brown
[ACPI] reduce stack usage
http://bugzilla.kernel.org/show_bug.cgi?id=2901

Written-by: Luming Yu <[email protected]>
Signed-off-by: Len Brown <[email protected]>

2005/01/26 02:38:19-05:00 len.brown
Merge intel.com:/home/lenb/src/26-stable-dev
into intel.com:/home/lenb/src/26-latest-dev

2005/01/26 02:32:41-05:00 len.brown
[ACPI] ACPICA 20050125 from Bob Moore

Fixed a recently introduced problem with the Global
Lock where the underlying semaphore was not created.
This problem was introduced in version 20050114, and
caused an AE_AML_NO_OPERAND exception during an Acquire()
operation on _GL.

The local object cache is now optional, and is disabled
by default. #define ACPI_ENABLE_OBJECT_CACHE to enable
the local cache.

Fixed an issue in the internal function
acpi_ut_evaluate_object() concerning the optional "implicit
return" support where an error was returned if no return
object was expected, but one was implicitly returned. AE_OK
is now returned in this case and the implicitly returned
object is deleted. acpi_ut_evaluate_object() is only
occasionally used, and only to execute reserved methods
such as _STA and _INI where the return type is known
up front.

Fixed a few issues with the internal convert-to-integer
code. It now returns an error if an attempt is made to
convert a null string, a string of only blanks/tabs, or a
zero-length buffer. This affects both implicit conversion
and explicit conversion via the ToInteger() operator.

The internal debug code in acpi_ut_acquire_mutex()
has been commented out. It is not needed for normal
operation and should increase the performance of the entire
subsystem. The code remains in case it is needed for debug
purposes again.
acpica-unix-20050125.patch

2005/01/24 23:32:11-05:00 len.brown
Merge

2005/01/21 17:45:56-05:00 len.brown
Merge intel.com:/home/lenb/src/26-stable-dev
into intel.com:/home/lenb/src/26-latest-dev

2005/01/21 17:44:33-05:00 len.brown
[ACPI] avoid benign AE_TYPE warnings
caused by "implicit return" BIOS workaround
returning unsolicited (and thus mis-typed) AML values.

Signed-off-by: Bob Moore <[email protected]>
Signed-off-by: Len Brown <[email protected]>

2005/01/21 00:28:02-05:00 len.brown
Merge intel.com:/home/lenb/src/26-stable-dev
into intel.com:/home/lenb/src/26-latest-dev

2005/01/21 00:18:05-05:00 len.brown
[ACPI] ACPICA 20050114 from Bob Moore

Added 2005 copyright to all ACPICA files.

Fixed an issue with the String-to-Buffer conversion code
where the string null terminator was not included in the
buffer after conversion, but there is existing ASL that
assumes the string null terminator is included. This is the
root of the ACPI_AML_BUFFER_LIMIT regression. This problem
was introduced in the previous version when the code was
updated to correctly set the converted buffer size as per
the ACPI specification. The ACPI spec is ambiguous and
will be updated to specify that the null terminator must
be included in the converted buffer. This also affects
the ToBuffer() ASL operator.

Fixed a problem with the Mid() ASL/AML operator where it
did not work correctly on Buffer objects. Newly created
sub-buffers were not being marked as initialized.

Fixed a problem in acpi_tb_find_table where incorrect string
compares were performed on the oem_id and oem_table_d table
header fields. These fields are not null terminated,
so strncmp is now used instead of strcmp.

Implemented a restriction on the Store() ASL/AML operator
to align the behavior with the ACPI specification.
Previously, any object could be used as the source
operand. Now, the only objects that may be used are
Integers, Buffers, Strings, Packages, Object References,
and DDB Handles. As acpi_gbl_enable_interpreter_slack
is FALSE by default, "acpi=strict" is needed to enable
this check.

Enhanced the optional "implicit return" support to allow
an implicit return value from methods that are invoked
externally via the AcpiEvaluateObject interface. This
enables implicit returns from the _STA and _INI methods,
for example.

Changed the Revision() ASL/AML operator to return the
current version of the AML interpreter, in the YYYYMMDD
format. Previously, it incorrectly returned the supported
ACPI version (This is the function of the _REV method).

Updated the _REV predefined method to return the currently
supported version of ACPI, now 3.

2005/01/20 22:58:53-05:00 len.brown
Merge intel.com:/home/lenb/src/26-stable-dev
into intel.com:/home/lenb/src/26-latest-dev

2005/01/20 22:43:52-05:00 len.brown
[ACPI] Add a module parameter to allow tuning how much bus-master activity
we remember when entering C3 -- /sys/module/processor/parameters/bm_history
Default varies with HZ -- 40ms for 25 - 800 HZ, 32ms for 1000 HZ.

Signed-off-by: Len Brown <[email protected]>

2005/01/20 21:52:27-05:00 len.brown
[ACPI] Make the bm_activity depend on "jiffies", instead of numbers
of the check being called. This means bus mastering activity
is assumed if bm_check isn't called; and multiple calls during
one jiffy will be |='ed.

Signed-off-by: Dominik Brodowski <[email protected]>
Signed-off-by: Len Brown <[email protected]>

2005/01/20 16:46:39-05:00 len.brown
Merge intel.com:/home/lenb/bk/26-latest-ref
into intel.com:/home/lenb/src/26-latest-dev

2005/01/20 13:23:36-05:00 len.brown
Merge intel.com:/home/lenb/bk/26-latest-ref
into intel.com:/home/lenb/src/26-latest-dev

2005/01/19 21:16:57-05:00 len.brown
Merge intel.com:/home/lenb/bk/26-latest-ref
into intel.com:/home/lenb/src/26-latest-dev

2005/01/07 14:48:18-05:00 len.brown
Merge intel.com:/home/lenb/bk/26-latest-ref
into intel.com:/home/lenb/src/26-latest-dev

2005/01/07 13:42:16-05:00 len.brown
[ACPI] Use kernel.h for ARRAY_SIZE() instead of using local NUM_OF().

Signed-off-by: Randy Dunlap <[email protected]>
Signed-off-by: Len Brown <[email protected]>

2005/01/06 02:06:24-05:00 len.brown
Merge intel.com:/home/lenb/src/26-stable-dev
into intel.com:/home/lenb/src/26-latest-dev

2005/01/05 23:56:52-05:00 torvalds
acpi video device enumeration: fix incorrect device list allocation

It didn't allocate space for the final terminating entry,
which caused it to overwrite the next slab entry, which in turn
sometimes ended up being a slab array cache pointer. End result:
total slab cache corruption at a random time afterwards. Very
nasty.

2005/01/04 22:57:14-05:00 len.brown
Merge intel.com:/home/lenb/bk/linux-2.6.10
into intel.com:/home/lenb/src/26-stable-dev

BKrev: 41fe5327BXOmplstrv49I26qAg7mIA

------------------------------------------------------------------------
r26548 | torvalds | 2005-01-31 07:43:58 -0800 (Mon, 31 Jan 2005) | 189 lines

Merge bk://kernel.bkbits.net/davem/net-2.6
into ppc970.osdl.org:/home/torvalds/v2.6/linux

2005/01/30 14:39:30-08:00 torvalds
Merge bk://bk.arm.linux.org.uk/linux-2.6-rmk
into ppc970.osdl.org:/home/torvalds/v2.6/linux

2005/01/30 13:20:58-08:00 kaber
[PATCH] Fix conntrack fragment route cache memory leak

Thanks to Russell King for some excellent debugging.

Conntrack defragments locally generated packets before they hit
ip_fragment. In this case the fragments have skb->dst set, and
that needs to be released.

Signed-off-by: Linus Torvalds <[email protected]>

2005/01/30 19:38:54+00:00 rmk
[ARM] [3/4] Introduce usr_entry macro to contain common entry code

This is the third of 4 patches which factor out common code in
the ARM exception entry assembly code, aiming towards a reduction
in the size of the changes required here for SMP support. These
patches are low impact, and will be merged over the coarse of the
next few days.

This patch addresses the code handling exception entry from user
modes.

2005/01/29 19:19:10-08:00 torvalds
Merge bk://bk.arm.linux.org.uk/linux-2.6-rmk
into ppc970.osdl.org:/home/torvalds/v2.6/linux

2005/01/29 09:24:47-08:00 axboe
[PATCH] cfq-iosched: in_driver accounting bug

Yet another accounting bug, this time hits on requeue. It is possible
for ->accounted to be set with ->in_flight, so don't nest the
cfq_account_completion() inside the ->in_flight check.

Signed-off-by: Jens Axboe <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

2005/01/29 09:24:34-08:00 ak
[PATCH] x86-64: Fix empty nodes handling with SRAT

Handle empty nodes in SRAT parsing. Avoids an oops at boot time.

Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

2005/01/29 09:24:19-08:00 ak
[PATCH] x86-64: Fix missing TLB flushes in change_page_attr

Fix bug in change_page_attr - with multiple pages it would not flush
correctly. Also add a small optimization of not flushing when not needed.

Found and fixed by Andrea.

Signed-off-by: Andi Kleen <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>

2005/01/29 13:08:30+00:00 rmk
[ARM] [2/4] Introduce inv_entry macro to contain common entry code

This is the second of 4 patches which factor out common code in
the ARM exception entry assembly code, aiming towards a reduction
in the size of the changes required here for SMP support. These
patches are low impact, and will be merged over the coarse of the
next 3 days.

This patch addresses the code handling exception entry from
invalid (irq, fiq, abort) modes. However, in converting to a
macro, a minor bug has been fixed which would merely cause a
misleading register dump.

2005/01/28 22:29:02-05:00 mkrikis
[PATCH] fix an oops in ata_to_sense_error

Signed-off-by: Jeff Garzik <[email protected]>

2005/01/28 16:13:31-08:00 Andries.Brouwer
[PATCH] document atkbd.softraw

Document atkbd.softraw (and shorten a few long lines nearby).

2005/01/28 16:08:48-08:00 torvalds
Merge bk://bk.arm.linux.org.uk/linux-2.6-rmk
into ppc970.osdl.org:/home/torvalds/v2.6/linux

2005/01/28 23:23:38+00:00 rmk
[ARM] [1/4] Introduce svc_entry macro for common entry code

This is the first of 4 patches which factor out common code in
the ARM exception entry assembly code, aiming towards a reduction
in the size of the changes required here for SMP support. These
patches are low impact, and will be merged over the coarse of the
next 4 days.

This patch addresses the code handling exception entry from
supervisor (kernel) mode.

2005/01/28 22:39:31+00:00 elf
[ARM PATCH] 2442/1: Simplifying NODES_SHIFT

Patch from Marc Singer

The special case for the Sharp LH processors is unnecessary. A macro
override makes it cleaner and concentrates the change where it ought
to be. The default in include/asm-arm/numnodes.h means that only
platforms that care to change the default need to do anything.

Signed-off-by: Marc Singer
Signed-off-by: Russell King

2005/01/28 22:14:31+00:00 ben-linux
[ARM PATCH] 2440/1: S3C2410 - serial auto-flow-control enable

Patch from Ben Dooks

Patch from Shannon Holland
Enable automatic flow control if requested.

Signed-off-by: Shannon Holland

Signed-off-by: Ben Dooks
Signed-off-by: Russell King

2005/01/28 22:08:02+00:00 ben-linux
[ARM PATCH] 2439/1: S3C2410 - serial driver parity selection

Patch from Ben Dooks

Patch from Dimitry Andric.
The s3c2410 serial driver selects the opposite parity
mode if parity is enabled.

Signed-off-by: Dimitry Andric

Signed-off-by: Ben Dooks
Signed-off-by: Russell King

2005/01/28 22:01:24+00:00 ben-linux
[ARM PATCH] 2438/1: S3C2410 - fix IO address calculations

Patch from Ben Dooks

Patch from Dimitry Andric.
The include/asm-arm/arch-s3c2410/io.h file converts
PC style port addresses to real ARM addresses, and
needs to return an `void __iomem *` to avoid a number
of warnings:
CC drivers/ide/ide-iops.o
drivers/ide/ide-iops.c: In function `ide_insw':
drivers/ide/ide-iops.c:49: warning: passing arg 1 of `__raw_readsw' makes pointer from integer without a cast
drivers/ide/ide-iops.c: In function `ide_insl':
drivers/ide/ide-iops.c:59: warning: passing arg 1 of `__raw_readsl' makes pointer from integer without a cast
drivers/ide/ide-iops.c: In function `ide_outsw':
drivers/ide/ide-iops.c:79: warning: passing arg 1 of `__raw_writesw' makes pointer from integer without a cast
drivers/ide/ide-iops.c: In function `ide_outsl':
drivers/ide/ide-iops.c:89: warning: passing arg 1 of `__raw_writesl' makes pointer from integer without a cast
CC lib/iomap.o
lib/iomap.c: In function `ioread8_rep':
lib/iomap.c:140: warning: passing arg 1 of `__raw_readsb' makes pointer from integer without a cast
lib/iomap.c: In function `ioread16_rep':
lib/iomap.c:144: warning: passing arg 1 of `__raw_readsw' makes pointer from integer without a cast
lib/iomap.c: In function `ioread32_rep':
lib/iomap.c:148: warning: passing arg 1 of `__raw_readsl' makes pointer from integer without a cast
lib/iomap.c: In function `iowrite8_rep':
lib/iomap.c:156: warning: passing arg 1 of `__raw_writesb' makes pointer from integer without a cast
lib/iomap.c: In function `iowrite16_rep':
lib/iomap.c:160: warning: passing arg 1 of `__raw_writesw' makes pointer from integer without a cast
lib/iomap.c: In function `iowrite32_rep':
lib/iomap.c:164: warning: passing arg 1 of `__raw_writesl' makes pointer from integer without a cast

Signed-off-by: Dimitry Andric

Signed-off-by: Ben Dooks
Signed-off-by: Russell King

2005/01/28 22:38:30+01:00 wim
[WATCHDOG] i8xx_tco.c-ICH4/6/7-patch

Added support for the ICH4-M, ICH6, ICH6R, ICH6-M, ICH6W and ICH6RW
chipsets. Also added support for the "undocumented" ICH7.

BKrev: 41fe523ecAJ3I6z55zHXaAI1vsDZ8Q

------------------------------------------------------------------------

2005-02-04 17:01:56

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

[Taking hpa out of CC:, I don't think he is interested anymore]

On Fri, Feb 04, 2005 at 08:06:31AM -0800, Larry McVoy wrote:

> You need to rethink your math, you are way off. I'll explain it so that
> the rest of the people can see this is just pure FUD.

There is no FUD in all what I said, for the simple reasons that we
are saying exactly the same thing:
* the data is there
* the metadata is there
* all that is missing is some metadata boundaries.

> My good friend Stelian would have you believe that you are missing 50%
> of your data when in fact you are missing NONE of your data, you have
> ALL of your data in an almost the identical form.

My good friend Larry would have you believe that I said something
I just never did. Go back and read the archives.

> What Stelian is complaining about is the patch set which is easily
> extractable from CVS is at a coarser granularity than the patch set
> extractable from BK. That's true [...]

Exactly. You see, you understood exactly what I did say. And I'm pretty
sure all the audience did.

> [...] but so what? Before BK you only
> a handful of patches between releases, now you have thousands.

I'm not interested in ranting. I already did say that BK was good
for the kernel development, what more would you like me to say ?

>
> I suppose what we could do is stick the BK changeset key into the
> delta history so that if you really wanted to get the BK level
> granularity you could.

This would be nice indeed and I think it would end up all whinning
since using that, one could be able to get the full history from the
bkcvs repository. But would you do that ?

> > [describes violating the license]
> > But you may consider this contrary to the 'non competitor' clause of
> > the BKL (clause 3d), that's why I need some authorization from you
> > before starting.
>
> And you most certainly do not have it, as you are aware it is a
> violation of the license.

I was merely proposing my help to do something you said you don't
wanna do because it would consume too much resources. Does this
proposal deserves such a rude answer ?

Moreover, I said "you may consider...", not "I am aware it is...".
Using BK to write a script which helps extracting the metadata is not
clearly a violation, since the script itself does not "compete with
the BitKeeper Software.". This part a clarification, and this is
what I asked for.

The distribution of the metadata is clearly a violation, but you
didn't quote that part of the proposal...

> Asking for an exception is about as polite as
> me asking for an exception from the GPL's requirements.

I don't see what 'polite' has to do with it. If one of us tends to
be rude I would say that's you...

Anyway, it is perfectly valid to ask the author(s) for an exception
to the distribution license. The author is free to reject the demand,
or to accept it if there is some interest in doing so.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-04 18:40:25

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

> > My good friend Stelian would have you believe that you are missing 50%
> > of your data when in fact you are missing NONE of your data, you have
> > ALL of your data in an almost the identical form.
>
> My good friend Larry would have you believe that I said something
> I just never did. Go back and read the archives.

Come on, claiming that you are only getting 50% of the information is
just blatent FUD. If you can't see that I can't help you.

> > I suppose what we could do is stick the BK changeset key into the
> > delta history so that if you really wanted to get the BK level
> > granularity you could.
>
> This would be nice indeed and I think it would end up all whinning
> since using that, one could be able to get the full history from the
> bkcvs repository. But would you do that ?

You get a commitment from the group of BK complainers that that is good
enough and we'll do it. It may take a while because in the current way
that BK stores metadata it would be prohibitively expensive. But we
want to change that anyway so we can prototype it on an internal tree
and see how well it works.

If/when we do this we'll reexport the 2.4 and 2.5 histories from scratch
so you get the info going backwards in time.

So, do you think you can sign up the usual suspects to being happy with
this answer? And do you mind spelling out exactly what it is that you
think is being offered so there is no confusion later?
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-04 20:16:53

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

> > So, do you think you can sign up the usual suspects to being happy with
> > this answer?
>
> I'll let them answer themselves.

You'll need to rally them to speak up or this is going nowhere. We
can't afford to spend engineering dollars one unhappy person at a time
to try and get you happy. We need concensus.

> > And do you mind spelling out exactly what it is that you
> > think is being offered so there is no confusion later?
>
> Informaly, exactly what I said before: Be able to find enough information
> in the CVS tree which would allow anybody to trace back each change
> to what was submitted by the author of the change (= patch + comment).

Yup, seems reasonable.

> How you can make this happen is another problem. The obvious way to
> implement this is using CVS branches and merges but this could be
> too much work for you.

Yeah, that's insane, CVS just isn't up to the task. We know, we import
history from CVS to BitKeeper all the time and it is very painful. We
probably understand the impedence mismatch better than anyone and it is
large.

> (and know I agreed at the moment), but thinking again about this I'm not
> sure anymore how "sticking the BK changeset key into the delta history"
> gives us "BK level granularity". From what I understand (but you are the
> SCM expert not me so I may be missing something) there is exactly
> one delta per 'trunk' changeset, so if you have a file being modified
> several times on a branch you will end with one single delta which is
> the merge of the separate patches. I'm not sure how, by adding several
> 'BK changeset keys' into the log entry of the merged delta you make
> one able to resplit the delta later.

First, you have to remember that BK is capturing 96% of the deltas made
to files. Some of those deltas get clumped into one CVS commit because
of the flattening of the graph structure. If each delta had the
changeset key for the BK changeset to which it belonged you could
split the coarse commit into the sub patches which happened on the
collapsed branch. You wouldn't get 100% of the information but you'd
have 96% of it in a way that could be used for debugging, which is what
I suspect you are after.

If that's not good enough then I'm not sure there is much point in
continuing the conversation.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-04 20:23:01

by Daniele Venzano

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On 03/feb/05, at 11:45, Stelian Pop wrote:

>> For now I'm keeping the latest stable 2.6 release of the files I need
>> in the svn repo, then when I need to sync with the rest of the world,
>> I
>> get the latest -bk patch and see if there are some related changes. If
>> so, I create a new branch, apply the -bk patch (only the interesting
>> part) and then apply my modifications on top of that.
>
> With the 'full' svn solution you lose some storage space but you
> gain in time. The above steps are automatic and you don't have to
> bother looking at the changes, decide if it matters or not, etc.

Well, I just have found a way to download single diffs out of bkbits,
so I can probably just interface with that. Remember, I have to track
only two files.
Mine was the point of view of a small kernel worker, I maintain only
one driver and I don't want to commit more resources to the task than
what would be reasonable. The difference in storage space is measurable
in several hundredth of megabytes, and that is a big gain on my poor
resources.

However, that does not remove any value from your howto, that is both
complete and useful.

--
Daniele Venzano
http://teg.homeunix.org

2005-02-04 20:29:36

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Fri, Feb 04, 2005 at 10:39:22AM -0800, Larry McVoy wrote:

> > > I suppose what we could do is stick the BK changeset key into the
> > > delta history so that if you really wanted to get the BK level
> > > granularity you could.
> >
> > This would be nice indeed and I think it would end up all whinning
> > since using that, one could be able to get the full history from the
> > bkcvs repository. But would you do that ?
>
> You get a commitment from the group of BK complainers that that is good
> enough and we'll do it. It may take a while because in the current way
> that BK stores metadata it would be prohibitively expensive. But we
> want to change that anyway so we can prototype it on an internal tree
> and see how well it works.
>
> If/when we do this we'll reexport the 2.4 and 2.5 histories from scratch
> so you get the info going backwards in time.

If we find enough information in the CVS tree which would allow
anybody to trace back each change to what was submitted by the author
of the change (this being a GNU patch and a patch comment), this
would be perfect for me. I can't speak for the 'group of BK
complainers' but I can't see a *technical* reason why they would
disagree with this.

> So, do you think you can sign up the usual suspects to being happy with
> this answer?

I'll let them answer themselves.

> And do you mind spelling out exactly what it is that you
> think is being offered so there is no confusion later?

Informaly, exactly what I said before: Be able to find enough information
in the CVS tree which would allow anybody to trace back each change
to what was submitted by the author of the change (= patch + comment).

How you can make this happen is another problem. The obvious way to
implement this is using CVS branches and merges but this could be
too much work for you.

You said before that:
> > I suppose what we could do is stick the BK changeset key into the
> > delta history so that if you really wanted to get the BK level
> > granularity you could.
(and know I agreed at the moment), but thinking again about this I'm not
sure anymore how "sticking the BK changeset key into the delta history"
gives us "BK level granularity". From what I understand (but you are the
SCM expert not me so I may be missing something) there is exactly
one delta per 'trunk' changeset, so if you have a file being modified
several times on a branch you will end with one single delta which is
the merge of the separate patches. I'm not sure how, by adding several
'BK changeset keys' into the log entry of the merged delta you make
one able to resplit the delta later.

What am I missing here ?

Thanks,

Stelian.
--
Stelian Pop <[email protected]>

2005-02-04 21:00:59

by Olaf Dietsche

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Stelian Pop <[email protected]> writes:

> I must test this...), plus 600 MB per working copy.

If you use svk <http://svk.elixus.org/> for the client side, there's
(almost?) no overhead.

Regards, Olaf.

2005-02-04 22:34:43

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Fri, Feb 04, 2005 at 12:11:57PM -0800, Larry McVoy wrote:

> > > So, do you think you can sign up the usual suspects to being happy with
> > > this answer?
> >
> > I'll let them answer themselves.
>
> You'll need to rally them to speak up or this is going nowhere. We
> can't afford to spend engineering dollars one unhappy person at a time
> to try and get you happy. We need concensus.

I understand.

> > > And do you mind spelling out exactly what it is that you
> > > think is being offered so there is no confusion later?
> >
> > Informaly, exactly what I said before: Be able to find enough information
> > in the CVS tree which would allow anybody to trace back each change
> > to what was submitted by the author of the change (= patch + comment).
>
> Yup, seems reasonable.

I'm glad we agree on this.

> > (and know I agreed at the moment), but thinking again about this I'm not
> > sure anymore how "sticking the BK changeset key into the delta history"
> > gives us "BK level granularity". From what I understand (but you are the
> > SCM expert not me so I may be missing something) there is exactly
> > one delta per 'trunk' changeset, so if you have a file being modified
> > several times on a branch you will end with one single delta which is
> > the merge of the separate patches. I'm not sure how, by adding several
> > 'BK changeset keys' into the log entry of the merged delta you make
> > one able to resplit the delta later.
>
> First, you have to remember that BK is capturing 96% of the deltas made
> to files. Some of those deltas get clumped into one CVS commit because
> of the flattening of the graph structure. If each delta had the
> changeset key for the BK changeset to which it belonged you could
> split the coarse commit into the sub patches which happened on the
> collapsed branch. You wouldn't get 100% of the information but you'd
> have 96% of it in a way that could be used for debugging, which is what
> I suspect you are after.

Isn't what you're proposing just making more simpler the same thing one
could already do today by a smart analysis of the log message contents ?

Explanation: Looking at a merge changeset log, one can split the log
into parts and then search from the set of files containing deltas in
this changeset the ones having each message part. Based on that,
individual sub-patches can be reconstruct. Yeah, it is prone to errors
if log messages are identical, but it would probably work most of the
time.

I still fail to see how one could get the sub-patches from changes
happening to *a same file*, like I was asking previously. Or is this
part what you call 'the 4% loss' ?

This isn't theoretical at all, it's very practical: suppose I have to
make several changes to, let's say, drivers/usb/Makefile. There are
several logical changes so I make a couple of incremental patches, just
like the kernel developers like.

I send those patches to Greg KH, he puts them into BK, and later on
Linus pulls from this tree. My changes go into the Linus tree using
a BK branch, and finally get exported to CVS as _a single delta_.

Now, suppose one of my patches introduced a problem. How can someone
not using BK isolate the patch which introduced the problem ? All he
can do is to back out the entire set of patches, and the whole point
of having split the patch initialy into logical changes is lost.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-04 22:11:12

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Fri, Feb 04, 2005 at 08:22:48AM -0800, Roland Dreier wrote:

> I don't pretend to understand all the tools or where information is
> being lost, but I did use Stelian's HOWTO to make a subversion kernel
> tree last night. I'm including log information for what happen to be
> the two most recent subversion changesets in my tree. I think we can
> all agree that quite a bit of useful information is lost to users of
> the subversion tree.
>
> ------------------------------------------------------------------------
> r26549 | torvalds | 2005-01-31 07:47:51 -0800 (Mon, 31 Jan 2005) | 208 lines
>
> Merge bk://linux-acpi.bkbits.net/to-linus
> into ppc970.osdl.org:/home/torvalds/v2.6/linux

[ ... very long list merged changesets skipped ... ]

> 2005/01/04 22:57:14-05:00 len.brown
> Merge intel.com:/home/lenb/bk/linux-2.6.10
> into intel.com:/home/lenb/src/26-stable-dev
>
> BKrev: 41fe5327BXOmplstrv49I26qAg7mIA

By the way Larry, wasn't the BKrev supposed to help with finding
the changesets on bkbits ?

Because I'm not sure if it works anymore (or how it is supposed to
work).

For example:

http://linux.bkbits.net:8080/linux-2.6/cset@41fe5327BXOmplstrv49I26qAg7mIA

gives only the merge changeset, obviously not the same as the above
revision.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-04 23:37:06

by Francois Romieu

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Stelian Pop <[email protected]> :
[...]
> Now, suppose one of my patches introduced a problem. How can someone
> not using BK isolate the patch which introduced the problem ? All he
> can do is to back out the entire set of patches, and the whole point
> of having split the patch initialy into logical changes is lost.

Nope: he digs the bk-commit mailing list archives.

For example, from Roland's mail
2005/01/31 01:37:39-05:00 len.brown
2005/01/31 01:35:48-05:00 len.brown
2005/01/31 00:15:20-05:00 len.brown
2005/01/31 00:12:40-05:00 len.brown
[etc.]

$ grep +/^ChangeSet.*2005/01/31.*len.brown ~/Mail/linux/bk-commit/200505
ChangeSet 1.1983.5.2, 2005/01/31 00:15:20-05:00, [email protected]
ChangeSet 1.1938.498.11, 2005/01/31 00:12:40-05:00, [email protected]
ChangeSet 1.1983.5.3, 2005/01/31 01:37:39-05:00, [email protected]
ChangeSet 1.1938.498.12, 2005/01/31 01:35:48-05:00, [email protected]

Same thing as in Roland's mail but the changes are isolated.

--
Ueimor

2005-02-05 19:40:02

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Sat, Feb 05, 2005 at 12:31:53AM +0100, Francois Romieu wrote:

> Stelian Pop <[email protected]> :
> [...]
> > Now, suppose one of my patches introduced a problem. How can someone
> > not using BK isolate the patch which introduced the problem ? All he
> > can do is to back out the entire set of patches, and the whole point
> > of having split the patch initialy into logical changes is lost.
>
> Nope: he digs the bk-commit mailing list archives.
>
> For example, from Roland's mail
> 2005/01/31 01:37:39-05:00 len.brown
> 2005/01/31 01:35:48-05:00 len.brown
> 2005/01/31 00:15:20-05:00 len.brown
> 2005/01/31 00:12:40-05:00 len.brown
> [etc.]
>
> $ grep +/^ChangeSet.*2005/01/31.*len.brown ~/Mail/linux/bk-commit/200505
> ChangeSet 1.1983.5.2, 2005/01/31 00:15:20-05:00, [email protected]
> ChangeSet 1.1938.498.11, 2005/01/31 00:12:40-05:00, [email protected]
> ChangeSet 1.1983.5.3, 2005/01/31 01:37:39-05:00, [email protected]
> ChangeSet 1.1938.498.12, 2005/01/31 01:35:48-05:00, [email protected]
>
> Same thing as in Roland's mail but the changes are isolated.

Interesting, I fergot about those commit mails, thanks for remining
me.

I think those emails could provide the missing piece of the puzzle
and we could generate the missing branches based on them.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-05 23:38:49

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Sat, Feb 05, 2005 at 08:38:48PM +0100, Stelian Pop wrote:
> > Nope: he digs the bk-commit mailing list archives.
> >
> Interesting, I fergot about those commit mails, thanks for remining
> me.
>
> I think those emails could provide the missing piece of the puzzle
> and we could generate the missing branches based on them.

Does that mean you don't need anything from us?
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-06 16:40:55

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Fri, 4 Feb 2005, Larry McVoy wrote:

> > > > > CVS BitKeeper [*]
> > > > > Deltas 235,956 280,212
>
> You need to rethink your math, you are way off. I'll explain it so that
> the rest of the people can see this is just pure FUD.
>
> To make sure this was apples to apples I went to the BK and CVS trees
> which are in lock step and reran the numbers:
>
> CVS BitKeeper % in CVS
> file deltas 210,609 218,742 96%
> changsets 26,603 59,220 44%
>
> You are not factoring out the ChangeSet deltas, which are BK metadata,
> they aren't part of the source tree. If you remove those then the
> difference between CVS and BK revision history is 209K deltas vs
> 221K deltas.
>
> In other words, the CVS tree is missing no more than 4% of the deltas
> to the source files.
>
> READ THAT AGAIN, PLEASE.
>
> The CVS tree has 96% of all the deltas to all your source files. 96%.

Before you start shouting, how about you get your numbers right? Your
first numbers were wrong and the second numbers are still wrong:

$ find -name \*,v -a ! -path ./BitKeeper\* -a ! -name ChangeSet,v | xargs rlog | egrep -e '^[0-9]{4}(/[0-9]{2}){2} [0-9]{2}(:[0-9]{2}){2}' -e '$Logical change 1.[0-9]+$' | wc -l
219242
$ find -name \*,v -a ! -path ./BitKeeper\* -a ! -name ChangeSet,v | xargs rlog | egrep '$Logical change 1.[0-9]+$' | wc -l
187576

That would bring us back to 85% and I leave it to you to find the bug.

> My good friend Stelian would have you believe that you are missing 50%
> of your data when in fact you are missing NONE of your data, you have
> ALL of your data in an almost the identical form.

The 44% number deserves an explanation, which our good friend Larry would
have you believe is completely unimportant and thinking otherwise is just
"FUD". Actually on the contrary it is the more important number.
The 85% number means that the history of a _single_ file can be restored
with a precision of 85% via bkcvs. If one is only interested in the
history of a single file, that 85% might be sufficient. OTOH a change to
the kernel is rarely just a change to a single file. A patch can consist
of changes to multiple files and this relationship between multiple single
file changes can only be restored with a precision of 44% via bkcvs.
IOW if one wants to not just look at some random file history, but
actually restore and use some of it, one can only get 44% of the 59220
snapshots of kernel history.

> What Stelian is complaining about is the patch set which is easily
> extractable from CVS is at a coarser granularity than the patch set
> extractable from BK. That's true but so what? Before BK you only
> a handful of patches between releases, now you have thousands.

Hmm, that argument is not exactly original anymore. If Linus would say,
that kernel history has to be exclusively recorded via bk and that the
history available via bkcvs is the maximum that must be made available to
other users, you would have a point.
So it's quite legitimate to at least ask occasionally how the 44%
granularity can be improved. CVS is obviously not really capable of it,
but RCS on which CVS is based on is. I attached an example script which
can convert the history of a bk file into RCS. The only thing that is
missing is to 1) generate a list revisions with their parents, 2) checkout
a given revision, 3) generate the log for a given revision. (*) I'm quite
sure that's trivial to do for you and this only took me a few hours
including testing and cleanup, it only takes a little more work to make
CVS happy and allow incremental updates, so your "3 month" number is
absolutely ridiculous.
Any bk user can now easily prove or disprove this by completing my example
script, I think the "bk prs" man page should be very helpful here. If you
want to complete it into your very own bk2cvs script, I'd be happy to
help.

bye, Roman

(*) In case anyone thinks this is simple without bk, try to find the
parents for this changeset
http://linux.bkbits.net:8080/linux-2.6/[email protected]

Attachments:

conv.rb (3.79 kB)

2005-02-06 17:39:19

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Ahh, Roman, always a joy to hear from you.

> > CVS BitKeeper % in CVS
> > file deltas 210,609 218,742 96%
> > changsets 26,603 59,220 44%
> >
> > In other words, the CVS tree is missing no more than 4% of the deltas
> > to the source files.
> >
> > READ THAT AGAIN, PLEASE.
> >
> > The CVS tree has 96% of all the deltas to all your source files. 96%.
>
> Before you start shouting, how about you get your numbers right? Your
> first numbers were wrong and the second numbers are still wrong:
>
> $ find -name \*,v -a ! -path ./BitKeeper\* -a ! -name ChangeSet,v | xargs rlog | egrep '$Logical change 1.[0-9]+$' | wc -l
> 187576

Bzzt. You forgot all the intial deltas which are not marked with the
logical change comment. And just to double check my logic I tried it
with rlog (much slower but whatever):

$ /tmp/linux-2.5-cvs find linux-2.5/ -name '*,v' |
xargs rlog | grep 'total revisions' |
awk 'BEGIN { n = 0 }; { n = n + $NF }; END { print n }'
237338
$ /tmp/linux-2.5-cvs perl REVS
files=22966 revs=237338

Imagine that, the numbers match perfectly.

It's always possible I've made a mistake but you really ought to check
your work a little bit before making false claims. It's trivial to do
what I did which is run the script over a single file and hand verify
that it is correct.

> > My good friend Stelian would have you believe that you are missing 50%
> > of your data when in fact you are missing NONE of your data, you have
> > ALL of your data in an almost the identical form.
>
> [Questionable complaints that he isn't getting enough information]

First, you can get all the granularity that you want from bkbits.net.
Find the file you want, find the revision, and go backwards to the
changeset. It takes you a few clicks to do that.

Second, your numbers are just plain wrong. You have 96% of the data
and anyone can validate that.

Third, as someone else pointed out you have the bk-commits list so you
have an archive of the patches and the commits.

Fourth, it is your choice to not use BitKeeper because you want to compete
with the people who are helping you. It's not that unreasonable that
you find yourself at something of a disadvantage because of that choice.
And the disadvantage is very slight as has been shown. You can argue
all you want about the amount of disadvantage but it is your choice that
has placed you in that position.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-07 01:45:40

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Sun, 6 Feb 2005, Larry McVoy wrote:

> > $ find -name \*,v -a ! -path ./BitKeeper\* -a ! -name ChangeSet,v | xargs rlog | egrep '$Logical change 1.[0-9]+$' | wc -l
> > 187576
>
> Bzzt. You forgot all the intial deltas which are not marked with the
> logical change comment. And just to double check my logic I tried it
> with rlog (much slower but whatever):
>
> $ /tmp/linux-2.5-cvs find linux-2.5/ -name '*,v' |
> xargs rlog | grep 'total revisions' |
> awk 'BEGIN { n = 0 }; { n = n + $NF }; END { print n }'
> 237338
> $ /tmp/linux-2.5-cvs perl REVS
> files=22966 revs=237338
>
> Imagine that, the numbers match perfectly.

Bzzt. Larry, I will make this one very easy, so that even you can follow
it. Let's take a simple file:

$ rlog REPORTING-BUGS,v | grep 'total revisions'
total revisions: 3; selected revisions: 3
$ rlog REPORTING-BUGS,v | egrep '$Logical change 1.[0-9]+$'
(Logical change 1.31)
(Logical change 1.3)

Now please check http://linux.bkbits.net:8080/linux-2.6/hist/REPORTING-BUGS
and tell me which number is correct.

> It's always possible I've made a mistake but you really ought to check
> your work a little bit before making false claims. It's trivial to do
> what I did which is run the script over a single file and hand verify
> that it is correct.

Ditto.

> > [Questionable complaints that he isn't getting enough information]

I challenge you to quote a single complaint I did in this mail.
All I did was adding the missing facts, which you like to forget to
mention and giving anyone the necessary information to verify them.

> First, you can get all the granularity that you want from bkbits.net.
> Find the file you want, find the revision, and go backwards to the
> changeset. It takes you a few clicks to do that.

Once again our dear Larry tells only half the story. It's true that single
file changes (which are in bkcvs are only available to 85%) are completely
available via bkweb to 100%. The problem is again how all these changes
relate to each other (the 44% number for bkcvs).
To give everyone an idea what this means, let's take a bit more
complicated file like http://linux.bkbits.net:8080/linux-2.6/hist/MAINTAINERS
Now try to figure out what all the revisions saying "Auto merged" actually
merge. It's actually possible, but one has to use heuristics which can
fail, so while one can restore most of the history of a single file, it's
not reliable anymore as soon as the branching becomes more complex.
The real trouble starts if one wants to know how all these files relate to
each other. Right now the bk tree contains over 59000 snapshots of how the
kernel looked at a specific point. To restore these snapshots again one
must apply the changesets in the correct order (this is equally true for
the commit mails), IOW one must know the parent revisions of a changeset
to really restore the history and to not just look at a part of it, but
bkweb unfortunately doesn't provide this information in full. Within a
branch the patch order is of course obvious, but one must also know where
a branch starts and ends, which is unfortunately not as obvious as one
might think. E.g. SCCS files are limited to a single branch level and bk
inherited this, so a branch of 1.2.3.4 is not 1.2.3.4.1.1 but something
like 1.2.4.1.
Larry, so far I have only stated facts, which are easily verifiable, how
about you come up with some facts of your own to prove me wrong? If I'm
just "complaining" and all the information is out there, it should be easy
for you to do...

> Fourth, it is your choice to not use BitKeeper because you want to compete
> with the people who are helping you. It's not that unreasonable that
> you find yourself at something of a disadvantage because of that choice.
> And the disadvantage is very slight as has been shown. You can argue
> all you want about the amount of disadvantage but it is your choice that
> has placed you in that position.

Well, I'm not the one who claimed "We don't do lockins. Period."
I'm just trying to figure out what that means...

bye, Roman

2005-02-07 02:11:04

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

> Bzzt. Larry, I will make this one very easy, so that even you can follow
> it. Let's take a simple file:
>
> $ rlog REPORTING-BUGS,v | grep 'total revisions'
> total revisions: 3; selected revisions: 3
> $ rlog REPORTING-BUGS,v | egrep '$Logical change 1.[0-9]+$'
> (Logical change 1.31)
> (Logical change 1.3)

Exactly. The second one shows 2 revs, which is what you counted, and
the first shows 3 which is what is actually there.

See, all you have to do is look:

slovax /tmp/linux-2.5-cvs/linux-2.5 rlog REPORTING-BUGS,v

RCS file: REPORTING-BUGS,v
Working file: REPORTING-BUGS
head: 1.3
branch:
locks: strict
access list:
symbolic names:
keyword substitution: o
total revisions: 3; selected revisions: 3
description:
BitKeeper to RCS/CVS export
----------------------------
revision 1.3
date: 2002/02/05 18:07:03; author: patch; state: Exp; lines: +3 -4
Import patch diff

(Logical change 1.31)
----------------------------
revision 1.2
date: 2002/02/05 17:40:40; author: torvalds; state: Exp; lines: +58
-0
(Logical change 1.3)
----------------------------
revision 1.1
date: 2002/02/05 17:40:40; author: torvalds; state: Exp;
Initial revision

As for your pointer to the revision history on the web which shows 2
revs, you just don't understand BK. There is a 1.0 delta in every file
that we hide by default. That would be the matching delta to the RCS
1.1 delta.

> > Fourth, it is your choice to not use BitKeeper because you want to compete
> > with the people who are helping you. It's not that unreasonable that
> > you find yourself at something of a disadvantage because of that choice.
> > And the disadvantage is very slight as has been shown. You can argue
> > all you want about the amount of disadvantage but it is your choice that
> > has placed you in that position.
>
> Well, I'm not the one who claimed "We don't do lockins. Period."
> I'm just trying to figure out what that means...

Hey, Roman, the statement above stands. You made the choice that you want
to go write a competing system. If you hadn't you could just use BK and
stop whining. Since you have made that choice, which is your right,
how about you produce your competing system? And stop whining that
we aren't giving you enough help. What is that you say? It's hard?
It's way harder if we don't give you a roadmap? Well gosh darn, that
must really suck for you. I'm really sorry that you can't figure it out
without our help but that's sort of the whole point, isn't it?
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-07 02:16:55

by Al Viro

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Mon, Feb 07, 2005 at 02:45:22AM +0100, Roman Zippel wrote:

[same wankfest]

*plonk*

If you ever need to send me mail - send it directly. Anything from you
to l-k will be handled by /dev/null here. And it would better on saner
topics not involving your crusades, TYSoFsckingM.

2005-02-08 14:57:31

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Sun, 6 Feb 2005, Larry McVoy wrote:

> [Larry continues to pull numbers out of his arse.]

Out of sympathy to Al I cut the crap short. If you (or anyone else) really
want to know, contact me privately.
The 85% number is of secondary interest only anyway, my (undisputed)
argumentation still stands, why the 44% is more important.

> > Well, I'm not the one who claimed "We don't do lockins. Period."
> > I'm just trying to figure out what that means...
>
> Hey, Roman, the statement above stands. You made the choice that you want
> to go write a competing system. If you hadn't you could just use BK and
> stop whining. Since you have made that choice, which is your right,
> how about you produce your competing system? And stop whining that
> we aren't giving you enough help. What is that you say? It's hard?
> It's way harder if we don't give you a roadmap? Well gosh darn, that
> must really suck for you. I'm really sorry that you can't figure it out
> without our help but that's sort of the whole point, isn't it?

1. Is the kernel history locked into bk?
Fact is that exporting the history from bk into a different system
requires extensive scm knowledge, which makes it rather unlikely that a bk
user can do this by himself. He also has various resaons to not ask for
such help, from that he just doesn't care to that he is afraid bk support
might be pulled.
This leaves the other users, which either can't or want to use bk, with
a reduced kernel history (as I have shown in the previous mails). The
practical consequence of this is that a majority of the kernel history is
locked into bk right now, with no way in sight to get it out of there.

2. Larry, do you really thought that kernel hackers have no other
interests than kernel hacking? Do you really thought that all kernel
hackers want to keep the kernel history forever in bk?
I never really wanted to use bk in first place and at that time the
license was annoying but I would have at least used it to push changes to
other bk users. The main reason I don't use bk was and is technical, it
simply doesn't do what I need. The license change just made it completely
unacceptable to even bother. So while I don't care much about bk, I care
of course about the kernel data and only this really sparked my interest
into scm systems. Isn't it ironic that without that license change I
likely wouldn't know as much about scm systems as I do now?
Anyway, if I wanted to write a simple bk clone, I could have done so
already easily without your help, but why should I do so if it's not
what I need in first place? Sorry to disappoint your ego, but the world
doesn't center around bk. Did you know, there are other scm systems out
there? Once one studied a few of them, one basically also knows how bk
works and it certainly helps to put your "facts" into perspective.

Assuming I wrote such a "competing system". What would it change? Could I
or anyone else then import the kernel data into it? How would a user
proceed in order to convert his data, so he can give such a system a real
test run?
Well, from your perspective I'm propably already working on a "competing
system", from mine I'm just playing around with some ideas from time to
time, but you certainly keep encouraging me to keep on it and to develop
them into something usable. It's not really a problem to continue using
the bkcvs kernel tree or any other large cvs tree for testing, the bk
data doesn't contain any information I don't know already.

Finally the main question remains (and it will come up over and over
again), how can I or anybody else get my data back out of bk? How can you
claim the kernel history is not property of BM and OTOH the publically
available data is provably not complete? You can continue to use my
interest in scm systems to discredit me as much as you want, but I doubt
we will ever find a suitable candidate, AFAICT anyone only remotely
suggesting using the data in an out of bk context got pretty much the same
treatment from you. Larry, could you please explain, how you exactly
intend to keep the evil scm developers out and at the same time leave
users a choice in their tools once they put their data into bk?

bye, Roman

2005-02-08 15:19:06

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Sigh. Roman, I started to write a reply but in reading over the thread
I realized you are just grinding your ax and have nothing new to say.
Sorry, go bother someone else, I'm busy.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-08 15:23:22

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Tue, Feb 08, 2005 at 03:57:14PM +0100, Roman Zippel wrote:

> On Sun, 6 Feb 2005, Larry McVoy wrote:
>
> > [Larry continues to pull numbers out of his arse.]
>
> Out of sympathy to Al I cut the crap short. If you (or anyone else) really
> want to know, contact me privately.
> The 85% number is of secondary interest only anyway, my (undisputed)
> argumentation still stands, why the 44% is more important.

Indeed. This number is what started the entire thread (again).

> > > Well, I'm not the one who claimed "We don't do lockins. Period."
> > > I'm just trying to figure out what that means...
> >
> > Hey, Roman, the statement above stands. You made the choice that you want
> > to go write a competing system. If you hadn't you could just use BK and
> > stop whining. Since you have made that choice, which is your right,
> > how about you produce your competing system? And stop whining that
> > we aren't giving you enough help. What is that you say? It's hard?
[...]

> This leaves the other users, which either can't or want to use bk, with
> a reduced kernel history (as I have shown in the previous mails). The
> practical consequence of this is that a majority of the kernel history is
> locked into bk right now, with no way in sight to get it out of there.

Sadly, the BK license does not prohibit only 'writing competing systems',
but also people who happen to work with people who work on such
products.

Since Red Hat, SuSE, IBM etc do a lot of work on such products, does
this mean that all the kernel hackers working for those companies have
a commercial BK license ?

A second point which strikes me is this clause in the new license:
(g) No reverse engineering: You may not yourself and may not
permit or enable anyone to: [...]
(iii) provide access to
the metadata created and managed by BitKeeper to any person
or entity which is not licensed to use the BitKeeper Soft-
ware.

Tell me now how it is possible under this license to post patches
generated by BK on lkml ?

Stelian.
--
Stelian Pop <[email protected]>

2005-02-08 15:36:03

by Catalin Marinas

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Roman Zippel <[email protected]> wrote:
> Did you know, there are other scm systems out
> there? Once one studied a few of them, one basically also knows how bk
> works and it certainly helps to put your "facts" into perspective.

On the same line of ideas, a script that some people might find useful:

http://wiki.gnuarch.org/moin.cgi/BKCVS_20to_20Arch_20Script_20for_20Linux_20Kernel

Catalin

2005-02-08 15:42:25

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Sat, Feb 05, 2005 at 03:38:41PM -0800, Larry McVoy wrote:

> On Sat, Feb 05, 2005 at 08:38:48PM +0100, Stelian Pop wrote:
> > > Nope: he digs the bk-commit mailing list archives.
> > >
> > Interesting, I fergot about those commit mails, thanks for remining
> > me.
> >
> > I think those emails could provide the missing piece of the puzzle
> > and we could generate the missing branches based on them.
>
> Does that mean you don't need anything from us?

If the kernel development was linear, it could be enough.

But with the branch'n'merge nature of BK it is hard if not impossible
to extract enough data from those patches (I looked at the history
of the last 2 months and I had several patch conflits due to a
changeset being included on several branches which were merged later,
several mails whose date was not the same as the changeset[*]).

What you could make available in the bkcvs export would be, for each
changeset (both for the changesets and for the merged changesets),
include three BKRevs: the changeset's one, the changeset's parent one,
and the changeset's merge parent one.

That information could be used to reconstruct the entire tree,
using either bk-commit-head (preferred) or bkbits, provided you
put the BKRev: tag into the bk-commit-head posts too.

Technicaly speaking this should be very easy for you to implement.

What do you think ?

Stelian.

*: for example this one:

ChangeSet 1.2030, 2005/02/02 13:52:52-08:00, [email protected]

Merge bk://drm.bkbits.net/drm-linus
into ppc970.osdl.org:/home/torvalds/v2.6/linux

is translated in CVS into:

1.26549
date 2005.02.03.00.21.37; author torvalds; state Exp;

...

1.26549
log
@Merge bk://drm.bkbits.net/drm-linus
into ppc970.osdl.org:/home/torvalds/v2.6/linux

--
Stelian Pop <[email protected]>

2005-02-08 15:47:59

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

> Since Red Hat, SuSE, IBM etc do a lot of work on such products, does
> this mean that all the kernel hackers working for those companies have
> a commercial BK license ?

Licenses are a lot like signing a lease. Things are negotiable and
we're reasonable people (contrary to public opinion).

> Tell me now how it is possible under this license to post patches
> generated by BK on lkml ?

bk export -tpatch is just fine. So is bk -r diffs -up. Done all the
time, no license violation, no problem except apparently for you.

I think your concern is based on the desire to create a competing
system. That skates you right up agains the license restrictions but
those restrictions are simply not a problem for people who are just
trying to get their job done.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-08 15:58:07

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Tue, 8 Feb 2005 15:57:14 +0100 (CET), Roman Zippel
<[email protected]> wrote:
> 1. Is the kernel history locked into bk?

Earlier Larry said this:

On Thu, 3 Feb 2005 12:20:49 -0800, Larry McVoy <[email protected]> wrote:
> > Speaking from the out-BK point of view, what would really be nice
> > is better granularity in the CVS export (a 1-1 changeset to CVS commit
> > mapping). I know this involves playing with CVS branches and could
> > be a bit tricky but should be doable.
>
> I have two problems with this request:
>
> - The idea that the granularity in CVS is unreasonable is pure
> nonesense. Here's the data as of this email:
>
> CVS BitKeeper [*]
> Deltas 235,956 280,212
>
> - It is not at all an easy thing to do in CVS, we looked at it and
> guessed it is about 3 man months of work.
>
> So let's see what's reasonable. In order for you to get the last 16%
> of the granularity, which you need because you want to compete with us,
> you'd like us to do another 3 man months of work. What would you say if
> you were me in this situation?

Roman, if you want this so bad why don't you just pay Larry for the
three month's work? It's just not reasonable to ask someone to do work
for free that the only purpose of is to help someone clone their
system. He's never said he won't make the changes, he just said he
won't do the work for free.

Don't bother with a big response about how all of the revision history
should be available for free and that you shouldn't have to pay to get
it. For historical reasons we're in the current position and you have
a solution which is to pay to have the work done.

--
Jon Smirl
[email protected]

2005-02-08 15:58:57

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Tue, Feb 08, 2005 at 04:43:44PM +0100, Stelian Pop wrote:
> On Sat, Feb 05, 2005 at 03:38:41PM -0800, Larry McVoy wrote:
>
> > On Sat, Feb 05, 2005 at 08:38:48PM +0100, Stelian Pop wrote:
> > > > Nope: he digs the bk-commit mailing list archives.
> > > >
> > > Interesting, I fergot about those commit mails, thanks for remining
> > > me.
> > >
> > > I think those emails could provide the missing piece of the puzzle
> > > and we could generate the missing branches based on them.
> >
> > Does that mean you don't need anything from us?
>
> If the kernel development was linear, it could be enough.
>
> But with the branch'n'merge nature of BK it is hard if not impossible
> to extract enough data from those patches (I looked at the history
> of the last 2 months and I had several patch conflits due to a
> changeset being included on several branches which were merged later,
> several mails whose date was not the same as the changeset[*]).
>
> What you could make available in the bkcvs export would be, for each
> changeset (both for the changesets and for the merged changesets),
> include three BKRevs: the changeset's one, the changeset's parent one,
> and the changeset's merge parent one.
>
> That information could be used to reconstruct the entire tree,
> using either bk-commit-head (preferred) or bkbits, provided you
> put the BKRev: tag into the bk-commit-head posts too.
>
> Technicaly speaking this should be very easy for you to implement.
>
> What do you think ?

I think you are dreaming. You've gone from wanting enough information
to supposedly debug your source tree to being explicit about wanting to
recreate the entire BK history in a different system. The former is a
reasonable request, I suppose, but the latter is just a blatent request
for us to help debug and stress test a competing system.

The answer is no, that's a clear violation of the license. The deal is
that you get your data and you get *your* metadata. Not the metadata
created by BitKeeper. You get what bk export -tpatch gives you, the
diffs and the checkin comments.

I'm quite unhappy you keep asking for this, it forces me into the
position of being the bad guy. You need to understand that we can
only take on so much risk and giving you BK for free was a huge amount
of risk. Giving you BK, and the right to use it to create a different
system, and/or the right to use the BK metadata to create a different
system is way too much risk. I'm sorry but we have to draw the line
somewhere. Could you please just obey the license and stop this
thread? Is that so hard? I don't come here every month and ask for
the GPL to be removed from some driver, that's essentially what you are
doing and I think pretty much everyone is sick of it.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-08 16:22:02

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Tue, 8 Feb 2005, Jon Smirl wrote:

> Roman, if you want this so bad why don't you just pay Larry for the
> three month's work?

Why should I pay for something, I could easily do myself in less time?

bye, Roman

2005-02-08 17:01:53

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

> Roman, if you want this so bad why don't you just pay Larry for the
> three month's work? It's just not reasonable to ask someone to do work
> for free that the only purpose of is to help someone clone their
> system.

That's an interesting idea, thanks, but we'd need both money and a new
hire. There is an opportunity cost with using up an engineer and we're
swamped with work. As I've said before, if you really like this sort of
stuff we'd like to hire you. Yeah, it's not as politically correct as
working on 100% GPLed source, I can't help that, but it is technically
challenging, far more so than most people realize and that makes it
fun.

If we get a new hire from the kernel list I'll stick the changeset markers
into the CVS tree so you can group the patches, I can see where that
might be helpful for debugging.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-08 17:18:11

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Tue, 8 Feb 2005, Larry McVoy wrote:

> I think you are dreaming. You've gone from wanting enough information
> to supposedly debug your source tree to being explicit about wanting to
> recreate the entire BK history in a different system. The former is a
> reasonable request, I suppose, but the latter is just a blatent request
> for us to help debug and stress test a competing system.
>
> The answer is no, that's a clear violation of the license.

You do realize what this practically means?

bye, Roman

2005-02-08 17:19:18

by Valdis Klētnieks

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Tue, 08 Feb 2005 17:15:53 +0100, Roman Zippel said:

> Why should I pay for something, I could easily do myself in less time?

Why does the phrase "Shut up and code..." suddenly wander through my mind???

Attachments:

(No filename) (226.00 B)

2005-02-08 17:24:02

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Tue, 8 Feb 2005 [email protected] wrote:

> > Why should I pay for something, I could easily do myself in less time?
>
> Why does the phrase "Shut up and code..." suddenly wander through my mind???

You didn't really read what I explained in detail, didn't you?

bye, Roman

2005-02-08 18:16:50

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Tue, Feb 08, 2005 at 06:17:30PM +0100, Roman Zippel wrote:
> Hi,
>
> On Tue, 8 Feb 2005, Larry McVoy wrote:
>
> > I think you are dreaming. You've gone from wanting enough information
> > to supposedly debug your source tree to being explicit about wanting to
> > recreate the entire BK history in a different system. The former is a
> > reasonable request, I suppose, but the latter is just a blatent request
> > for us to help debug and stress test a competing system.
> >
> > The answer is no, that's a clear violation of the license.
>
> You do realize what this practically means?

It means exactly what it says, we're not going to help you work on
a competing system. You are welcome to do that on your own, you are
welcome to download all the patches you want from bkbits.net and figure
out how to place them in your system, but we aren't going to help you
any further than that, it doesn't make business sense for us to do so.

We have to do things which make business sense because it is our business
which provides you with the technology and the infrastructure you use.
It costs money to do that, squeeze off the supply of money and the free
ride goes away.

I understand that you are unhappy with BK not being open source but
unhappiness doesn't pay the bills. We are providing you with a great
tool, good support, and excellent infrastructure. Not only do you not
have a replacement for all of that you are actively trying to damage
our business which provides you with all of that. That may force an
open source answer but it would also do a lot of damage to the kernel
development in the process.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-08 18:52:27

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Tue, 8 Feb 2005, Larry McVoy wrote:

> > > I think you are dreaming. You've gone from wanting enough information
> > > to supposedly debug your source tree to being explicit about wanting to
> > > recreate the entire BK history in a different system. The former is a
> > > reasonable request, I suppose, but the latter is just a blatent request
> > > for us to help debug and stress test a competing system.
> > >
> > > The answer is no, that's a clear violation of the license.
> >
> > You do realize what this practically means?
>
> It means exactly what it says, we're not going to help you work on
> a competing system.

Does that really matter? Above also means it couldn't be exported into the
perfect scm system, because it then could be reexported into some other
lame scm system and so would help their development.

> You are welcome to do that on your own, you are
> welcome to download all the patches you want from bkbits.net and figure
> out how to place them in your system,

Hmm, it seems you didn't bother reading what I wrote either.

> We have to do things which make business sense because it is our business

Does that include making empty promises?

bye, Roman

2005-02-09 00:08:14

by Theodore Ts'o

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Roman,

I suspect the most of the folks on LKML are sick and tired of
this particular thread. Could you (and Larry) please take this
off-line, please? Everyone who has an opinion on this matter is not
likely to change their minds, so continued rehashing of old arguments
is just noise IMHO.

Me, I'm very happy using BitKeeper and am glad that Larry and
his team of merry men have graciously provided that tool for us to
use. In other situations I use quilt, when it is the right tool for
the job, and I am grateful to Andrew and Andreas and all those who
have provided that tool. Other times I have played with svn because
that was what a project leader decided to use, and so thanks to the
svn developers.... and as far as CVS is concerned, I suppose I'll
grudgingly have to give thanks to them too, being pioneers of
distributed SCM --- but these days, I'm glad I rarely have to use
CVS. :-)

I don't know how many years it was before people decided to
give up on the emacs vs. vi wars, but can we please put a more hasty
end to the bk license flamewars? Many thanks,

- Ted

2005-02-09 02:05:40

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Tue, 8 Feb 2005, Theodore Ts'o wrote:

> I don't know how many years it was before people decided to
> give up on the emacs vs. vi wars, but can we please put a more hasty
> end to the bk license flamewars? Many thanks,

It's not really the same, if it would be just about personal preferences,
I would be the first to shut up (unless someone buys the beer :) ).
If people want to use bk because it does the job for them, I'm happy
for them.
The current problem is more serious and I want that bk users to understand
that. A large part of kernel history is currently practically locked into
bk. bk isn't doing what I need, so naturally I'm looking for alternatives,
but I don't have the freedom to take my data and try it with some other
tool. Was that really part of the deal when bk was introduced that I'm
denied of this freedom?
If bk users think this alright, I'd really like to hear from them how this
fits into a free software project. With every other free software project
it's possible to just take the data and import into something else, why
isn't this possible with Linux?

bye, Roman

2005-02-09 02:24:39

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, 9 Feb 2005 03:05:18 +0100 (CET), Roman Zippel
<[email protected]> wrote:
> The current problem is more serious and I want that bk users to understand
> that. A large part of kernel history is currently practically locked into
> bk. bk isn't doing what I need, so naturally I'm looking for alternatives,
> but I don't have the freedom to take my data and try it with some other
> tool. Was that really part of the deal when bk was introduced that I'm
> denied of this freedom?

Larry has said he will do the work you want if you pay him. This is
not lack of freedom it is lack of cash. So if you really believe in
this cough up the dough for Larry, and then only if he refuses to do
the work after being paid do you have valid reasons to complain. This
is not an argument about freedom, it's trying to get someone else to
do work for you for free. Start a PayPal account and get all of the
unhappy people to contribute. I'll chip in $10 just so that I don't
have to read these rants anymore.

--
Jon Smirl
[email protected]

2005-02-09 02:36:49

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Tue, 8 Feb 2005, Jon Smirl wrote:

> Larry has said he will do the work you want if you pay him.

Usually I'm all for giving the benefit of the doubt, but in this case I'd
prefer to know exactly, what I would get for the money.
But as I said by now I know enough about this that I can do the job
myself, all I need is a cooperating bk user.

bye, Roman

2005-02-09 02:41:33

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, 9 Feb 2005 03:35:37 +0100 (CET), Roman Zippel
<[email protected]> wrote:
> Hi,
>
> On Tue, 8 Feb 2005, Jon Smirl wrote:
>
> > Larry has said he will do the work you want if you pay him.
>
> Usually I'm all for giving the benefit of the doubt, but in this case I'd
> prefer to know exactly, what I would get for the money.

Write up a proposal of what you need. Send it to Larry and ask for a
quote. Larry will probably even help you fill in things you missed in
the proposal. Come to an agreement on terms for the proposal. Raise
the cash, send it to Larry, wait for the work to be finished. Now you
have your solution.

--
Jon Smirl
[email protected]

2005-02-09 02:43:42

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, Feb 09, 2005 at 03:35:37AM +0100, Roman Zippel wrote:
> Hi,
>
> On Tue, 8 Feb 2005, Jon Smirl wrote:
>
> > Larry has said he will do the work you want if you pay him.
>
> Usually I'm all for giving the benefit of the doubt, but in this case I'd
> prefer to know exactly, what I would get for the money.
> But as I said by now I know enough about this that I can do the job
> myself, all I need is a cooperating bk user.

Translation: someone else who is welling to violate the BK license
because I don't want to take the risk.

Nice, Roman. All I need is a cooperating third party who is willing to
give me your code under a different (albeit invalid) license.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-09 02:48:17

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Tue, 8 Feb 2005, Larry McVoy wrote:

> Nice, Roman. All I need is a cooperating third party who is willing to
> give me your code under a different (albeit invalid) license.

Short version: Bullshit.
Long version: See previous mails.

bye, Roman

2005-02-09 02:58:48

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Tue, 8 Feb 2005, Jon Smirl wrote:

> Write up a proposal of what you need. Send it to Larry and ask for a
> quote. Larry will probably even help you fill in things you missed in
> the proposal. Come to an agreement on terms for the proposal. Raise
> the cash, send it to Larry, wait for the work to be finished. Now you
> have your solution.

I appreciate your optimism, but think about it: why would he insist on
doing it himself and prefers to call me thief instead, if we were talking
about the same kind of "exporting the complete kernel history"?

bye, Roman

2005-02-09 03:05:11

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, 9 Feb 2005 03:57:37 +0100 (CET), Roman Zippel
<[email protected]> wrote:
> Hi,
>
> On Tue, 8 Feb 2005, Jon Smirl wrote:
>
> > Write up a proposal of what you need. Send it to Larry and ask for a
> > quote. Larry will probably even help you fill in things you missed in
> > the proposal. Come to an agreement on terms for the proposal. Raise
> > the cash, send it to Larry, wait for the work to be finished. Now you
> > have your solution.
>
> I appreciate your optimism, but think about it: why would he insist on
> doing it himself and prefers to call me thief instead, if we were talking
> about the same kind of "exporting the complete kernel history"?

Write up a proposal and send it in for a quote. See if Larry is a man
of his word and quotes you a fair price for doing the work. Post the
proposal and the quote. Then see if there is enough interest to fund
the project. There are probably ten tech news web sites that will make
a lead story out of this. Negotiating a true settlement is much better
than sending thousand of useless emails that achieve nothing.

--
Jon Smirl
[email protected]

2005-02-09 03:40:42

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, Feb 09, 2005 at 03:47:49AM +0100, Roman Zippel wrote:
> Hi,
>
> On Tue, 8 Feb 2005, Larry McVoy wrote:
>
> > Nice, Roman. All I need is a cooperating third party who is willing to
> > give me your code under a different (albeit invalid) license.
>
> Short version: Bullshit.
> Long version: See previous mails.

Let's try that again:

Short version: let's violate a license.
Long version: let's violate a license and see if we get in trouble.

Roman, we can do this forever. Every time you cause a fuss more people
become aware of how unreasonable you are being. That's fine with me,
come on back and flame some more.

Our position is clear: we are trying to do the right thing for Linux,
we've developed a business model which works, we're committed to helping
Linux. Yup, you're right, we aren't helping *you* in a way you like.
On the other hand, we are helping you, we've been doing it for years,
and we've been doing it in spite of you being a complete jerk about it.
And we'll keep doing it no matter how much of a jerk you want to
be because, surprise!, it isn't really about you. It's about Linux.
And in so far as Linux is concerned, you are even less relevant than we
are and we aren't that big of a deal. So Linux wins, you get to complain,
life goes on.

You know, you could change all this. Instead of complaining that we
are somehow hurting you, which virtually 100% of the readers know is
nonsense, you could be producing an alternative answer which is better.
Instead of doing that you keep whining that we aren't helping you do that
and that isn't fair. Hey, come on, get a grip. I'm not out hear whining
that the GPL isn't fair to my agenda even though it frequently isn't.
I work around it if it is important. If not using BK is important to you
then don't use it. If having a free software SCM system is important to
you then create one. But whining that we aren't helping you is beyond
pathetic and I am 100% positive I'm not alone in that opinion.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-09 05:20:43

by Kevin Puetz

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Olaf Dietsche wrote:

> Stelian Pop <[email protected]> writes:
>
>> I must test this...), plus 600 MB per working copy.
>
> If you use svk <http://svk.elixus.org/> for the client side, there's
> (almost?) no overhead.
>
> Regards, Olaf.

erm, svk is cool and all, but it keeps a local repository mirror (not
necessarily full I suppose, but usually it is). So it's *much* heavier on
the client side than normal svn. Pays off in several ways, but just because
it keeps it's weight in the depot folder instead of the wc folder doesn't
make it ligher (unless you use several wc's I suppose).

2005-02-09 05:48:32

by Gene Heskett

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Tuesday 08 February 2005 21:57, Roman Zippel wrote:
>Hi,
>
>On Tue, 8 Feb 2005, Jon Smirl wrote:
>> Write up a proposal of what you need. Send it to Larry and ask for
>> a quote. Larry will probably even help you fill in things you
>> missed in the proposal. Come to an agreement on terms for the
>> proposal. Raise the cash, send it to Larry, wait for the work to
>> be finished. Now you have your solution.
>
>I appreciate your optimism, but think about it: why would he insist
> on doing it himself and prefers to call me thief instead, if we
> were talking about the same kind of "exporting the complete kernel
> history"?
>
>bye, Roman

Roman, Larry has a perfectly valid reason to call you a jerk, because
you are being one, and have been one for at least 50% of your posts
on this list over the last 2 years or more. Right this instant my
most fervent wish is for you to go get a job at M$ or something, and
leave us the hell alone. Either put up the $$ for Larry to take time
from other profitable endeavors and do it, or STFU.

Jon, the only way I'd contribute a tenner to match your is if somebody
held it in escrow till the job is done.

--
No Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.32% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attorneys please note, additions to this message
by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.

2005-02-09 07:06:57

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Feb 8, 2005, [email protected] (Larry McVoy) wrote:

> I think you are dreaming. You've gone from wanting enough information
> to supposedly debug your source tree to being explicit about wanting to
> recreate the entire BK history in a different system.

> The answer is no, that's a clear violation of the license.

So you've somehow managed to trick most kernel developers into
granting you power over not only the BK history, in such a way that
anyone willing to extract all the information available from the BK
repository and share it with others is forbidden from doing so by the
license?

> I'm quite unhappy you keep asking for this, it forces me into the
> position of being the bad guy. You need to understand that we can
> only take on so much risk and giving you BK for free was a huge amount
> of risk.

I'd much rather you didn't ``give´´ it at all, then people wouldn't be
locked into it, and the community might have come up with something as
efficient earlier with the extra push. Now we're faced with choices
such as keeping on with a presumably technically-good but non-Free
software, or switching to a Free and hopefully as-good software and
losing history. Clever trick, indeed!

> Giving you BK, and the right to use it to create a different
> system, and/or the right to use the BK metadata to create a different
> system is way too much risk.

Is it even legal to attempt to stop someone from sharing information
that is not owned by you? Or did you get to own all of the metadata
in the repository?

> I don't come here every month and ask for the GPL to be removed from
> some driver, that's essentially what you are doing

I don't think so. What he's doing is more along the lines of `hey,
this allegedly-GPLed driver contains a piece of binary firmware whose
source code is not there, could we either replace it with actual GPLed
code or remove the driver?

--
Alexandre Oliva http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org}

2005-02-09 08:58:43

by Miles Bader

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Kevin Puetz <[email protected]> writes:
>> If you use svk <http://svk.elixus.org/> for the client side, there's
>> (almost?) no overhead.
>>
>> Regards, Olaf.
>
> erm, svk is cool and all, but it keeps a local repository mirror (not
> necessarily full I suppose, but usually it is). So it's *much* heavier on
> the client side than normal svn. Pays off in several ways, but just because
> it keeps it's weight in the depot folder instead of the wc folder doesn't
> make it ligher (unless you use several wc's I suppose).

Hmmm, I thought that several other systems had similar (or worse)
overhead -- most notably that bk and darcs have no real notion of a
"repository", but always store the entire history in every source tree.
Such a model seems to simplify the user interface in some cases, but
obviously can impact disk usage...

However I have no real experience with either bk or darcs; please
correct me if I'm wrong about this.

[This is as opposed to arch, which has a repository (local/remote)
model, and allows even repositories to contain only deltas from some
other repository.]

-Miles
--
Yo mama's so fat when she gets on an elevator it HAS to go down.

2005-02-09 09:05:52

by Miles Bader

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Gene Heskett <[email protected]> writes:
> Roman, Larry has a perfectly valid reason to call you a jerk, because
> you are being one, and have been one for at least 50% of your posts
> on this list over the last 2 years or more. Right this instant my
> most fervent wish is for you to go get a job at M$ or something, and
> leave us the hell alone.

Please take the grade-school flames elsewhere.

Thanks,

-Miles
--
In New York, most people don't have cars, so if you want to kill a person, you
have to take the subway to their house. And sometimes on the way, the train
is delayed and you get impatient, so you have to kill someone on the subway.
[George Carlin]

2005-02-09 13:48:16

by David Roundy

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, Feb 09, 2005 at 05:58:22PM +0900, Miles Bader wrote:
> Kevin Puetz <[email protected]> writes:
> > erm, svk is cool and all, but it keeps a local repository mirror (not
> > necessarily full I suppose, but usually it is). So it's *much* heavier
> > on the client side than normal svn. Pays off in several ways, but just
> > because it keeps it's weight in the depot folder instead of the wc
> > folder doesn't make it ligher (unless you use several wc's I suppose).
>
> Hmmm, I thought that several other systems had similar (or worse)
> overhead -- most notably that bk and darcs have no real notion of a
> "repository", but always store the entire history in every source tree.
> Such a model seems to simplify the user interface in some cases, but
> obviously can impact disk usage...
>
> However I have no real experience with either bk or darcs; please
> correct me if I'm wrong about this.

wrt darcs, you're mostly correct. There is the possibility of a "partial"
repository, which doesn't have the full history, but that isn't the
default, and therefore tends to be less well-debugged. On the other hand,
if you never try to *look* at the history, you're pretty much safe--the
bugs tend to show up when you try to browse that history which you *do*
have.
--
David Roundy
http://www.darcs.net

2005-02-09 14:48:38

by d.c

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

El 09 Feb 2005 05:06:02 -0200,
Alexandre Oliva <[email protected]> escribi?:

> So you've somehow managed to trick most kernel developers into
> granting you power over not only the BK history, in such a way that
> anyone willing to extract all the information available from the BK
> repository and share it with others is forbidden from doing so by the
> license?

BK history is not part of "linux kernel", and those who use BK know why they're
using BK and what are the consecuence so calling them "tricked" is quite
misleading from your part.

I don't understand people. Before BK, linux kernel development *never* had a
"history" and you had to develop against -pre patches. When linus switched
to BK, he said clearly that he wasn't going to change that, and it has not
changed, in fact these days you can use -bk patches which are much better
than -pre's. Then Larry put a CVS gateway just to made people happy and give
them things that have *NEVER* been guaranteed. So, "kernel history" is *not* part
of the "linux kernel development", is just a nice addon for those who optionally
want to use BK, because the _official_ way of getting changes for the linux
kernel are patches at kernel.org which don't have any way of keeping track of
"kernel history", not BK and/or CVS. Everyone who claims the contrary is
wrong, period.

Besides, Larry is exporting most of that story in CVS, so those who accuse
Larry of "stealing our freedom" should think twice that Larry doesn't even
to _listen_ you and he doen't needs to care about your request because
it's not him who decided to use BK, and it's not him who decided to lose
the "kernel history" using a propietary tool. What you really want is change
the official kernel development (GNU diff patches at kernel.org) for something
better that can track all that "kernel history", and you are not going to change
that shouting at Larry because (again) it's *not* him who decided to use BK.

(of course if you ask linus he'll tell exactly what I said, that BK is just a _option_
and that you can get all the patches at kernel.org with all the consequences
- losing the "kernel history" because GNU diff don't really supports that)

2005-02-09 15:51:23

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, Feb 09, 2005 at 05:06:02AM -0200, Alexandre Oliva wrote:
> So you've somehow managed to trick most kernel developers into
> granting you power over not only the BK history

It's exactly the same as a file system. If you put some files into a
file system does the file system creator owe you the knowledge of how
those files are maintained in the file system? Since when is that part
of the deal? Does that mean that I can insist you provide me with a
detailed specification, without an attached GPL, of how GCC works so I
can clone the technology into my commercial compiler?

It's the same with any software package. I know Red Hat uses Oracle,
why aren't you telling Oracle to disclose how Oracle works inside?
Is it possible that Red Hat signed a license agreement that prevents
them from accessing that information? You bet it is. Did Oracle trick
Red Hat into agreeing to this awful arrangement? No, they spelled it
out in their license and Red Hat signed it.

What's going on here is no different than Red Hat deciding they don't
want to pay for Oracle so they are reverse engineering Oracle and
transferring the technology into MySQL.

> I'd much rather you didn't ``give?? it at all, then people wouldn't be
> locked into it, and the community might have come up with something as
> efficient earlier with the extra push.

We're the only vendor I've ever heard of who provides a mirror of
the data in a competing free product. Does Oracle do that for you?
Of course not. But somehow we are the bad guys locking you in? OK, if
your position is that we are locking you in then you'd have no problem
if we dropped the CVS gateway, that's of no value to you, right? And we
rate limit the web interface so you can't get your patches without BK,
that's of no value either, right?

> > I don't come here every month and ask for the GPL to be removed from
> > some driver, that's essentially what you are doing
>
> I don't think so. What he's doing is more along the lines of `hey,
> this allegedly-GPLed driver contains a piece of binary firmware whose
> source code is not there, could we either replace it with actual GPLed
> code or remove the driver?

You are missing the point that there are two licenses. Roman is trying
to tell us to change our license. The point you missed was that that's
the same as me telling you to change the GPL for my benefit.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-09 17:31:24

by Nicolas Pitre

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, 9 Feb 2005, Larry McVoy wrote:

> On Wed, Feb 09, 2005 at 05:06:02AM -0200, Alexandre Oliva wrote:
> > So you've somehow managed to trick most kernel developers into
> > granting you power over not only the BK history
>
> It's exactly the same as a file system. If you put some files into a
> file system does the file system creator owe you the knowledge of how
> those files are maintained in the file system?

No, this is not a good analogy at all.

If I don't want to use a certain filesystem, I mount it and copy the
files over to another filesystem. What users are interested in are the
files themselves of course, and the efficiency with which the filesystem
handles those files. BK is the efficient filesystem here, but anyone
should be able to freely copy files over to another filesystem without
any need for the filesystem internals knowledge. If the target
filesystem is 8.3 without lowercase support then so be it and people
will need to use a separate file to hold the extra details that cannot
berepresented natively in the target filesystem. But absolutely 0% of
the information is lost.

Again, the BK value is in the efficiency and reliability it has to
handle a tree like the Linux kernel, not in the Linux kernel tree. It's
not necessary for you to give away that value in order to provide the
simple information needed to reconstruct the Linux tree structure as
people are asking.

Nicolas

2005-02-09 17:38:54

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, 9 Feb 2005 12:17:48 -0500 (EST), Nicolas Pitre <[email protected]> wrote:
> On Tue, 8 Feb 2005, Larry McVoy wrote:
>
> Larry, why can't you compete only on the tool instead of claiming
> exclusive rights on the test bench as well?

Nicolas, Larry has not said he won't make the changes that Roman
wants, instead he has said he won't make the changes for FREE!. There
is a perfectly reasonable solution. Raise the funds to pay for the
needed changes.

It is just not reasonable to expect Larry to fund work whose sole
purpose is to compete with himself. He is going a long way in saying
he will do the work if paid at standard rates.

Again, this is not Larry's fault, it is Linus that picked the tool.

--
Jon Smirl
[email protected]

2005-02-09 17:44:53

by Valdis Klētnieks

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, 09 Feb 2005 12:30:54 EST, Nicolas Pitre said:

> If I don't want to use a certain filesystem, I mount it and copy the
> files over to another filesystem. What users are interested in are the
> files themselves of course, and the efficiency with which the filesystem
> handles those files.

That's Larry's point exactly - you don't care (or shouldn't, unless you're
writing a competing product) any more about the BK internal metadata than
you should care about how ext3 handles the journal or the free list.

Attachments:

(No filename) (226.00 B)

2005-02-09 18:24:41

by Nicolas Pitre

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, 9 Feb 2005, Jon Smirl wrote:

> On Wed, 9 Feb 2005 12:17:48 -0500 (EST), Nicolas Pitre <[email protected]> wrote:
> > On Tue, 8 Feb 2005, Larry McVoy wrote:
> >
> > Larry, why can't you compete only on the tool instead of claiming
> > exclusive rights on the test bench as well?
>
> Nicolas, Larry has not said he won't make the changes that Roman
> wants, instead he has said he won't make the changes for FREE!. There
> is a perfectly reasonable solution. Raise the funds to pay for the
> needed changes.

I'm not talking about Roman at all here. Quoting the relevant message:

On Tue, 8 Feb 2005, Stelian Pop wrote:

> What you could make available in the bkcvs export would be, for each
> changeset (both for the changesets and for the merged changesets),
> include three BKRevs: the changeset's one, the changeset's parent one,
> and the changeset's merge parent one.
>
> That information could be used to reconstruct the entire tree,
> using either bk-commit-head (preferred) or bkbits, provided you
> put the BKRev: tag into the bk-commit-head posts too.
>
> Technicaly speaking this should be very easy for you to implement.

I'm sure it's not the actual developer cost which is in cause here.

Larry turned it down with the usual "we're'll fear you if we do that"
answer although I still have problems seeing why BK would be suplented
with that info available. The SCM problem is much much more than just a
tree to bench test on.

So if Larry could realize that he has nothing to fear even if that info
is available to the competition, he might even spend less time debating
this stupid topic over and over and dedicate his time to more rentable
activities, actually making more money as a result.

Nicolas

2005-02-09 18:46:38

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, Feb 09, 2005 at 12:17:48PM -0500, Nicolas Pitre wrote:
> On Tue, 8 Feb 2005, Larry McVoy wrote:
> > You know, you could change all this. Instead of complaining that we
> > are somehow hurting you, which virtually 100% of the readers know is
> > nonsense, you could be producing an alternative answer which is better.
>
> IMHO something is flawed in this whole argument. Why would someone be
> interested into any alternative answer for working on the Linux kernel
> tree if the whole thing can't be imported into it with the same
> granularity as can be found in BK? IOW what's the point to alternatives
> if you can't retrieve the entire workset?

Please explain to me where the data is being lost. 100% of the patches
are available on bkbits.net with no license agreement required. They
always have been.

The problem is that you want us to tell you how BK manages those patches.
That was never part of the agreement, in fact we made it clear in the
license that that information was considered to be IP and could not be
distributed. How BK does that is our business, not yours. If you want
to know how BK does it you must go figure it out without the benefit
of BK itself or metadata managed by BK.

While I understand that you don't like it, is there no sense of fairness
left? We did the hard work to create BK. Some of us worked for *years*
without pay to create this product. Some of us put our life savings
into the product. It's our IP, we paid heavily to create this, is it
so unreasonable of us to want to protect our work? I believe we are
within our legal rights, or so our legal team tells us, but that should
be beside the point. It's our work. We paid for it. We certainly
don't have any obligation to tell you how we did it and to us it seems
pretty unreasonable that you don't just go off and do the work yourself.
And pretty unadmirable as well, don't you have any faith in your own
abilities?
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-09 18:48:28

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, 9 Feb 2005 13:24:06 -0500 (EST), Nicolas Pitre <[email protected]> wrote:
> Larry turned it down with the usual "we're'll fear you if we do that"
> answer although I still have problems seeing why BK would be suplented
> with that info available. The SCM problem is much much more than just a
> tree to bench test on.

Larry has said, write up a proposal for changes you want in bk. Send
it to him for a quote. Come up with the cash and he will do the work.

All I see is a bunch of complaints and no concrete proposal for the
work to be done.

He has provided a legal means for achieving what the complainers want
done. So stop complaining and start writing the proposal.

--
Jon Smirl
[email protected]

2005-02-09 20:18:58

by Nicolas Pitre

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, 9 Feb 2005, Larry McVoy wrote:

> On Wed, Feb 09, 2005 at 12:17:48PM -0500, Nicolas Pitre wrote:
> > On Tue, 8 Feb 2005, Larry McVoy wrote:
> > > You know, you could change all this. Instead of complaining that we
> > > are somehow hurting you, which virtually 100% of the readers know is
> > > nonsense, you could be producing an alternative answer which is better.
> >
> > IMHO something is flawed in this whole argument. Why would someone be
> > interested into any alternative answer for working on the Linux kernel
> > tree if the whole thing can't be imported into it with the same
> > granularity as can be found in BK? IOW what's the point to alternatives
> > if you can't retrieve the entire workset?
>
> Please explain to me where the data is being lost. 100% of the patches
> are available on bkbits.net with no license agreement required. They
> always have been.

Are you saying that it is now OK to write scripts that would bit bang on
the bkbits http interface to fetch patches/comments with the purpose of
populating an alternate scm? Andreas tried that a while ago but you
threatened to shut the service down entirely if he was to continue.

> The problem is that you want us to tell you how BK manages those patches.
> That was never part of the agreement, in fact we made it clear in the
> license that that information was considered to be IP and could not be
> distributed. How BK does that is our business, not yours.

I completely agree here. But just to be on the same page, let me quote
Stelian's proposal here again:

On Tue, 8 Feb 2005, Stelian Pop wrote:

| What you could make available in the bkcvs export would be, for each
| changeset (both for the changesets and for the merged changesets),
| include three BKRevs: the changeset's one, the changeset's parent one,
| and the changeset's merge parent one.
|
| That information could be used to reconstruct the entire tree,
| using either bk-commit-head (preferred) or bkbits, provided you
| put the BKRev: tag into the bk-commit-head posts too.
|
| Technicaly speaking this should be very easy for you to implement.

Is the above actually part of the protected BK IP? Can anyone run with
such information and clone BK in a smooth breeze with that? I don't see
how such info can tell how BK manages it.

> If you want
> to know how BK does it you must go figure it out without the benefit
> of BK itself or metadata managed by BK.

I think what people want here is the tree structure representation in
whatever form, not necessarily in the BK format. One example of that
was provided above. You can't deny others access to the whole of the
Linux kernel development history even if their purpose is to import it
into a competing system (more on that below).

> While I understand that you don't like it, is there no sense of fairness
> left? We did the hard work to create BK. Some of us worked for *years*
> without pay to create this product. Some of us put our life savings
> into the product. It's our IP, we paid heavily to create this, is it
> so unreasonable of us to want to protect our work? I believe we are
> within our legal rights, or so our legal team tells us, but that should
> be beside the point. It's our work. We paid for it. We certainly
> don't have any obligation to tell you how we did it and to us it seems
> pretty unreasonable that you don't just go off and do the work yourself.

Again I wholeheartedly agree with you above. But that's not exactly the
point. You certainly have the right to protect your work. But please
admit that the Linux kernel developers own the right to move (100% not
96%) of the development tree with all its branches _they_ produced. In
other words, the product of the current Linux BK repository is the
result of those Linux developers not yours. Of course BK is the
indispensable tool that made it possible, but BK could have been
developed even without the Linux BK repository.

So what people are asking for again and again is a way to represent in
alternate form to BK internal metadata the Linux development tree
structure but without you to give away the BK IP, be that in XML, plain
text, or just with BK refs attached to patches like suggested above.
Unless I'm completely dumb, this won't reveal anything about BK itself
and how you did it?

Now obviously enough some people will run away with that raw data and
try to import the Linux kernel source tree in their own scm of choice.
That's why I'm asking you "and so what?" Granted all the efforts you
put into BK as you say above, do you really expect the alternative scm
to suddenly be as functional and scalable as BK in a single swoop just
because they can import the Linux BK tree with the same patch
granularity as BK does? If you worked so hard on BK for many years it
means the competition should be far behind. Therefore what do you fear?

Note that if someone actually needs a big tree to test bench an
alternate scm there are alternatives to the kernel -- gcc is a good
example. So allowing the Linux kernel tree to be imported into another
scm is not really a requirement for developing on said scm.

> And pretty unadmirable as well, don't you have any faith in your own
> abilities?

Do you have faith in your own work? If so allowing others to import the
complete Linux development tree into alternative scm shouldn't scare you
so much. You probably know already that they won't be able to scale as
well as BK, and we're not even talking about the ability to perform push
and pull type operations here. So what do you actually fear? I'll be
fully on your side once I grasp the actual threat to BK.

Note I'm trying to understand what the actual problem is here. I'm not
concerned by the non-free nature of BK at all. It is the best tool for
distributed development available, and it's been applied to Linux
development with no $ cost. I'm just wondering why providing some
additional info which would allow for rebuilding the tree with all
changeset relationships (into whatever alternate inferior scm that's not
the point) would uncover your IP. It's just a graph with patches
attached to it in the end. No need to tell us how BK manages it.

Nicolas

2005-02-09 23:23:11

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Tue, 8 Feb 2005, Larry McVoy wrote:

Larry, it's interesting how you try to distract from the main problem
(which you don't mention with a single word) and instead continues to
badmouth me. Let's take a look.

> Short version: let's violate a license.

Wrong, if I wanted to violate the license, I could have done so already
and I certainly wouldn't make such a fuss about it.
To remind you the main problem was and is still, that the kernel history
is locked into bk. At this point I'm not really sure, whether all bk user
realize this, as you constantly try to distract them with random
accusation against my person.

> Our position is clear: we are trying to do the right thing for Linux,

That's of course a noble goal, but what if such help is attached to
conditions? Then it might of course happen that such help is rejected,
the question is only where is the problem. For you it's of course clear it
must be the potential receivers fault, looking critically at the
conditions is out of question for you.

> You know, you could change all this. Instead of complaining that we
> are somehow hurting you, which virtually 100% of the readers know is
> nonsense, you could be producing an alternative answer which is better.

Another smoke bomb. You already made it clear, that even if I did that, I
couldn't import the kernel history into it anyway. There are already
possible alternatives, which should be able to store the kernel history as
currently recorded in bk, but we will never know because any attempt to do
so will be constructed as license violation.

I skipped all the insults, which you use to further distract attention
from the actual problem and instead try to descredit me personally. You
must be getting quite desperate to start hitting that low. :(

bye, Roman

2005-02-09 23:31:32

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Wed, 9 Feb 2005, Jon Smirl wrote:

> Larry has said, write up a proposal for changes you want in bk. Send
> it to him for a quote. Come up with the cash and he will do the work.

Here is a simple one: restore the parent information in the gnupatch
option as they were about a year ago visible in the mails to bk-commits,
e.g.:
http://marc.theaimsgroup.com/?l=bk-commits-head&m=107558318022033&w=2

Now please go to Larry and see what you get.

bye, Roman

2005-02-09 23:52:10

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Thu, 10 Feb 2005 00:31:05 +0100 (CET), Roman Zippel
<[email protected]> wrote:
> On Wed, 9 Feb 2005, Jon Smirl wrote:
> > Larry has said, write up a proposal for changes you want in bk. Send
> > it to him for a quote. Come up with the cash and he will do the work.
>
> Here is a simple one: restore the parent information in the gnupatch
> option as they were about a year ago visible in the mails to bk-commits,
> e.g.:
> http://marc.theaimsgroup.com/?l=bk-commits-head&m=107558318022033&w=2
>
> Now please go to Larry and see what you get.

1) I don't believe anyone considers a link to an email archive a
serious proposal.
2) I don't care about what you and Larry come up with. I have no need
to get to the kernel source from other tools. I was only pointing out
that there was a solution on the table for achieving your goals and
that you are ignoring it.

Roman, You're the one that is complaining, write up your own proposal
and get a quote. LKML is not the place for your continued whining that
Larry should add features that only you want and that you don't want
to pay for. You have been given a way to solve your problem, make use
of it!

--
Jon Smirl
[email protected]

2005-02-09 23:53:36

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Wed, Feb 09, 2005 at 03:13:48PM -0500, Nicolas Pitre wrote:
> Are you saying that it is now OK to write scripts that would bit bang
> on
> the bkbits http interface to fetch patches/comments with the purpose
> of
> populating an alternate scm? Andreas tried that a while ago but you
> threatened to shut the service down entirely if he was to continue.

Go for it, if it becomes a problem we'll rate limit it. Our first
concern is that the BK users can get at the trees so if you are eating up
the bandwidth too much we'll slow down the web interface.

> | What you could make available in the bkcvs export would be, for each
> | changeset (both for the changesets and for the merged changesets),
> | include three BKRevs: the changeset's one, the changeset's parent
> one,
> | and the changeset's merge parent one.
>
> Is the above actually part of the protected BK IP?

Yes.

> I think what people want here is the tree structure representation in
> whatever form, not necessarily in the BK format.

That's fine, they can do that. Get the patches and figure out how to
put them back together. These people do know how to use patch, right?
OK, then they are welcome to patch things in, when they don't work, find
a place they do work and create a branch, patch them, repeat. Haven't
they ever dealt with a patch reject before? It's not that hard.

> You can't deny others access to the whole of the
> Linux kernel development history even if their purpose is to import it
> into a competing system (more on that below).

I'm not denying anyone that. The history consists of the diffs and the
checkin comments, you have that.

> Again I wholeheartedly agree with you above. But that's not exactly the
> point. You certainly have the right to protect your work. But please
> admit that the Linux kernel developers own the right to move (100% not
> 96%) of the development tree with all its branches _they_ produced.

_They_ didn't produce the branching structure, BitKeeper did that.
Go look, there isn't a way to type a command which produces a branch in
BK. So claiming that metadata is property of the developers is nonsense,
they didn't produce, it isn't physically possible for them to produce it.

That's part of BK's design and power. 100% of what any developer
produced is already available on the web, in the form that has been
used for more than 10 years as the preferred form, a unified diff.
And we added comments because those are useful and you typed them in.
You guys have been importing patches for more than a decade, since when
did that become a problem?

> Now obviously enough some people will run away with that raw data and
> try to import the Linux kernel source tree in their own scm of choice.
> That's why I'm asking you "and so what?"

That's fine if they want to do that, they have the patches. What they
don't get is the tree structure that BK has because that gives them the
ability to go back and forth and say "well, BK did it this way and it
worked, why doesn't it work in our system?".

> Note that if someone actually needs a big tree to test bench an
> alternate scm there are alternatives to the kernel -- gcc is a good
> example. So allowing the Linux kernel tree to be imported into another
> scm is not really a requirement for developing on said scm.

Indeed. I don't suppose there is any chance you could get people to go
play with the gcc tree?

> I'm just wondering why providing some additional info which would allow
> for rebuilding the tree with all changeset relationships (into whatever
> alternate inferior scm that's not the point) would uncover your IP.

I think you are fishing for BK internal information and I'm not going
to bite. The bottom line is that we laid out some rules, the BK users
agreed to them, that's the deal. If you don't like the deal then go
build an alternative. You can use all the patches you want from BK but
you don't get to use BK's metadata.

--lm

2005-02-10 00:13:33

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Thu, Feb 10, 2005 at 12:22:39AM +0100, Roman Zippel wrote:
> > You know, you could change all this. Instead of complaining that we
> > are somehow hurting you, which virtually 100% of the readers know is
> > nonsense, you could be producing an alternative answer which is better.
>
> Another smoke bomb. You already made it clear, that even if I did that, I
> couldn't import the kernel history into it anyway.

Sure you can. You can get every patch from bkbits.net. You can write a
program which imports them. You can figure out a graph structure which
allows all the imports reject free.

Your complaint is that BK has already figured out one such graph that
has those characteristics and you know that it would be easier if we
told you what that is. Yes, I know that to be true, it would be easier
for you. However, we're long past the pretense that this is so you can
do your kernel engineering, we all know that this is so you can work on
a clone of BK, whoops, a much better SCM system.

No business will survive if they give away their advantage to their
competitors, which is what you are asking us to do. Even if I wanted
to do it I have a board that would promptly fire me if I did.

OK, that's it for me, I have to go work on slides for a talk so have the
big fun, I'm signing off on this thread.

Cheers,
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-10 00:15:20

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Wed, 9 Feb 2005, Larry McVoy wrote:

(I just sent a similiar mail in private and didn't immediately realize it
didn't went to lkml, so sorry, who gets it twice.)

> On Wed, Feb 09, 2005 at 03:13:48PM -0500, Nicolas Pitre wrote:
> > I think what people want here is the tree structure representation in
> > whatever form, not necessarily in the BK format.
>
> That's fine, they can do that. Get the patches and figure out how to
> put them back together. These people do know how to use patch, right?
> OK, then they are welcome to patch things in, when they don't work, find
> a place they do work and create a branch, patch them, repeat. Haven't
> they ever dealt with a patch reject before? It's not that hard.

No, just impossible.

As I mentioned already previously, the main problem is restoring the
changeset information, so one actually knows how the kernel tree looked at
a specific point. I also mentioned that it's not really possible to find
for all changesets their parent changesets, e.g. changeset 1.889 has 416
branches which include 3560 changesets, that means some of the branches
have over 3000 potential parents. Due to parallel changes to a file on
multiple branches one can reduce that number, but it's likely still
greater than one.

Now for a while I hoped to at least find the end of branch, that means
where two branches get merged again. The gnupatches contain some
information of what they merge, so that one could use the log text to find
the changeset. The problem is that even these gnupatches don't contain
information to reliable find its parents. First, they don't include other
merge logs, so there are again multiple potential parents if there are
merge changeset near the changeset identified by the log text. Second,
there are completely empty changesets with no log text and so no
indication what they are merging, I currently have 171 of them and of
these 13 are at a start of the branch (and therefore have no usable
information at all of either parent).

So why is this parent information so important? If the patches are not
applied in the correct order, one simply gets the wrong kernel snapshot.
What makes this more difficult are the merge changesets, as they don't
contain the complete information of what has changed compared to one of
the parents, they just contain the file conflicts. If one had the correct
parent information this would not be a problem, repeating the merge is
one of the smaller problems. But with fuzzy parent information one also
only gets fuzzy merge results and if the parents have n potential parents
that results in n^2 possible merge results in the worst case. Since the
merge changeset only contains conflict information, that makes it rather
likely one detects a problem (e.g. that a changeset doesn't apply) very
late. So if one detects such problem after m merges, we have a worst case
of n^(m+1). IOW the complexity of a bkweb export grows exponentially with
the size of the repository! And there is still no guarantee to get a
correct (that means only one) result.
So doing the work is one thing, getting a result within my lifetime would
be nice too.

bye, Roman

2005-02-10 00:50:50

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Thu, Feb 10, 2005 at 01:14:43AM +0100, Roman Zippel wrote:
> [long explanation which is summarized as "it's hard"]
> So doing the work is one thing, getting a result within my lifetime would
> be nice too.

I understand the complexity you are facing. This may be hard for you
to believe but we have solved problems that are quite a bit more complex
than what you are trying to do and then gave you the solution for free.

This problem is nowhere near as hard as you are making it out to be
but it is hard. But it's not that bad, we do this every time we do
a CVS import, we have to intuit the changeset boundaries themselves,
which is actually harder than what you are trying to do. Think about
taking revision history without any changeset grouping and recreating
the changesets (aka patches). We do that all the time, automatically.
If we can do that then you can do this.

Yeah, it's hard, though, I admit that. This is a bit of a cheap shot
but you did say that this stuff was easy, remember? Yeah, you may hate
me for making you realize it is hard but maybe you'll start to respect
what it is we have given you. That would be cool.

Anyway, you can do this. It will probably take you a couple of months
of 80 hour weeks. Does that suck? Maybe. Bear in mind that we have
given you *years* of 80 hour weeks. Years. Many of them unpaid.
So suck it up and get to work, we did.

You might have fun doing this, you know. These problems, while hard,
have some very satisfying mathematical qualities and that's really fun
coming from the kernel background where things are far less deterministic.
You can actually write proofs about how things work for a change.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-10 05:47:28

by James Bruce

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

While I agree with your overall sentiment, please compare apples to
apples regarding the license. You said:

Larry McVoy wrote:
> I don't come here every month and ask for
> the GPL to be removed from some driver, that's essentially what you are
> doing and I think pretty much everyone is sick of it.

The GPL doesn't state that "You and anyone at your company are not
allowed to work on any operating system software under any non-GPL
license." While that would be a perfectly valid license (just like the
BK one), obviously it would generate a fairly steady stream of
complaints. It's not like people have stopped complaining about how the
GPL forces them to release any code they link with it. The boundary of
a license will always create friction. This will be especially true as
in the BK license, which was expressly designed to be irritating to a
certain class of people (who now whine about it).

A more directly relevant example would be the following: What if a new
version of CVS had a license with a clause stating the following: "Any
repository touched by CVS 1.2 may not be imported into into BK, unless
you first remove all checkin comments. This is because we don't help
people who are competing with us." Sure, that's a 100% legitimate
license, and binding due to standard copyright goodness, yet I would
expect BitMover people to complain about it. What we have now is
exactly the same thing in reverse. Get used to the complaints because
your license is achieving exactly what you meant it to do.

- Jim

2005-02-10 06:08:52

by James Bruce

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Roman, please give up on importing 100% of the history. There's no
point arguing something if you already know what the other person's
answer will be. Larry will not change his mind under any currently
foreseeable circumstances. Yes, there is "meta-data lockin" whether
anyone at BitMover will admit it or not, but no that will not change.

Linux survived in the past without much history, and if a replacement
arrives, people can make the switch even with a degraded history. In
very little time that switchover would seem as remote as the pre-BK
times are now.

Right now I don't see why its necessary to track the Linux repo in 100%
detail for SCM development; There are plenty of other big trees to test
on if you need every detail. Time spent tracking Linux are probably
better spent improving an alternative SCM, most of which have plenty of
wishlist items awaiting developers. For kernel development, yes it's
painful for SCM developers or purists, but you can still work just fine
with patches. Maintainers certainly benefit from BK, but for developers
on the leaves of the hierarchy there's not that much difference.

- Jim Bruce

2005-02-10 07:01:44

by Horst H. von Brand

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Nicolas Pitre <[email protected]> said:
> On Wed, 9 Feb 2005, Larry McVoy wrote:
> > On Wed, Feb 09, 2005 at 05:06:02AM -0200, Alexandre Oliva wrote:
> > > So you've somehow managed to trick most kernel developers into
> > > granting you power over not only the BK history

> > It's exactly the same as a file system. If you put some files into a
> > file system does the file system creator owe you the knowledge of how
> > those files are maintained in the file system?

> No, this is not a good analogy at all.

It is just fine.

> If I don't want to use a certain filesystem, I mount it and copy the
> files over to another filesystem. What users are interested in are the
> files themselves of course, and the efficiency with which the filesystem
> handles those files. BK is the efficient filesystem here, but anyone
> should be able to freely copy files over to another filesystem without
> any need for the filesystem internals knowledge. If the target
> filesystem is 8.3 without lowercase support then so be it and people
> will need to use a separate file to hold the extra details that cannot
> berepresented natively in the target filesystem. But absolutely 0% of
> the information is lost.

But what you want is not the files, but the whole history of the filesystem
(what was written/changed/deleted when).

> Again, the BK value is in the efficiency and reliability it has to
> handle a tree like the Linux kernel, not in the Linux kernel tree. It's
> not necessary for you to give away that value in order to provide the
> simple information needed to reconstruct the Linux tree structure as
> people are asking.

linux-2.6.10.tar.bz2, and you even get the -bk patches!
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513

2005-02-10 09:37:10

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Feb 9, 2005, [email protected] (Larry McVoy) wrote:

> On Wed, Feb 09, 2005 at 05:06:02AM -0200, Alexandre Oliva wrote:
>> So you've somehow managed to trick most kernel developers into
>> granting you power over not only the BK history

> It's exactly the same as a file system. If you put some files into a
> file system does the file system creator owe you the knowledge of how
> those files are maintained in the file system?

No, the `how' is not the missing bit. The missing bit is `what'.

Taking your file system analogy, consider this:

- you only have a filesystem that supports 8.3 file names, all-caps.

- I offer you a new filesystem layer that can supports file names with
arbitrary lengths, lower-case letters, blanks, etc. This is
actually implemented using an 8.3 filesystem structure, adding to it
some special filename mapping files that a filesystem upper layer will
interpret to present users with arbitrary filenames. The actual 8.3
filenames used under the hood are chosen with a well-defined
algorithm, that I even offer as part of the filesystem documentation.

- the catch: copy operations from this filesystem to any other are not
allowed to carry over the filename map that my filesystem happens to
maintain from 8.3, so the result of the copy to any other filesystem
will have 8.3 names, even in a copy to a filesystem that does
support arbitrary file names.

When you claim the Linux community got a very good tool from you, and
that those who use your tool are far better off, and those who don't
are worse off, you fail to realize that there is one thing that all of
us lost in the process: a compelling reason to help develop a Free VCS
tool that could play the role that BK currently does. Sure, if Linus
hadn't adopted BK, nobody would have any of the metadata that Roman is
requesting. However, I suspect many would be working towards creating
a tool that would provide everyone with such a tool.

Also, by offering them a tool that will let them get access to the
metadata, but not export it in a form that other tools that supported
changesets and multi-branch patch tracking, you set the people who
chose to adopt BK up for a difficult situation for as soon as a Free
Software tool would be able to offer a similar amount of convenience
as BK does: should they ever decide to switch, they'd have to give up
all of the metadata that was part of the reason for the switch in the
first place, and start over from scratch in the new VCS. So they
might be further compelled to stick with this proprietary piece of
software, just because the fact that a Free Software equivalent exists
may not be enough for them to offset the penalty of giving up the
metadata.

This was the clever trick I alluded to. Well done! Too bad for us.

> Since when is that part of the deal? Does that mean that I can
> insist you provide me with a detailed specification, without an
> attached GPL, of how GCC works so I can clone the technology into my
> commercial compiler?

Why, sure, and you do get that. The source code can serve as a pretty
good specification and, as long as you don't copy it directly, you're
free to use it in your proprietary compiler, commercial or not.

> It's the same with any software package. I know Red Hat uses Oracle,
> why aren't you telling Oracle to disclose how Oracle works inside?

Because that's not the point. Should Oracle keep part of the data
stored in the database for itself, I would. But they don't. It's our
data in there.

What BK is doing to us is equivalent to using a database with a
predetermined set of queries, that operate on the current state of the
database, plus an additional operation to dump all of the transactions
(not the state, the transactions) that have ever been performed on
this database, in a randomized order. The transactions are all in
diff format, so one could theoretically serialize them and figure out
the complete state of the database at a certain point, but not
necessarily intermediate states, which, in the presence of concurrent
transactions without an intervening serialization event, may not even
exist. And then one could import the state into another database and
perform queries.

So, you see, having the list of patches/transactions/changesets
without information of what goes atop of what doesn't stop people from
figuring out the current state of the database, but it sure requires
them to do a lot of work to get to it, except for the predetermined
queries.

> What's going on here is no different than Red Hat deciding they don't
> want to pay for Oracle so they are reverse engineering Oracle and
> transferring the technology into MySQL.

Guess what: we would be able to take all of our data and move it into
a different database.

> We're the only vendor I've ever heard of who provides a mirror of
> the data in a competing free product.

It's not a complete mirror. If it was, we wouldn't be having this
discussion. The mirror intentionally excludes information that
actually differentiate BK from CVS and SVN, but not from other
changeset tracking VCS tools.

> Does Oracle do that for you?

I do believe it's possible to have replicated databases with one of
the replicas running Oracle and another replicate running other
database server, yes.

> But somehow we are the bad guys locking you in?

Well, you are locking us in, I don't think there's no debate over
that. I didn't say you were bad guys for that, I even applauded the
nice trick you played on Linus and other BK adopters. Personally, I
dislike proprietary software, and will stay away from it as much as
I can.

In this sense, you and Oracle are on the same boat: both have made
some contributions to the Free Software community, but only in so far
as it helps them promote their bottom line, a proprietary software
product.

> OK, if your position is that we are locking you in then you'd have
> no problem if we dropped the CVS gateway, that's of no value to you,
> right?

Of course not. It would make the lock-in even stronger. It might be
a good step towards getting people to move away from BK. I believe
you offer the CVS gateway because you know you have to in order to
keep Linux hosted on BK. If you didn't, people would be compelled to
find a way out as quickly as possible.

It just so happens that the CVS gateway is enough for most, but not
all, uses, and there are only a few people who care enough about the
missing info to make a fuss about it, so you keep the balance towards
you and BK. Good for you.

--
Alexandre Oliva http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org}

2005-02-10 09:53:57

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi,

On Wed, 9 Feb 2005, Larry McVoy wrote:

> This problem is nowhere near as hard as you are making it out to be
> but it is hard. But it's not that bad, we do this every time we do
> a CVS import, we have to intuit the changeset boundaries themselves,
> which is actually harder than what you are trying to do. Think about
> taking revision history without any changeset grouping and recreating
> the changesets (aka patches). We do that all the time, automatically.
> If we can do that then you can do this.

That's simply not true and you know that.
I repeat, the bkweb information is not enough to recreate the history,
incomplete data will only produce incomplete results no matter how hard
you try and I delivered the _proof_ for that.

bye, Roman

2005-02-10 10:14:27

by linux

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

I really got bored of this thread.Can you all question your self on thing?
If someone starts reading right now the sources of the linux kernel will be
able to understand every aspect and part of the code???
Do you understand every aspect?

Is it still "opensource" or starts to be a "closedsource" software "product
" despite the fact that is still free to the community.
DONT say again the source is there ,you dont have but to read it.
Someone in this list ( very popular ) has said some years ago that even now
Micro$oft gives out the source noone will be able to do some changes and
some of us wont be able to understand this product ever never.
So in the name of this idea is still linux an "opensource" idea or a
"closed" one delivered from those who have the information already to
maintain it and give it to us freely. I tend to develop in linux only 4
years now and i tried to make some changes in the kernel so that they can
conform according to my needs... i have to say that it was very very time
consuming.....no doc...no comments...no explanation....I think that a lot of
us have the same question....

"Opensource" || "Closedsource" ???????????

2005-02-10 15:13:20

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Thu, Feb 10, 2005 at 01:08:20AM -0500, James Bruce wrote:

> Roman, please give up on importing 100% of the history. There's no
> point arguing something if you already know what the other person's
> answer will be. Larry will not change his mind under any currently
> foreseeable circumstances. Yes, there is "meta-data lockin" whether
> anyone at BitMover will admit it or not, but no that will not change.
>
> Linux survived in the past without much history, and if a replacement
> arrives, people can make the switch even with a degraded history. In
> very little time that switchover would seem as remote as the pre-BK
> times are now.

Nice conclusion to this thread.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-10 16:41:18

by Steve Lee

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Roman, besides BK being closed source, how exactly is it lacking for
your needs? If what it lacks is a good idea and helps many, Larry and
crew might be willing to add whatever it is you need.

2005-02-10 19:25:09

by Vojtech Pavlik

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Thu, Feb 10, 2005 at 10:42:15AM -0600, Steve Lee wrote:

> Roman, besides BK being closed source, how exactly is it lacking for
> your needs? If what it lacks is a good idea and helps many, Larry and
> crew might be willing to add whatever it is you need.

A feature I lack is 'floating changesets', that would keep always at the
top of the history, rediffed, remerged and updated as other changesets
come in.

I know quilt can do it, but quilt can't do other things I like on bk.

--
Vojtech Pavlik
SuSE Labs, SuSE CR

2005-02-10 19:34:53

by d.c

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

El Thu, 10 Feb 2005 00:22:39 +0100 (CET),
Roman Zippel <[email protected]> escribi?:

> To remind you the main problem was and is still, that the kernel history
> is locked into bk. At this point I'm not really sure, whether all bk user
> realize this, as you constantly try to distract them with random

Yes, it's locked, so?

Roman, You are losing lot of time discussing this. What you need to do is to talk
with bk USERS. Larry has no influence in people's decisions, and people has
chosen BK and they know very well what they were doing.

If everyone at lkml agreed that it's neccesary to use a tool which exports 100% of
the kernel history to develop the kernel, and BK users would tell larry "we need this
feature or we will stop using BK" you'd get what you want in a jiffie.

2005-02-10 19:50:52

by Dmitry Torokhov

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Thu, 10 Feb 2005 20:23:19 +0100, Vojtech Pavlik <[email protected]> wrote:
> On Thu, Feb 10, 2005 at 10:42:15AM -0600, Steve Lee wrote:
>
> > Roman, besides BK being closed source, how exactly is it lacking for
> > your needs? If what it lacks is a good idea and helps many, Larry and
> > crew might be willing to add whatever it is you need.
>
> A feature I lack is 'floating changesets', that would keep always at the
> top of the history, rediffed, remerged and updated as other changesets
> come in.
>
> I know quilt can do it, but quilt can't do other things I like on bk.
>

I have switched from BK to BK + quilt - I use BK to do pull from
various trees and merge it all together and then quilt to drop my
work-in-progress on top and refresh (rediff) so there are no offests.

So far it is very handy

--
Dmitry

2005-02-10 21:17:11

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

Hi Alexandre,

It seems like you've made up your mind that we are operating out of pure
self interest and have no desire to help you or anyone else unless we
get something out of it. In other words, we're making our decisions
based on the net positive/negative effect on our business.

Is that a fair assessment of your position?

It's clear that the path we took has generated illwill amongst some of
you, the web is full of BK license is awful pages, etc. It's a matter
of the public record that we had at least some idea that this was going
to happen, we didn't expect that you'd all be happy with this arrangement.

So if we knew that doing this would hurt our business, which according
you is the only thing we care about, then why would we do it? The usual
response is the marketing value we get out of it. Yes, we certainly do
get some positive marketing out of this. We also get a ton of negative
marketing, you are aware of that, right?

Is it your opinion that the postive marketing we get outweighs the
negative? That's possible, I don't think it's true, but lets suppose it
is. That's only part of the picture. The other part is that we are
taking a huge risk by giving away the product to very group that is most
motivated to copy it, and if a good enough copy existed it is unlikely
our business would success.

If you are willing to believe that we have good good enough management
here that we were aware of this, and we added up the illwill and the IP
risk and did it anyway. Why? Why would any business do something that
was obviously a poor business decision? Please don't take the cheap
shot and say we are idiots, the founder of your company has advised us
from day one as have others. We knew what we were doing.

Can you offer any plausible explanation other than a good faith desire
to help the open source community, albeit in a non-traditional way?

This is perhaps a hard concept to grasp but we are basically saying we're
willing to help everyone except the people who want to take our business
away. I would guess that over 99.999% of the open source in the world
has nothing to do with SCM. That's a lot of source that we can help.
You are saying we are an evil money grubbing corporation because we
don't want to give our technology to our competitors. Fair enough,
that's true, we don't. What you aren't admitting is that we have done
a lot of good for your community, we continue to provide the tools,
the support, the infrastructure, and we do it in spite of it not being
a very good business decision. If we get no credit in your mind for
all of that then I don't think we have any basis for further discussion.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-11 08:39:18

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Thu, Feb 10, 2005 at 08:34:37PM +0100, d.c wrote:

> El Thu, 10 Feb 2005 00:22:39 +0100 (CET),
> Roman Zippel <[email protected]> escribi?:
>
> > To remind you the main problem was and is still, that the kernel history
> > is locked into bk. At this point I'm not really sure, whether all bk user
> > realize this, as you constantly try to distract them with random
>
> Yes, it's locked, so?
>
> Roman, You are losing lot of time discussing this. What you need to do is to talk
> with bk USERS. Larry has no influence in people's decisions, and people has
> chosen BK and they know very well what they were doing.

Not exactly true. The BK licence changed a few times since Linus
started using it. IIRC, the restrictions on 'exporting the metadata'
as well as the one on 'work with someone hacking on other random SCM'
were NOT in the licence at that time.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-11 09:04:38

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Thu, Feb 10, 2005 at 01:17:00PM -0800, Larry McVoy wrote:

> So if we knew that doing this would hurt our business, which according
> you is the only thing we care about, then why would we do it? The usual
> response is the marketing value we get out of it. Yes, we certainly do
> get some positive marketing out of this. We also get a ton of negative
> marketing, you are aware of that, right?
[...]

What you also get is a lot of testing coming from a real-world big
project, and you seem to give a lot of importance to this:

> I think you are dreaming. You've gone from wanting enough information
> to supposedly debug your source tree to being explicit about wanting to
> recreate the entire BK history in a different system. The former is a
> reasonable request, I suppose, but the latter is just a blatent request
> for us to help debug and stress test a competing system.

You are also saying:
> You are saying we are an evil money grubbing corporation because we
> don't want to give our technology to our competitors. Fair enough,
> that's true, we don't.

But that's not the point. The point is, as Nicolas said it very well:
> Again, the BK value is in the efficiency and reliability it has to
> handle a tree like the Linux kernel, not in the Linux kernel tree. It's
> not necessary for you to give away that value in order to provide the
> simple information needed to reconstruct the Linux tree structure as
> people are asking.

And the whole discussion happens because you disagree on this one.

I'm fed up now so I'll stop here.

Stelian.
--
Stelian Pop <[email protected]>

2005-02-11 15:31:30

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Feb 10, 2005, [email protected] (Larry McVoy) wrote:

> It seems like you've made up your mind that we are operating out of pure
> self interest and have no desire to help you or anyone else unless we
> get something out of it. In other words, we're making our decisions
> based on the net positive/negative effect on our business.

> Is that a fair assessment of your position?

It sounds like a reasonable assessment, although your wording can be
read as if I meant that in a negative way. AFAIK a publicly-traded
company (which I believe is not your case, but anyhow) is *required*
to behave like that. I actually expect most businesses to behave
like. It's not a judgment, a good vs evil thing. I actually think
you made a very clever move, and if it wasn't for self benefit, you're
very lucky :-)

> It's clear that the path we took has generated illwill amongst some of
> you

Those of us who are religious about Free Software (myself included)
can't help being upset because one of the most visible Free Software
packages relies on proprietary software for an important part of its
development process, and even more so because of the lock-in tactics
played by their developers.

> So if we knew that doing this would hurt our business, which according
> you is the only thing we care about, then why would we do it?

I think you correctly assessed the situation and decided to take the
risk that, in spite of the negative reactions you'd get, you'd still
get good visibility, a very important showcase, and a number of happy
users that would be willing to recommend your software to others who
might be looking for a VCS.

People who are happy with what they get seldom make their feelings
noisily public; those that are unhappy are more likely to make a fuss
out of it. This is just human nature.

In spite of a bit of negative publicity, you're still better off, even
after taking into account all of the costs you incur because of the
involvement with Linux.

> Is it your opinion that the postive marketing we get outweighs the
> negative?

I do believe so, yes. The fact that one of the most representative
Open Source projects chose BK, in spite of BK being proprietary,
sounds like a very powerful claim to me.

> If you are willing to believe that we have good good enough management
> here that we were aware of this, and we added up the illwill and the IP
> risk and did it anyway. Why? Why would any business do something that
> was obviously a poor business decision? Please don't take the cheap
> shot and say we are idiots, the founder of your company has advised us
> from day one as have others. We knew what we were doing.

I don't think you are idiots. I actually admire your bold move, and
think it was a very clever one. But that doesn't make me happy about
it, because BK is not Free Software, and I, who prefer to use only
Free Software, am denied access to information that is available to
others that don't feel that strongly (or at all) about this matter.

> Can you offer any plausible explanation other than a good faith desire
> to help the open source community, albeit in a non-traditional way?

I don't see what you've done as helping the open source community. I
think the use of BK undermines the bottom line of promoting Free
Software and living by what we promote. It is embarrassing to me to
admit that Linux uses BK as a VCS, just like it is embarrassing to
admit that Red Hat uses proprietary software from Oracle, just like it
is embarrassing to admit that I had to use an MS-Windows box to
perform services for a Red Hat customer shortly after I joined Red
Hat, after almost 10 years without using MS-DOS or MS-Windows. None
of these bode well to the message I try to spread when I talk about
Free Software.

> You are saying we are an evil money grubbing corporation because we
> don't want to give our technology to our competitors.

I'm not saying that. I'm saying you're a clever corporation who are
promoting your bottom line by providing services and tools to an Open
Source project, and I'm grateful for that, especially for the CVS
gateway, but unfortunately you're also hurting the Free Software
bottom line by having got Linux into a lock-in position.

I'm not concerned in any way that you don't want to give your
technology to your competitors. I don't want your technology, since
it's proprietary. I'd just like to have access to the information
about Linux that your technology is intentionally hiding from me.

The bit I don't understand is that you've claimed you'd be willing to
implement the code needed to export the additional information that
Roman, myself and probably many others would like to have, if someone
would pay for that, but you're not willing to grant him access to this
information such that he can write the code himself. How come you
wouldn't welcome a BK-export piece of software that you could use
yourself to create and maintain the CVS tree, without having to
develop and maintain the software, and insist on developing such
software yourself, but only if someone else pays for it?

--
Alexandre Oliva http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org}

2005-02-11 15:48:48

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Fri, Feb 11, 2005 at 01:30:22PM -0200, Alexandre Oliva wrote:
> > Can you offer any plausible explanation other than a good faith desire
> > to help the open source community, albeit in a non-traditional way?
>
> I don't see what you've done as helping the open source community.

So in your mind, aiding the open source community is done only through
creating more open source. Directly. The fact that open source projects
which use BK are more productive than if they were not using BK is
irrelevant and of no value, correct? The fact that they are creating
more open source more quickly because of their use of BK is irrelevant
and of no value, correct?

In that case, get the free BK users to agree with you and we'll pull the
plug on free BK. The point of giving away BK for free is to help you.
If you have consensus that it isn't helping then we'll shut it down.

On the other hand, if you can't achieve that consensus then perhaps you
might consider broadening your definition of "help" to include something
other than "more GPLed source".
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-11 15:53:55

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Fri, Feb 11, 2005 at 01:01:46PM -0200, Alexandre Oliva wrote:
> I don't believe his claim, and I can prove it with a dumb example.
>
> Consider three patches, A, J and U, such that A and U are identical,
> and J is a patch that reverses them.
>
> You can determine the final state of the tree given these 3 patches,
> but you can't determine the history. It could be that A was installed
> first, then J reverted it, then U put it back in. Or it could be that
> U went in first, J reverted it, and A was put back in. How could one
> know? Presumably by looking at the check in messages, presumably.

They all have dates, that's one hint, and they all should apply with no
patch rejects, that's another hint. You're right that this still
doesn't make it possible to create exactly the same tree as the BK tree
but it makes it possible to create a useful tree.

You are also right that figuring out the merges is a pain. So what?
We never said that we'd figure out how to do all this well and then
teach you how to do it well.
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-11 16:13:52

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Fri, 11 Feb 2005 07:39:47 -0800 (PST), Alexandre Oliva
<[email protected]> wrote:
> The bit I don't understand is that you've claimed you'd be willing to
> implement the code needed to export the additional information that
> Roman, myself and probably many others would like to have, if someone
> would pay for that, but you're not willing to grant him access to this
> information such that he can write the code himself. How come you
> wouldn't welcome a BK-export piece of software that you could use
> yourself to create and maintain the CVS tree, without having to
> develop and maintain the software, and insist on developing such
> software yourself, but only if someone else pays for it?

Think about it from Larry's side for a minute. BK is a proprietary
piece of software, it is not open source. That means that anyone who
works on it needs to be an employee or contractor of Bitmover and have
signed all of the appropriate non-disclosure and non-compete
documents. These documents exist at all proprietary software
companies, they are not specific to Bitmover. None of you guys are
willing to sign those documents.

It's not Larry choosing not to have you do the work, you are self
selecting not to do it because you won't sign the contracts. Larry
also has to be reasonably confident that if you do sign you won't
violate them.

The conclusion is that someone else who is willing to sign the
documents has to do the work. That person needs to be hired and paid.
It is unreasonable to ask Larry to add features like you want on his
own dime.

There is a solution on the table:
1) Written proposal describing in detail the commands you want added to bk
2) Submit it for a quote.
3) Raise the money
4) Negotiate for exact delivery dates.

Since no one has bothered to put together a real proposal, I can only
conclude that you are more interested in writing email complaints than
actually achieving a solution.

--
Jon Smirl
[email protected]

2005-02-11 16:18:48

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

The mails have started flowing in saying "I don't agree with Alexandre
and please don't pull the plug" so a point of clarification. We have
no intention of shutting down the BK free product. We are aware that
there are 10's of thousands of developers in the open source world
who do not agree with Alexandre's narrow view of things. You're fine,
we're not taking BK away. I only trying to get Alexandre to see that
his definition of "help" is somewhat narrow-minded.

Cheers,

--lm

On Fri, Feb 11, 2005 at 07:48:42AM -0800, lm wrote:
> On Fri, Feb 11, 2005 at 01:30:22PM -0200, Alexandre Oliva wrote:
> > > Can you offer any plausible explanation other than a good faith desire
> > > to help the open source community, albeit in a non-traditional way?
> >
> > I don't see what you've done as helping the open source community.
>
> So in your mind, aiding the open source community is done only through
> creating more open source. Directly. The fact that open source projects
> which use BK are more productive than if they were not using BK is
> irrelevant and of no value, correct? The fact that they are creating
> more open source more quickly because of their use of BK is irrelevant
> and of no value, correct?
>
> In that case, get the free BK users to agree with you and we'll pull the
> plug on free BK. The point of giving away BK for free is to help you.
> If you have consensus that it isn't helping then we'll shut it down.
>
> On the other hand, if you can't achieve that consensus then perhaps you
> might consider broadening your definition of "help" to include something
> other than "more GPLed source".
> --
> ---
> Larry McVoy lm at bitmover.com http://www.bitkeeper.com

--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-11 16:54:27

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Feb 11, 2005, [email protected] (Larry McVoy) wrote:

> On Fri, Feb 11, 2005 at 01:30:22PM -0200, Alexandre Oliva wrote:
>> > Can you offer any plausible explanation other than a good faith desire
>> > to help the open source community, albeit in a non-traditional way?

>> I don't see what you've done as helping the open source community.

Sorry. I should have said the Free Software community. The Open
Source community doesn't generally care about the moral issues related
with freedom and avoiding the use of proprietary software.

Still, this sentence, taken out of the context where it was, sounded
much stronger than I meant. I don't know whether you quoted it out of
context on purpose, to make it (and myself) an easier target for
criticism, or just because you didn't feel like quoting the
explanation for it below, in which I used the correct term to refer to
the Free Software community.

> So in your mind, aiding the open source community is done only through
> creating more open source. Directly.

No, there are several other ways to help both the Free Software and
the Open Source community. But getting them to use proprietary
software isn't a way to help promote Free Software. It does undermine
the message of software freedom.

> The fact that open source projects which use BK are more productive
> than if they were not using BK is irrelevant and of no value,
> correct?

IMNSHO, the most justifiable use of proprietary software is in
developing a free alternative, like in the beginning of the GNU
project, when no completely-free operating system existed.

Using proprietary software just because you can, without making
efforts to switch to Free Software as soon as possible, is a sure way
to help the proprietary software side win the battle against free
software.

Entrapping yourself into a piece of proprietary software that will not
only forbid you from working on free alternatives, but also prevent
you from sharing the information you stored in it yourself is, IMHO, a
mistake.

> The fact that they are creating more open source more quickly
> because of their use of BK is irrelevant and of no value, correct?

It surely does have some value. I don't think such value outweights
the lock-in.

> In that case, get the free BK users to agree with you and we'll pull the
> plug on free BK.

Heh. Yeah, right. Don't count on everybody being as religious as I
am on these matters.

--
Alexandre Oliva http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org}

2005-02-11 17:25:50

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Feb 11, 2005, Jon Smirl <[email protected]> wrote:

> It's not Larry choosing not to have you do the work, you are self
> selecting not to do it because you won't sign the contracts.

No. We don't want access to the BK software. We want access to the
data that is stored in the repository, that's all.

--
Alexandre Oliva http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org}

2005-02-11 17:30:49

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Feb 11, 2005, [email protected] (Larry McVoy) wrote:

> You are also right that figuring out the merges is a pain. So what?
> We never said that we'd figure out how to do all this well and then
> teach you how to do it well.

We're not asking for you to teach us how to do it. We're just asking
you to let us know the information that is (presumably) stored in the
repository. Now if you tell me the information isn't stored there at
all, you just throw it away at check in time and then figures it all
out again on the fly upon request, fine, I'll believe you. I just
wouldn't have thought that's the way it works.

--
Alexandre Oliva http://www.ic.unicamp.br/~oliva/
Red Hat Compiler Engineer aoliva@{redhat.com, gcc.gnu.org}
Free Software Evangelist oliva@{lsd.ic.unicamp.br, gnu.org}

2005-02-11 19:03:38

by none given

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Fri, February 11, 2005 11:18 am, Larry McVoy said:
>The mails have started flowing in saying "I don't agree with Alexandre
>and please don't pull the plug" so a point of clarification. We have
>no intention of shutting down the BK free product. We are aware that
>there are 10's of thousands of developers in the open source world
>who do not agree with Alexandre's narrow view of things. You're fine,
>we're not taking BK away. I only trying to get Alexandre to see that
>his definition of "help" is somewhat narrow-minded.
>

Then why don't you stop threatening to take it away every time someone
points out to you that your "help" for free software isn't ideal? Just
can't help yourself? Your cheap shot at Alexandre doesn't change the fact
that you've shown yet again why people who believe in free software should
work to replace BK.

Cheers,
Hank

_________________________________________________________________
Don?t just search. Find. Check out the new MSN Search!
http://search.msn.click-url.com/go/onm00200636ave/direct/01/

2005-02-11 19:51:06

[permalink] [raw]

Subject: Re: [RFC] Linux Kernel Subversion Howto

On Fri, Feb 11, 2005 at 10:56:02AM -0800, none given wrote:
> On Fri, February 11, 2005 11:18 am, Larry McVoy said:
> >The mails have started flowing in saying "I don't agree with Alexandre
> >and please don't pull the plug" so a point of clarification. We have
> >no intention of shutting down the BK free product. We are aware that
> >there are 10's of thousands of developers in the open source world
> >who do not agree with Alexandre's narrow view of things. You're fine,
> >we're not taking BK away. I only trying to get Alexandre to see that
> >his definition of "help" is somewhat narrow-minded.
> >
>
> Then why don't you stop threatening to take it away every time someone
> points out to you that your "help" for free software isn't ideal? Just
> can't help yourself? Your cheap shot at Alexandre doesn't change the fact
> that you've shown yet again why people who believe in free software should
> work to replace BK.

Maybe because I'm a little sick and tired of all the "BK is evil" crap
without any accompanying "thank your all the good it's done"?

If every time someone sent you a bug report on your open source software
it started with "this software sucks, and you're a pinko anti-capitalist
commy" how long would it be before you stopped giving it away? Don't
bother to answer that, I know you are perfect and that you'd keep doing
it any way because yours is The One True Way (tm).
--
---
Larry McVoy lm at bitmover.com http://www.bitkeeper.com

2005-02-11 20:00:55