Hi folks!!
Just to let you know I created a bitkeeper repository for the
framebuffer layer. Anyone wanting to add stuff please send me a key
generated by ssh-keygen. Thanks.
. ---
|o_o |
|:_/ | Give Micro$oft the Bird!!!!
// \ \ Use Linux!!!!
(| | )
/'_ _/`\
___)=(___/
* James Simmons <[email protected]> on Tue, Apr 16, 2002:
>
> Hi folks!!
>
> Just to let you know I created a bitkeeper repository for the
> framebuffer layer. Anyone wanting to add stuff please send me a key
> generated by ssh-keygen. Thanks.
>
Please tell us that primary framebuffer/input/console development will
continue in the CVS drop-in tree on SourceForge? Bitkeeper is unable to
support this (easier, more efficient) style of development.
M. R.
> Please tell us that primary framebuffer/input/console development will
> continue in the CVS drop-in tree on SourceForge? Bitkeeper is unable to
> support this (easier, more efficient) style of development.
Could you please explain why you think CVS is easier and more efficient?
Last I checked, BK was a superset of CVS, but could be used pretty much
identically to CVS if that's what you want.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
* Larry McVoy <[email protected]> on Tue, Apr 16, 2002:
> > Please tell us that primary framebuffer/input/console development will
> > continue in the CVS drop-in tree on SourceForge? Bitkeeper is unable to
> > support this (easier, more efficient) style of development.
>
> Could you please explain why you think CVS is easier and more efficient?
> Last I checked, BK was a superset of CVS, but could be used pretty much
> identically to CVS if that's what you want.
A drop-in tree (also called "shadow trees" by Keith Owens of kbuild), is a
small set of files intended to be applied against a larger parent body of
code. For example, a kernel subsystem or backend project (linuxconsole,
LinuxSH, Linux-MIPS) will only maintain the minimal number of files that
are specific to that backend, e.g. include/asm-mips/, arch/mips,
/arch/mips64, etc. for any files local to the project. These could also be
driver files that have architecture-specific changes that haven't been sent
upstream yet. Because drop-in trees only contain the files that the
project developers care about, they tend to be much smaller and easier to
maintain. A drop-in tree is usually applied either by copying it over the
stock kernel tree (older method) or by using a simple script that symlinks
the drop-in tree contents (preferred).
I haven't seen a lot of formalization of the basic drop-in tree concepts,
but there are numerous projects that use this style even if they don't
call them drop-in trees, including linux1394. These types of projects
usually maintain a single directory that is "cp -r"'d on top of the
corresponding directory in the full stock kernel.
From my understanding of Bitkeeper, you can only clone from the master
repository, and you cannot specify a subset of files to work on when doing
so. Therefore, if you only want to modify the files that pertain to your
backend port, you must first clone the entire Linux tree (or whatever was
imported into the master repository) and then make your local changes. This
is a bad thing for a couple of reasons. It becomes a maintainence nightmare
when resolving conflicts with files outside of your project's scope. If I
maintain the console subsystem, why should I care about the upheaval of SCSI
layer? Because CVS is simple enough to not require you to clone an entire
repository first, this is never an issue with using drop-in trees.
Secondly, size of the working repository and the size of updates becomes an
issue. The kernel source sitting on my HD is approximately 151M. Changes
between -pre versions usually range from 5-10MB. Am I supposed to waste this
much bandwidth just to clone the master repository and pull updates?
This is why CVS is more suited for this "style" of development. Purely
technical reasons, not religious. Now if James is using the drop-in tree
as the import files in the BK repository, versus a full kernel tree, then
that's a different matter. But I'm really not interested in complex BK
commands that will somewhat allow the development of drop-in trees (if such
commands even exist). It may seem silly to you, but I prefer KISS methods
of development and CVS is just fine for what we're trying to do in the
kernel backends. BK is overkill and an impedement.
Granted a lot of the responsibilty of submitting patches and sync'ing falls
on the maintainers shoulders, so perhaps my argument falls on deaf ears
where they are concerned (as they are more concerned with keeping up with
Linus than with project developers such as myself).
[Note I didn't go into any detail about how drop-in trees are tagged
and sync'd against the stock kernels...]
M. R.
On Tue, Apr 16, 2002 at 07:08:18PM -0500, M. R. Brown wrote:
> * Larry McVoy <[email protected]> on Tue, Apr 16, 2002:
>
> > > Please tell us that primary framebuffer/input/console development will
> > > continue in the CVS drop-in tree on SourceForge? Bitkeeper is unable to
> > > support this (easier, more efficient) style of development.
> >
> > Could you please explain why you think CVS is easier and more efficient?
> > Last I checked, BK was a superset of CVS, but could be used pretty much
> > identically to CVS if that's what you want.
>
> A drop-in tree (also called "shadow trees" by Keith Owens of kbuild), is a
> small set of files intended to be applied against a larger parent body of
> code. For example, a kernel subsystem or backend project (linuxconsole,
> LinuxSH, Linux-MIPS) will only maintain the minimal number of files that
> are specific to that backend, e.g. include/asm-mips/, arch/mips,
> /arch/mips64, etc. for any files local to the project.
Ahh, OK, we're already working on this. We call 'em nested repositories
and one of the problems they solve is exactly the problem you described.
Think of them as CVS modules, with a little more formality, and you're
about there. They also solve a bunch of performance problems.
I tend to agree with your comments about not wanting the whole tree, to
some extent. You are aware, of course, that your drop in may not work
if the rest of the tree has moved on. So the drop in has a limited
life span in isolation. With that caveat, drop ins are nice and we'll
have them before too long. Unlike CVS, we like to be able to reproduce
the tree accurately so there is more work to do.
On your comments about CVS being less complex, I don't agree at all.
Almost all of the BK complexity is to handle problems CVS doesn't
handle at all. Another way to say that is when you hit those problems
BK is much much less complex that CVS. For example, a simple file
rename is a nightmare in CVS and a non-issue in BK, it just propogates.
If you want to eliminate learning "bk mv foo.c bar.c", just don't do
that and all of that complexity is never used.
I'll be the first to admit that BK is a big system, but it's no more
complex than CVS if you limit yourself to CVS-like operations. And
when you go beyond those limits, then BK becomes less complex to the
user just as CVS is starting to fall over. Or am I missing something?
Have you read http://www.bitkeeper.com/cvs2bk.html ? That covers the
translation.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
* Larry McVoy <[email protected]> on Tue, Apr 16, 2002:
> >
> > A drop-in tree (also called "shadow trees" by Keith Owens of kbuild), is a
> > small set of files intended to be applied against a larger parent body of
> > code. For example, a kernel subsystem or backend project (linuxconsole,
> > LinuxSH, Linux-MIPS) will only maintain the minimal number of files that
> > are specific to that backend, e.g. include/asm-mips/, arch/mips,
> > /arch/mips64, etc. for any files local to the project.
>
> Ahh, OK, we're already working on this. We call 'em nested repositories
> and one of the problems they solve is exactly the problem you described.
> Think of them as CVS modules, with a little more formality, and you're
> about there. They also solve a bunch of performance problems.
>
That's good to hear, and interesting as well. Any particular reason why
this wasn't implemented until now? Did you not see a high demand for
satellite projects, or was it just assumed that you'd only ever need the
clone method of development?
> I tend to agree with your comments about not wanting the whole tree, to
> some extent. You are aware, of course, that your drop in may not work
> if the rest of the tree has moved on. So the drop in has a limited
> life span in isolation. With that caveat, drop ins are nice and we'll
> have them before too long. Unlike CVS, we like to be able to reproduce
> the tree accurately so there is more work to do.
>
This requires consistency on the part of the drop-in tree maintainer, to
sync to and from the mainline kernel. Usually, only the maintainer sends
patches upstream, but experienced project developers can easily merge
changes coming from the mainline kernel. I'm not aware of how BK may
automate this or make it easier, but this is one of those "use rigid
guidelines where CVS falls short" situations. *Shrug*, a bit of extra
work, but still easier than tracking an entire tree's worth of changes.
> On your comments about CVS being less complex, I don't agree at all.
> Almost all of the BK complexity is to handle problems CVS doesn't
> handle at all. Another way to say that is when you hit those problems
> BK is much much less complex that CVS. For example, a simple file
> rename is a nightmare in CVS and a non-issue in BK, it just propogates.
>
I meant as far as styles of development. Yes, CVS can cause some headaches
if used improperly, and some shortcomings are either excused or hacked
over. However for small projects or projects with a group of close-knit
developers BK is considerably more complex to setup and use. A lot of the
"problems" that BK solves are either not required for smaller projects or
are second nature to those experienced in using CVS.
Yes, I see the immediate benefits of being able to clone a repository and
do local development, pulling in updates or pushing changes mainstream
where appropiate. Fortunately, I have Arch to study for this, which I have
to say I'm much more comfortable working with. Yes, I've used BK in the
past (for the PPC tree), but it's been about 2 years so maybe it's improved
since then?
> If you want to eliminate learning "bk mv foo.c bar.c", just don't do
> that and all of that complexity is never used.
>
> I'll be the first to admit that BK is a big system, but it's no more
> complex than CVS if you limit yourself to CVS-like operations. And
> when you go beyond those limits, then BK becomes less complex to the
> user just as CVS is starting to fall over. Or am I missing something?
> Have you read http://www.bitkeeper.com/cvs2bk.html ? That covers the
> translation.
Right, but CVS-like operations in BK currently won't allow me to do drop-in
trees. It's not so much a complexity of use issue as it's an issue of how
things are implemented or done in CVS vs. BK. Because of its simplistic
view of repositories and how to work with them, CVS wins with small, finely
cultivated (excuse the lack of a better term) trees, whereas BK's complex
view of repositories does not.
I'm not trying to argue as to which is better, I'm just trying to make the
point that I think it's better to use whatever tool is best suited for the
task. Preaching the benefits of BK over CVS makes no sense when the
desired features aren't there or usuable, so unfortunately your blanket
statements of "BK is a superset of CVS" don't do me any good. Hopefully this
will filter up to the kernel backend/subsystem maintainers before we see more
drop-in trees needlessly turning into BK repositories.
One of the listed BK features is being able to collaborate with other folks
who've cloned a repository - are the backend/subsystem maintainers even
doing this? Is there any collaboration going on between say, the Linux/ARM
and USB BK repositories? Or is the plan to let things get sorted out when
changes are pushed straight to Linus? Any maintainers want to comment on the
usefulness (or lack thereof) of BK for maintaining a kernel port (anybody
except Larry)? Is the fact that syncing with Linus has become
significantly easier the sole reason that BK has become so widespread?
M. R.
On Tue, Apr 16, 2002 at 09:41:49PM -0500, M. R. Brown wrote:
> * Larry McVoy <[email protected]> on Tue, Apr 16, 2002:
> > Ahh, OK, we're already working on this. We call 'em nested repositories
>
> That's good to hear, and interesting as well. Any particular reason why
> this wasn't implemented until now? Did you not see a high demand for
> satellite projects, or was it just assumed that you'd only ever need the
> clone method of development?
BK supports gzipped repositories and hard linked repositories, so a
compressed tree takes about 87MB (when the checked out tree is 169MB),
and a hard linked tree takes effectively no space at all. So making
subsets had very little benefit and removed a significant advantage:
with BK when you say "as of such and such a changeset", there is no
ambiguity as to what you mean, it implies the whole tree. There is no
chance of doing that with a subset tree. Small gain in disk space,
big loss in reproduceablilty. Seems like a bad tradeoff.
> This requires consistency on the part of the drop-in tree maintainer, to
> sync to and from the mainline kernel. Usually, only the maintainer sends
> patches upstream, but experienced project developers can easily merge
> changes coming from the mainline kernel. I'm not aware of how BK may
> automate this or make it easier, but this is one of those "use rigid
> guidelines where CVS falls short" situations. *Shrug*, a bit of extra
> work, but still easier than tracking an entire tree's worth of changes.
That's not a widely shared opinion. Check with the people maintaining
their own trees, tracking the mainline and/or some branch. Saying
"CVS falls short" is quite the understatement. Try doing the same
thing with CVS (or whatever else) and you'll see what I mean. CVS
is great for simple things; the kernel isn't simple.
There is a basic flaw in your reasoning. You're assuming that you
can do all your work in isolation. That's not true, the rest of the
tree is changing around you and you need to build/test against that.
Not doing so is just cutting corners and while I'm sure you are smart
enough not to get burned too often by this, I'm equally sure you have
and will get burned by it again. Noone is immune. And all of this is
to save some disk space? Disk drives are about $1/GB on pricewatch.
A kernel tree, compressed, takes less than .1GB. So you saved yourself
10 whole cents. Wheeeeeee!
Suppose you are a paid developer. Around here, a kernel hack gets about
$100K/year, and actually costs a company about $180K/year. OK, let's say
you live someplace really cheap so it is 1/4th that. So that's $22.5
an hour. Or 37.5 cents a minute. So if you waste 20 seconds because
you saved yourself 100MB of space, you blew it. It doesn't make sense.
> I meant as far as styles of development. Yes, CVS can cause some headaches
> if used improperly, and some shortcomings are either excused or hacked
> over. However for small projects or projects with a group of close-knit
> developers BK is considerably more complex to setup and use. A lot of the
> "problems" that BK solves are either not required for smaller projects or
> are second nature to those experienced in using CVS.
>
> Yes, I see the immediate benefits of being able to clone a repository and
> do local development, pulling in updates or pushing changes mainstream
> where appropiate. Fortunately, I have Arch to study for this, which I have
> to say I'm much more comfortable working with. Yes, I've used BK in the
> past (for the PPC tree), but it's been about 2 years so maybe it's improved
> since then?
So you are looking at a 2 year old view of a a tool which has existed
for 4 years. Yes, of course it has improved, it has improved a lot.
How about you go use it and then make your comments? Come on, get a clue.
Judging anything which is moving that fast by a 2 year old snapshot is
obviously wrong.
A sane person might conclude you have some other agenda.
> Right, but CVS-like operations in BK currently won't allow me to do drop-in
> trees. It's not so much a complexity of use issue as it's an issue of how
> things are implemented or done in CVS vs. BK. Because of its simplistic
> view of repositories and how to work with them, CVS wins with small, finely
> cultivated (excuse the lack of a better term) trees, whereas BK's complex
> view of repositories does not.
Nonsense. You can do drops right now. Just split the tree apart and
clone the parts you need. All the nested repos are going to do is
give you some syntatic sugar which allows you to do just that.
Note that I'm not suggesting that you personally do that, it's becoming
apparent to me that you either have an ax to grind or you are just plain
lazy, and either way I'd love for you to keep using CVS, that would be
great.
At the same time, it would also be great if you left off describing tools
you don't use. I don't know what part of the kernel you work on, but
I can assure you that I won't go complaining about it unless I actually
use it and think about it first.
> One of the listed BK features is being able to collaborate with other folks
> who've cloned a repository - are the backend/subsystem maintainers even
> doing this? Is there any collaboration going on between say, the Linux/ARM
> and USB BK repositories?
There is certainly some of that, look at the histories to see the merges.
I think Linus does most of those but I see merges originated by others
in their private trees all the time, look at the openlogging tree.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
* Larry McVoy <[email protected]> on Tue, Apr 16, 2002:
>
> > This requires consistency on the part of the drop-in tree maintainer, to
> > sync to and from the mainline kernel. Usually, only the maintainer sends
> > patches upstream, but experienced project developers can easily merge
> > changes coming from the mainline kernel. I'm not aware of how BK may
> > automate this or make it easier, but this is one of those "use rigid
> > guidelines where CVS falls short" situations. *Shrug*, a bit of extra
> > work, but still easier than tracking an entire tree's worth of changes.
>
> That's not a widely shared opinion. Check with the people maintaining
> their own trees, tracking the mainline and/or some branch. Saying
> "CVS falls short" is quite the understatement. Try doing the same
> thing with CVS (or whatever else) and you'll see what I mean. CVS
> is great for simple things; the kernel isn't simple.
>
Not sure I follow here, you're a bit vague about "doing the same thing".
Do you mean syncing from the mainline kernel? It's easier to sync from
Linus because BK is being used to do it (e.g. because the Linus kernel is
tracked by BK?)?
For the LinuxSH project, we moved from tracking the full kernel tree (via
CVS) to a drop-in tree structure mainly because of the difficulty in
tracking all upstream kernel changes at once. It wasn't really related to
what CVS did or did not offer.
> There is a basic flaw in your reasoning. You're assuming that you
> can do all your work in isolation. That's not true, the rest of the
> tree is changing around you and you need to build/test against that.
> Not doing so is just cutting corners and while I'm sure you are smart
> enough not to get burned too often by this, I'm equally sure you have
> and will get burned by it again. Noone is immune. And all of this is
> to save some disk space? Disk drives are about $1/GB on pricewatch.
> A kernel tree, compressed, takes less than .1GB. So you saved yourself
> 10 whole cents. Wheeeeeee!
>
I never said anything about working in isolation in any of my posts. A
drop-in tree is only usable when "dropped in" or symlinked on top of a full
kernel tree. Because it's not a complete tree, it won't build otherwise.
In order to build and test a drop-in tree, you must have the current
mainline kernel version to drop against (whatever kernel version that has
been specified in the drop-in tree). If something breaks upstream
(which happens more often than not, esp. in architecture ports), then
you'll see it as soon your build fails or when your kernel panics. It's
impossible to develop a drop-in tree in complete isolation, so I'm not sure
why you thought this was possible?
Say cache handling is modified kernel-wide and it breaks a number of
architecture backends (this is a real world example, somewhere around
2.4.10 or so this actually happened). Until you work out a fix for say,
the SuperH architecture, you pull in and modify the global cache-handling
files and modify it locally. Once a proper fix has been developed, you
send your arch-specific cache routines upstream and remove the global
cache-handling files from your drop-in tree.
Also, the size of a drop-in tree is a side effect of its composition, it
was used to illustrate the benefit of the simplicity of drop-in trees - if
you recall the introductory paragraph of my original reply to yours, I said
that drop-in trees are most useful for kernel subsystems and backend
(architecture) ports. The number of files that are specific to these
projects are a fraction of the size of the full kernel, and because of
this, it gives you a simpler, more readable view of your "corner" of the
kernel. I don't know why you assumed I was on some fanatical quest to save
disk space. However, I do mind how much of my broadband bandwidth is used
(since I'm stuck with a flaky ADSL link) and I'm sure developers still on 56K
links also mind.
[snip]
>
> > Right, but CVS-like operations in BK currently won't allow me to do drop-in
> > trees. It's not so much a complexity of use issue as it's an issue of how
> > things are implemented or done in CVS vs. BK. Because of its simplistic
> > view of repositories and how to work with them, CVS wins with small, finely
> > cultivated (excuse the lack of a better term) trees, whereas BK's complex
> > view of repositories does not.
>
> Nonsense. You can do drops right now. Just split the tree apart and
> clone the parts you need. All the nested repos are going to do is
> give you some syntatic sugar which allows you to do just that.
>
Cool, finally you give a straight answer to an honest query. You orignally
asked why I thought BK was inefficient as opposed to CVS, so I assume you
misunderstood my response to James. I'm operating on the available info I
have on BK, where cloning is an all or nothing operation. Is this not the
case? Can you selectively pull in parts of the tree (similiar to drop-in
functionality) and is anyone doing it? This is why I was opposed to the
linuxconsole drop-in tree moving to BK, not because of "some other agenda"
but because it screws up how I develop and test the tree. I guess in
retrospect, you really didn't have a valid entrance into the thread, but I
guess any post including "Bitkeeper" catches your eye?
So, splitting trees in BK. Do you mean by using native BK commands, or by
planning ahead and using multiple repositories? Perhaps when you've
finished flaming me and defaming CVS you can give me a valid response as to
how one would maintain a drop-in tree using BK. The isolation argument
doesn't fly since drop-in trees are far from isolated, and the disk space
thing was taken a bit out of context, but that's understandable.
You seem to be stuck on "why CVS sucks" while I'm trying to convey "why CVS
does what I need it to do" as far as drop-in trees are concerned. From
every other pro-BK anti-everyother-RCS post I've read from you, you seem
conditioned to attack (or belittle) anyone that doesn't accept BK as
gospel. I understand taking pride in your work, but damn! Take it easy.
If you really wanted to help me (or convert me) to using BK, you'd just
tell me how the heck to do a drop-in tree, and I'd walk away enlightened.
[snide personal insults snipped]
>
> At the same time, it would also be great if you left off describing tools
> you don't use. I don't know what part of the kernel you work on, but
> I can assure you that I won't go complaining about it unless I actually
> use it and think about it first.
>
I was able to ascertain that BK didn't support drop-in trees by just
reading http://www.bitmover.com. Was there somewhere else I should've been
instead?
M. R.
On Wed, Apr 17, 2002 at 12:04:12AM -0500, M. R. Brown wrote:
> Not sure I follow here, you're a bit vague about "doing the same thing".
> Do you mean syncing from the mainline kernel?
Syncing from the main kernel, synching from someone else, syncing with
a coworker, whatever. It doesn't really matter. With the limitation
that BK wants you to eat everything that the other guy has that you don't,
it all just works. And you can sync from N different places all of which
happened to have the same patch in the same changeset, and it gets applied
once, BK knows it has it already.
BK isn't perfect, but you are looking at it through CVS colored glasses.
Take 'em off, it's got some stuff that will save you time and effort.
> For the LinuxSH project, we moved from tracking the full kernel tree (via
> CVS) to a drop-in tree structure mainly because of the difficulty in
> tracking all upstream kernel changes at once. It wasn't really related to
> what CVS did or did not offer.
Disagree. CVS sucks at tracking an external source of change while
maintaining your own changes at the same time, in the same tree.
It really sucks at that, you are forced into a broken branching model
with awful merge characteristics. Try the same thing with BK, it's
a lot nicer, most stuff just automerges, and the stuff that doesn't
only needs to be merged once. Your claim that it isn't about CVS
doesn't make much sense when CVS forces you to split your subtree
out. Having an _option_ to do that is one thing, being forced to
do it is another.
> I never said anything about working in isolation in any of my posts. A
> drop-in tree is only usable when "dropped in" or symlinked on top of a full
> kernel tree.
And if that tree has changes in the files that you are "dropping in"?
You just stomped on them. Or you are left with an even more broken
merge model than CVS. And what about file renames.
> Also, the size of a drop-in tree is a side effect of its composition, it
> was used to illustrate the benefit of the simplicity of drop-in trees - if
> you recall the introductory paragraph of my original reply to yours, I said
> that drop-in trees are most useful for kernel subsystems and backend
> (architecture) ports.
Agreed. We're headed towards breaking up the kernel tree into
subrepositories for that reason.
> However, I do mind how much of my broadband bandwidth is used
> (since I'm stuck with a flaky ADSL link) and I'm sure developers still on 56K
> links also mind.
BK's pretty stingy with bandwidth. I think the openlogging tree has
some problems, but other than that, BK uses very little bandwidth, far
less than CVS. It knows what needs to be transfered, CVS has to look,
over your net connection. We have lots of people tracking the kernel
trees via a modem. You can play games to save more bandwidth too,
if you know you have a common gca in a local tree, and you want to get
a local copy of some other "branch", clone -r back to the gca and then
pull from the branch into the new repository. You'll save transfering
all the stuff that you already had.
> So, splitting trees in BK. Do you mean by using native BK commands, or by
> planning ahead and using multiple repositories? Perhaps when you've
> finished flaming me and defaming CVS you can give me a valid response as to
> how one would maintain a drop-in tree using BK.
Perhaps when you have finished flaming me, you can go use the tool, read
the docs, and learn what it can do. Part of the problem I have is that
you are, in a public forum, saying that BK is this and that, based on your
perceptions from 2 years ago when you said you used it on the ppc tree.
I can't imagine that you used it very much, because there is not a
single changeset in the tree with your login on it. So maybe you are
making your comments based on a 2 year old reading of the docs, rather
than actually using it. That is pretty annoying and a waste of time.
Go learn the tool, if you want an education, then buy some training.
As for splitting trees, "bk help csetprune". It not only splits trees,
it maintains the repository changeset history graph, no small feat.
But it's a 1-way, 1-time operation, you can't unsplit. So it's something
that I suspect will happen when Linus wants it.
And you could find this information by
http://www.bitkeeper.com
click on search
type in "split"
hit return,
it's the 3rd item down.
> You seem to be stuck on "why CVS sucks" while I'm trying to convey "why CVS
> does what I need it to do" as far as drop-in trees are concerned.
Except it *doesn't* do what you need it to do. You proceeded to hand
wave over the many places where CVS won't work. You haven't addressed
either of the two big problems, renames and content changes in the
upstream tree. In the 2 months of usage, Linus' tree has seen 1289
renames, of which 767 where real renames, not deletes. How does your
drop in model handle those? What about content changes that you don't
have yet, how does it handle that? In both cases, the answer is "it
doesn't". So you are doing work that you don't need to do, by hand.
That seems misguided when there is a tool that will do that work for
you.
> conditioned to attack (or belittle) anyone that doesn't accept BK as
> gospel. I understand taking pride in your work, but damn! Take it easy.
It's got nothing to do with gospel, it's got everything to do with you
making claims based on no work on your part, not even recent usage.
BK isn't anywhere near Linus' definition of the best (i.e., it can't
get better). Nowhere near. But it's quite useful if you figure out
how to use it. And we're making better as fast as we can.
Contrast your comments / homework with Jeff Garzik, just for fun. BK
has some limitations he didn't like, so he asked a few polite questions,
figured out how to work around the limitations, and wrote it up in a
doc (have you read it?). His approach has been widely adopted, there
are at last count around 130 slightly different BK trees sitting on
bkbits.net.
In summary, because I've had enough of this thread, your drop in tree
is a hack, it doesn't handle even the basics of SCM, and you haven't
shown how to handle those. BK has plenty of problems, but it least it
handles the basics. What you are doing is like working in a file system
which makes you edit the raw disk to do renames and/or content merges.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
Hi,
On Tue, 16 Apr 2002, Larry McVoy wrote:
> [bk commercials removed]
Am I the only who wished hearing you say only once "Hey, this offtopic
here, let's move the discussion to the right ml."?
bye, Roman
* Larry McVoy <[email protected]> on Tue, Apr 16, 2002:
> On Wed, Apr 17, 2002 at 12:04:12AM -0500, M. R. Brown wrote:
> > Not sure I follow here, you're a bit vague about "doing the same thing".
> > Do you mean syncing from the mainline kernel?
>
> Syncing from the main kernel, synching from someone else, syncing with
> a coworker, whatever. It doesn't really matter. With the limitation
> that BK wants you to eat everything that the other guy has that you don't,
> it all just works. And you can sync from N different places all of which
> happened to have the same patch in the same changeset, and it gets applied
> once, BK knows it has it already.
>
> BK isn't perfect, but you are looking at it through CVS colored glasses.
> Take 'em off, it's got some stuff that will save you time and effort.
>
That's cool. That's what should be in a SCM. I guess we may have a
misunderstanding where I say "drop-in tree" and you assume I mean using CVS
to tag and branch a subset of the kernel source? The drop-in tree is what I
described in my initial response to you so many posts ago. It doesn't have
to be specifically tied to a CVS-centric mode of development, but before some
of the other SCMs came into fruition, CVS was the easiest (or most popular
if you insist it wasn't easy) way to do it. Now we have advanced SCMs
available, and I hope to use one that will still support drop-in trees
while allowing me access to all the whizbang features that will make my
life easier.
> > For the LinuxSH project, we moved from tracking the full kernel tree (via
> > CVS) to a drop-in tree structure mainly because of the difficulty in
> > tracking all upstream kernel changes at once. It wasn't really related to
> > what CVS did or did not offer.
>
> Disagree. CVS sucks at tracking an external source of change while
> maintaining your own changes at the same time, in the same tree.
> It really sucks at that, you are forced into a broken branching model
> with awful merge characteristics. Try the same thing with BK, it's
> a lot nicer, most stuff just automerges, and the stuff that doesn't
> only needs to be merged once. Your claim that it isn't about CVS
> doesn't make much sense when CVS forces you to split your subtree
> out. Having an _option_ to do that is one thing, being forced to
> do it is another.
>
I'd hate to sound like I'm trolling or being a broken record. But I don't
currently have that option with BK. This is the option I want, to be able
to not have to worry about pulling all changes or being able to cordon off
only the bit I want/need to work with. See the above as to why CVS isn't
just the means, it's just the tool currently used for drop-in maintenance.
Give me proper drop-in support using BK (or any other modern SCM tool), and
I'm sold.
> > I never said anything about working in isolation in any of my posts. A
> > drop-in tree is only usable when "dropped in" or symlinked on top of a full
> > kernel tree.
>
> And if that tree has changes in the files that you are "dropping in"?
> You just stomped on them. Or you are left with an even more broken
> merge model than CVS. And what about file renames.
>
When my drop-in tree file changes upstream, I'll find out in the next
upstream sync. Obviously if my drop-in tree is current, that file hasn't
changed yet. File renames are broken, but I never pretended they weren't. I
guess what you're saying is it comes down to me weighing out feature-rich
set vs. feature-less set, where BK is the former, and I should expect to have
to sacrifice my entire development model (drop-in trees, not CVS) for the
bloated model that BK currently *enforces* (see your above paragraph about
having the option).
> > Also, the size of a drop-in tree is a side effect of its composition, it
> > was used to illustrate the benefit of the simplicity of drop-in trees - if
> > you recall the introductory paragraph of my original reply to yours, I said
> > that drop-in trees are most useful for kernel subsystems and backend
> > (architecture) ports.
>
> Agreed. We're headed towards breaking up the kernel tree into
> subrepositories for that reason.
>
This was already being done with the satellite projects and drop-in
trees, but didn't have the benefit of being SCM-controlled *globally*. I'd
very much like to see this happen. Was there any timeframe on this or
should I just start searching lkml archives?
>
> > So, splitting trees in BK. Do you mean by using native BK commands, or by
> > planning ahead and using multiple repositories? Perhaps when you've
> > finished flaming me and defaming CVS you can give me a valid response as to
> > how one would maintain a drop-in tree using BK.
>
> Perhaps when you have finished flaming me, you can go use the tool, read
> the docs, and learn what it can do. Part of the problem I have is that
> you are, in a public forum, saying that BK is this and that, based on your
> perceptions from 2 years ago when you said you used it on the ppc tree.
> I can't imagine that you used it very much, because there is not a
> single changeset in the tree with your login on it. So maybe you are
> making your comments based on a 2 year old reading of the docs, rather
> than actually using it. That is pretty annoying and a waste of time.
> Go learn the tool, if you want an education, then buy some training.
>
Nope, I said that based on what I read at http://www.bitkeeper.com, BK didn't
support drop-in trees *today*. I said that the last time I actively used
the tool was 2 years ago, but I haven't had a reason to use it since then.
In a public forum, which you use to promote sales and use of your product, I
stated quite simply and plainly that BK won't allow me to do drop-in trees,
with the simplicity of CVS. Even using a full kernel source tree as CVS
HEAD and branching subsets of HEAD into small drop-in like branches does
what I need to do in CVS. This is even before you start worrying about
automerging and file renaming - I can't do the above using BK - for kernel
development.
Two years ago, or today, BK still doesn't do it, not without requiring
massive structural changes to the master repository by the project owner -
not the satellite maintainers. This requires a significant amount of
planning ahead and effort from Linus (or whoever ends up doing it), where
using the drop-in tree model that burden is placed on the satellite
maintainer. When I decide to start and maintain the "linux toaster
driver", I pull out the source I need from mainline, and setup my drop-in
tree from that and my new imports. The BK way to do it is to simply "clone
the master" and work from that. Not a drop-in.
I voiced a legitimate concern about why drop-in trees were useful, and why
I wouldn't be able to contiue their development using BK. You proceeded (for
a couple of posts at least) to dodge that concern with arguments of why CVS
pales in comparison to BK. That I didn't really care about, and maybe you
should be asking yourself, "If I didn't have the correct answer and I can't
accurately address this guy's concern, why did I bother replying in the first
place?"
> As for splitting trees, "bk help csetprune". It not only splits trees,
> it maintains the repository changeset history graph, no small feat.
> But it's a 1-way, 1-time operation, you can't unsplit. So it's something
> that I suspect will happen when Linus wants it.
>
> And you could find this information by
>
Right. We can't do it, only the repository owner can. By your own
admission the drop-in tree as I described it isn't supported:
"Ahh, OK, we're already working on this. We call 'em nested repositories
and one of the problems they solve is exactly the problem you described.
Think of them as CVS modules, with a little more formality, and you're
about there. They also solve a bunch of performance problems."
The "csetprune" command you've pointed me to apparently doesn't do this.
> > You seem to be stuck on "why CVS sucks" while I'm trying to convey "why CVS
> > does what I need it to do" as far as drop-in trees are concerned.
>
> Except it *doesn't* do what you need it to do. You proceeded to hand
> wave over the many places where CVS won't work. You haven't addressed
> either of the two big problems, renames and content changes in the
> upstream tree. In the 2 months of usage, Linus' tree has seen 1289
> renames, of which 767 where real renames, not deletes. How does your
> drop in model handle those? What about content changes that you don't
> have yet, how does it handle that? In both cases, the answer is "it
> doesn't". So you are doing work that you don't need to do, by hand.
> That seems misguided when there is a tool that will do that work for
> you.
>
Yep, you're correct, except about the handwaving part. Last I checked, 2.2
was still being maintained, and 2.4 is also being maintained. Believe it or
not, I have to be bothered with tracking changes between all of these
versions *and* 2.5, and at least the 2.2 variant still needs to be done by
hand. 2.4 and 2.5 are now SCM-controlled by BK. That's great, and I'm
being serious here. But it can't cope with the drop-in style, and at this
point I can't be coerced into a new mode of development. If BK could sanely
handle drop-in trees (the right way), trust me, I'd already be using it. But
I'm still forced to do the merging work by hand, if I insist on maintaining
the drop-in tree (without CVS), even if using a BK clone as my source base.
My tool needs to do what I tell it to, not the other way around.
> > conditioned to attack (or belittle) anyone that doesn't accept BK as
> > gospel. I understand taking pride in your work, but damn! Take it easy.
>
> It's got nothing to do with gospel, it's got everything to do with you
> making claims based on no work on your part, not even recent usage.
> BK isn't anywhere near Linus' definition of the best (i.e., it can't
> get better). Nowhere near. But it's quite useful if you figure out
> how to use it. And we're making better as fast as we can.
>
If you were able to propose a sane way to continue using the drop-in model
using BK, then I'd have no problems taking you seriously. My only claim
was that BK couldn't handle drop-in trees. I didn't say it sucked overall
as a SCM or that I wanted to protest because it wasn't open. The "work" was
me perusing the bitkeeper webpage, and going back and forth with you, the guy
I thought would be the most intelligent on all of BK's features. Are you
saying the only way for me to evaluate a tool (one that I haven't used in
ages) is to use it? I can't rely on documentation or peer reviews and
comments? Wouldn't my time be spared if you just told me what it would or
wouldn't support?
> Contrast your comments / homework with Jeff Garzik, just for fun. BK
> has some limitations he didn't like, so he asked a few polite questions,
> figured out how to work around the limitations, and wrote it up in a
> doc (have you read it?). His approach has been widely adopted, there
> are at last count around 130 slightly different BK trees sitting on
> bkbits.net.
>
This?
http://www.uwsg.iu.edu/hypermail/linux/kernel/0202.2/1060.html
It was a good read, and easy to follow, but it still does the opposite of
what I need to do. I want multiple small project-specific trees, not a
myriad of full kernel trees. Everything else gave me a good starting point
for doing SCM-controlled kernel development using BK. Thanks for the
pointer.
> In summary, because I've had enough of this thread, your drop in tree
> is a hack, it doesn't handle even the basics of SCM, and you haven't
> shown how to handle those. BK has plenty of problems, but it least it
> handles the basics. What you are doing is like working in a file system
> which makes you edit the raw disk to do renames and/or content merges.
Point taken about the drop-in tree being a hack, when used within the
limitations of CVS. With a SCM that could do it intelligently, it would be
overly useful, but unfortunately BK can't do it at all (or, it does it in
the complete reverse).
I already have been looking at alternatives to CVS that would allow me to
maintain a master kernel source repository with many satellite "branches" that
would be equivalent to small drop-in trees for the various kernel ports and
subsystems. I would also need a reasonable way to do merges between the
satellites and the trunk, and being able to do it locally with cloning would
be a major plus.
I've found that Arch comes the closest to what I need, so I've been "doing
my homework" with it. When I do run into snags, or when Arch trips over
itself when I'm trying to maintain my satellites at least I do have the source
instead of the requirement that I bitch to you about missing functionality. I
suppose for the project I'm working on, once it gets to the point where I
need to grab up to the minute changes, I can be bothered with setting up
either an Arch<->BK gateway or find another means to migrate updates from
the latter.
If BK can gain (intelligent, where satellite maintainers can split, not
just root) drop-in support to where it doesn't make sense for me to use
Arch for Linux kernel projects, I'd start using it (and recommend it) for my
satellite projects immediately. Again, for me it boils down to using the
best tool for the job.
M. R.
On Tue, Apr 16, 2002 at 08:37:21PM -0700, Larry McVoy wrote:
> and will get burned by it again. Noone is immune. And all of this is
> to save some disk space? Disk drives are about $1/GB on pricewatch.
> A kernel tree, compressed, takes less than .1GB. So you saved yourself
> 10 whole cents. Wheeeeeee!
Could you just stop throwing around useless arguments.
Where is that $1/GB drive that can replace the 800MB drive in my Compaq
1120 laptop? There isn't any, because Compaq locked the drive geometry
in the bios to only work with this one particular 800MB drive that isn't
produced anymore. And that drive, even if I find another one, is
definitely not going to be 80 cents.
> Suppose you are a paid developer. Around here, a kernel hack gets about
> $100K/year, and actually costs a company about $180K/year. OK, let's say
> you live someplace really cheap so it is 1/4th that. So that's $22.5
> an hour. Or 37.5 cents a minute. So if you waste 20 seconds because
> you saved yourself 100MB of space, you blew it. It doesn't make sense.
Actually, driving out to the store to get a bigger drive, making a backup
of the old drive, opening the machine and putting the new drive in, then
restoring the backup from tape. I'd say that would take an afternoon, if
the streamer is fast enough. I'd rather spend a couple of seconds
avoiding the amount of bloat.
Another thing, here in the US people are pretty used to having a flat
rate for dialup, but what about people that have to pay by the minute
for their 5KB/s dialup connection. Saving 100MB in data transfer will
not just save them 5 hours and 41 minutes of download time, but also
a whole lot in phone charges.
> A sane person might conclude you have some other agenda.
The number of suspected conspiracies (even on lkml) seems to be on the
rise lately. Maybe it's an aftereffect of september 11th?
Linus isn't accepting my patches, it must be a conspiracy.
CML2 isn't being merged in to the tree, it must be a conspiracy.
Hostile takeover of the IDE subsystem, it must be a conspiracy.
Someone doesn't think BK solves everything, it must be a conspiracy.
Jan
On Wed, Apr 17, 2002 at 12:47:18PM -0400, Jan Harkes wrote:
> Where is that $1/GB drive that can replace the 800MB drive in my Compaq
> 1120 laptop? There isn't any, because Compaq locked the drive geometry
> in the bios to only work with this one particular 800MB drive that isn't
> produced anymore. And that drive, even if I find another one, is
> definitely not going to be 80 cents.
You're right, it's $79.95 for the whole laptop at the buy-it-now price
on Ebay, go get it.
http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=2017735168
> rate for dialup, but what about people that have to pay by the minute
> for their 5KB/s dialup connection. Saving 100MB in data transfer will
> not just save them 5 hours and 41 minutes of download time, but also
> a whole lot in phone charges.
Indeed. So how about you go make a test tree with BK and the same test
tree with CVS, do a set of operations like checkout, update, etc., with
both and report back on the bandwidth stats? What you'll find is that
they are about the same for the initial checkout, BK will be a bit more
actually because it copies the revision files not the checked out files,
and then for ever subsequent operation, BK is less.
Suppose you have the linux-2.5 tree and you want to get the USB tree,
you can _locally_ clone the 2.5 tree to usb and then pull from the USB
tree. Thereby saving lotso bandwidth. One of our developers is behind
a modem and he does this sort of thing all the time.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
Oh my!!!!
To answer the question are we going to drop CVS drop in support of the
linux console project. No!! The reasons have nothing to do with politics
or who is better. The reason for this is some people want to use CVS and
don't want to spend the time learning bitkeeper. That is their choose. I
don't want to force anyone to do something they don't want. Have both
bitkeeper and CVS is a plus in many ways to many people.
P.S
I'm just learning bitkeeper so if it is possible to do a drop in
techique with bitkeeper I would be happy to learn how to do it.
. ---
|o_o |
|:_/ | Give Micro$oft the Bird!!!!
// \ \ Use Linux!!!!
(| | )
/'_ _/`\
___)=(___/
On Wed, Apr 17, 2002 at 08:52:15AM -0500, M. R. Brown wrote:
> > > For the LinuxSH project, we moved from tracking the full kernel tree (via
> > > CVS) to a drop-in tree structure mainly because of the difficulty in
> > > tracking all upstream kernel changes at once. It wasn't really related to
> > > what CVS did or did not offer.
> >
> > Disagree. CVS sucks at tracking an external source of change while
> > maintaining your own changes at the same time, in the same tree.
> > It really sucks at that, you are forced into a broken branching model
> > with awful merge characteristics. Try the same thing with BK, it's
> > a lot nicer, most stuff just automerges, and the stuff that doesn't
> > only needs to be merged once. Your claim that it isn't about CVS
> > doesn't make much sense when CVS forces you to split your subtree
> > out. Having an _option_ to do that is one thing, being forced to
> > do it is another.
>
> I'd hate to sound like I'm trolling or being a broken record. But I don't
> currently have that option with BK. This is the option I want, to be able
> to not have to worry about pulling all changes or being able to cordon off
> only the bit I want/need to work with.
Here's how you do it with BK.
a) pick a baseline that you want. Like 2.4 or 2.5. clone it.
b) put your changes on top of it.
c) for each tree to which you want to apply your changes, clone it,
and then pull from your tree.
Just go try that for a month's worth of changes and then tell me if it
isn't easier for you.
The deal is that you are trying to take your style of working, which as
far as I can tell, is based on CVS being broken, and insisting on that
style. If you really want to work that way, you can, you are just losing
a lot in the process. But whatever. If you want to maintain a drop in
tree from a BK tree, then just check out the part of the tree you want
and plop the files into CVS. Cort exported subsets of the trees as
patches for Linus for 2 years, it works.
> > And if that tree has changes in the files that you are "dropping in"?
> > You just stomped on them. Or you are left with an even more broken
> > merge model than CVS. And what about file renames.
>
> When my drop-in tree file changes upstream, I'll find out in the next
> upstream sync. Obviously if my drop-in tree is current, that file hasn't
> changed yet. File renames are broken, but I never pretended they weren't. I
> guess what you're saying is it comes down to me weighing out feature-rich
> set vs. feature-less set, where BK is the former, and I should expect to have
> to sacrifice my entire development model (drop-in trees, not CVS) for the
> bloated model that BK currently *enforces* (see your above paragraph about
> having the option).
That "bloated model" makes all the hard aspects of your model go away.
So you are trading off your model against your time. That's the whole
point. For the cost of some local storage, you get some of your time
back to do useful work. Read that again. That's the whole point.
An SCM tool is supposed to *save you time*. What you are doing is
replacing missing features by you manually fixing up the places where
your tools didn't work.
You'd have a strong point if the model that BK was using wasted your time,
but you haven't made that case.
> > Part of the problem I have is that
> > you are, in a public forum, saying that BK is this and that, based on your
> > perceptions from 2 years ago when you said you used it on the ppc tree.
>
> Nope, I said that based on what I read at http://www.bitkeeper.com, BK didn't
> support drop-in trees *today*. I said that the last time I actively used
> the tool was 2 years ago, but I haven't had a reason to use it since then.
> In a public forum, which you use to promote sales and use of your product, I
OK, one day this might not be true, but so far, we haven't made a dime
off the people in this forum. The people reading this forum don't spend
money, they write/read code. The fact that this forum has not been
a money make for us has been true for 4 years. I'm not sure how many
years it needs to be true before it might occur to you that the reason
I post here about BK is for some other reason than to make money.
BK is my primary way, these days, of trying to help out with the kernel.
I'd like it to help as many people as possible because I like Linux.
All of these posts are focussed on one thing: making people more productive
while doing development on the kernel. Your posts annoyed me because they
made it sound like BK slows you down. In the case you are talking about,
I think the opposite is true. You might think that too if you actually
tried it.
> When I decide to start and maintain the "linux toaster
> driver", I pull out the source I need from mainline, and setup my drop-in
> tree from that and my new imports. The BK way to do it is to simply "clone
> the master" and work from that. Not a drop-in.
So what? Do a hardlinked clone, it costs you nothing, and do your work
in there. BK will solve all the merge/rename problems that are inevitable
and you can focus on your work.
> Are you
> saying the only way for me to evaluate a tool (one that I haven't used in
> ages) is to use it?
Yes. Just like the only way to know how an application performs on a
platform is to run it. Documentation, peer reviews, the words out of
my mouth, they are all like benchmark results, they are noise compared
to you actually trying to do what you want to do.
> It was a good read, and easy to follow, but it still does the opposite of
> what I need to do. I want multiple small project-specific trees, not a
> myriad of full kernel trees. Everything else gave me a good starting point
> for doing SCM-controlled kernel development using BK. Thanks for the
> pointer.
So use hardlinked clones. See the clone -l, and the relink docs. You'll
burn 87MB for the baseline and then 1240 inodes for the directories in
each hardlinked clone. If inodes take 4K each, that's 5MB, which isn't
free, but that's the best we can. We'll cut that in half when we get
rid of the SCCS directories.
If you wanted to complain that BK doesn't restore the hardlinks after a
file is updated in multiple clones, that's a legit complaint. You can
solve that with a post-incoming trigger which runs relink, but it would
be nicer if there was a way to make BK do that automagically.
> for me it boils down to using the best tool for the job.
Hey, I'm all for that. If Arch worked better for Linus and the
maintainers, I'd be the first to urge them to switch. We aren't getting
financial benefit from the kernel use of BK, which is a bummer, but that
is OK, that wasn't the point. The point was to reduce the workload on
the critical people in the equation and that's what we focussed on.
I'm not at all convinced that you have explored BK enough to know whether
it will help you or not, and I'd appreciate it if you did so rather than
claiming it doesn't work. All you are saying is that it doesn't work
exactly how you work now. That may or may not be the same as saying it
can't help you (or others). You won't know until you go explore.
Both Linus and I were fairly convinced that the BK take-it-all whether
you want it or not model was a showstopper. Then Garzik figured out
a way around that which works well enough for now. A lot of useful
work is happening faster because of what he figured out. My complaint
with you is that you are assuming that you can't make BK work faster
for you without trying it or thinking about it. Does BK mimic your
working model? No. Does that mean it can't help you? Not clear.
That's the point.
I think BK can help you and you are so stuck on one way of doing things
that you are ignoring that possibility. That's OK with me so long as
it is just you, but you're posting in a public forum, to a group of
people that I'm trying to help, and as long as you do, I'll be here,
annoying as heck, explaining why you are wrong. I'm not saying that you
are wrong about BK drop-ins, you're right about that. We're fixing it as
fast as we can, it's in my top 3 work items right now. In the meantime,
if you poked around, there are lots of ways to use BK and I'm 99% sure
you could find one that would result in less work overall for you than
what you are doing now. I don't care if *you* find one, I care if
others find one. We're trying to maximize productive work for the
largest number of people, and I find your posts counterproductive to
that goal.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
* Larry McVoy <[email protected]> on Wed, Apr 17, 2002:
>
> I think BK can help you and you are so stuck on one way of doing things
> that you are ignoring that possibility. That's OK with me so long as
> it is just you, but you're posting in a public forum, to a group of
> people that I'm trying to help, and as long as you do, I'll be here,
> annoying as heck, explaining why you are wrong. I'm not saying that you
> are wrong about BK drop-ins, you're right about that. We're fixing it as
> fast as we can, it's in my top 3 work items right now. In the meantime,
> if you poked around, there are lots of ways to use BK and I'm 99% sure
> you could find one that would result in less work overall for you than
> what you are doing now. I don't care if *you* find one, I care if
> others find one. We're trying to maximize productive work for the
> largest number of people, and I find your posts counterproductive to
> that goal.
The cool thing is, since the linuxconsole drop-in wasn't going anywhere, I
can move on to more useful pursuits instead of going back and forth with you.
If you want to spend your time denegrating other SCMs and people that don't
subscribe to your divine development tenets, more power to you. I've
learned enough from this thread to not waste time arguing with you. Next
time someone says something about Bitkeeper that I happen to agree or
disagree with, do you suppose I should just e-mail them privately? Or
should I post here to make sure it gets under your scrutinizing eye?
Apologies to those who read this God awful thread without hitting Control-D
first :P.
M. R.