LinuxLists.cc - git pull on Linux/ACPI release tree

[permalink] [raw]

Subject: git pull on Linux/ACPI release tree

Hi Linus,

please pull from:

git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git release

This will update the file shown below.

thanks!

-Len

drivers/acpi/processor_idle.c | 20 +++++++++++++++-----
1 files changed, 15 insertions(+), 5 deletions(-)

through these commits:

David Shaohua Li:
[ACPI] correct earlier SMP deep C-states on HT patch

with this log:

commit 927fe18397b3b1194a5b26b1d388d97e391e5fd2
Merge: e4f5c82a92c2a546a16af1614114eec19120e40a
1e483969930a82e16767884449f3a121a817ef00
Author: Len Brown <[email protected]>
Date: Mon Dec 5 17:08:40 2005 -0500

Pull 5165 into release branch

commit 1e483969930a82e16767884449f3a121a817ef00
Author: David Shaohua Li <[email protected]>
Date: Thu Dec 1 17:00:00 2005 -0500

[ACPI] correct earlier SMP deep C-states on HT patch

http://bugzilla.kernel.org/show_bug.cgi?id=5165

Change polarity of test for PLVL2_UP flag.
Skip promotion/demotion code when not needed.

Signed-off-by: Shaohua Li <[email protected]>
Signed-off-by: Venkatesh Pallipadi <[email protected]>
Signed-off-by: Len Brown <[email protected]>

2005-12-23 05:42:53

[permalink] [raw]

Subject: git pull on Linux/ACPI release tree

Hi Linus,

please pull from:

git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git release

This will update the files shown below.

thanks!

-Len

drivers/acpi/processor_thermal.c | 4 ++--
drivers/acpi/utilities/utmisc.c | 20 ++++++++++----------
include/acpi/acglobal.h | 2 +-
3 files changed, 13 insertions(+), 13 deletions(-)

through these commits:

Alex Williamson:
[ACPI] increase owner_id limit to 64 from 32

Len Brown:
[ACPI] fix build warning from owner_id patch

Thomas Renninger:
[ACPI] fix passive cooling regression

with this log:

commit 9d6be4bed65a3bd36ab2de12923bff4f4530bd86
Author: Len Brown <[email protected]>
Date: Thu Dec 22 22:23:06 2005 -0500

[ACPI] fix build warning from owner_id patch

Signed-off-by: Len Brown <[email protected]>

commit 75b245b3259133360845bc6de3aecb8a6bd6ab59
Author: Thomas Renninger <[email protected]>
Date: Wed Dec 21 01:29:00 2005 -0500

[ACPI] fix passive cooling regression

Return logic was inverted.
Going for changing the return value to not return zero as it is makes
more sense regarding the naming of the function (cpu_has_cpufreq()).

http://bugzilla.kernel.org/show_bug.cgi?id=3410

Signed-off-by: Thomas Renninger <[email protected]>
Signed-off-by: Len Brown <[email protected]>

commit 05465fd5622202d65634b3a9a8bcc9cbb384a82a
Author: Alex Williamson <[email protected]>
Date: Thu Dec 8 15:37:00 2005 -0500

[ACPI] increase owner_id limit to 64 from 32

This is an interim patch until changes in an updated
ACPICA core increase the limit to 255.

Signed-off-by: Alex Williamson <[email protected]>
Signed-off-by: Len Brown <[email protected]>

2006-01-07 15:54:42

[permalink] [raw]

Subject: git pull on Linux/ACPI release tree

Hi Linus,

please pull this batch of trivial patches from:

git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git release

This will update the files shown below.

thanks!

-Len

Documentation/DocBook/kernel-api.tmpl | 1
Documentation/kernel-parameters.txt | 4 +++
Documentation/pm.txt | 2 -
MAINTAINERS | 2 -
drivers/acpi/Kconfig | 2 -
drivers/acpi/scan.c | 27 +++++++++++++---------
drivers/acpi/tables.c | 4 +--
7 files changed, 26 insertions(+), 16 deletions(-)

through these commits:

Borislav Petkov:
[ACPI] remove Kconfig "default y" for laptop drivers

Len Brown:
[ACPI] document processor.nocst parameter
[ACPI] [email protected] replaces [email protected]
[ACPI] reduce kernel size: move 5BK .bss to 2.5KB .init.data
[ACPI] [email protected] replaces [email protected]

Randy Dunlap:
[ACPI] fix kernel-doc warnings in acpi/scan.c

with this log:

commit f9a204e1de73a7007de66fb289e1d64a7665c9b4
Author: Borislav Petkov <[email protected]>
Date: Fri Jan 6 01:31:00 2006 -0500

[ACPI] remove Kconfig "default y" for laptop drivers

Signed-off-by: Borislav Petkov <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Len Brown <[email protected]>

commit d758a8fa8ce0566947677f7e70bf70c57ad9445c
Author: Randy Dunlap <[email protected]>
Date: Fri Jan 6 01:31:00 2006 -0500

[ACPI] fix kernel-doc warnings in acpi/scan.c

Signed-off-by: Randy Dunlap <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Len Brown <[email protected]>

commit 036d25f79ddfbc9878da24ef8e468a6d22caa605
Author: Len Brown <[email protected]>
Date: Fri Jan 6 16:19:26 2006 -0500

[ACPI] [email protected] replaces [email protected]

Signed-off-by: Len Brown <[email protected]>

commit 04348e69e7e78ad69a09b2e1157f628d6c764370
Author: Len Brown <[email protected]>
Date: Fri Dec 30 02:44:59 2005 -0500

[ACPI] reduce kernel size: move 5BK .bss to 2.5KB .init.data

put __initdata on sdt_entry[], as it is accessed only by __init functions.

http://bugzilla.kernel.org/show_bug.cgi?id=1311

Signed-off-by: Len Brown <[email protected]>

commit 6968e50c2b53805219618203db3d98566dfa1fdd
Author: Len Brown <[email protected]>
Date: Fri Dec 30 00:32:49 2005 -0500

[ACPI] [email protected] replaces [email protected]

Signed-off-by: Len Brown <[email protected]>

commit 41c0d8680f7f24e1abaaeeabc6723277b2e34b81
Author: Len Brown <[email protected]>
Date: Wed Dec 28 12:43:51 2005 -0500

[ACPI] document processor.nocst parameter

Signed-off-by: Len Brown <[email protected]>

2006-01-07 18:22:45

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Sat, 7 Jan 2006, Len Brown wrote:
>
> please pull this batch of trivial patches from:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git release

Len,

I _really_ wish you wouldn't have those automatic merges.

Why do you do them? They add nothing but ugly and unnecessary history, and
in this pull, I think almost exactly half of the commits were just these
empty merges.

There's just no point, except to make the history harder to read.

So please stop it. You have some of the ugliest history around, and it's
all just because you have some automated process that merges unnecessarily
all the time.

If you do merges because you want to _test_ the development with a merged
tree, that doesn't have to happen in the development branch itself. Nobody
else cares about such a merge except the tester (unless the test fails of
course, and you need to fix up the result - at which point it's not an
automated merge any more).

Linus

2006-01-08 07:47:42

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

Hi Linus,

adding [email protected]...

>> please pull this batch of trivial patches from:
>>
>>
>git://git.kernel.org/pub/scm/linux/kernel/git/lenb/linux-acpi-2.6.git release
>
>Len,
>
>I _really_ wish you wouldn't have those automatic merges.
>
>Why do you do them? They add nothing but ugly and unnecessary
>history, and in this pull, I think almost exactly half of the
>commits were just these empty merges.

Is it possible for it git, like bk, to simply ignore merge commits in its summary output?

Note that "Auto-update from upstream" is just the place-holder comment
embedded in the wrapper script in git/Documentation/howto/using-topic-branches.txt
All instances of it here are from me manually updating --
the only "auto" happening here is the automatic insertion of that comment:-)

I think that Tony's howto above captures two key requirements
from all kernel maintainers -- which the exception of you --
who hang out in the middle of the process rather than
at the top of the tree.

1. It is important that we be able (and encouraged, not discouraged)
to track the top of tree as closely as we have time to handle.
Divergence and conflicts are best handled as soon as they are noticed
and can be a huge pain if left to fester and discovered
only when it is time to push patches upstream.
Plus, tracking the top of tree means we force more folks to
track the top of tree, and so it gets more testing. This is goodness.

Earlier in your release cycle when changes are appearing faster,
my need/desire to sync is greater than later in the cycle when changes
are smaller and infrequent. On average, I think that one sync/day
from upstream is an entirely reasonable frequency.

2. It is also important that we be able to cherry pick individual patches
in our trees so that they don't block each other from going upstream.
Tony's using-topic-branches.txt above is the best way I know of doing that.
I think it is a big improvement over the bk model since I can have a simple
branch for each patch or group of patches rather than an entire repository
dedicatd to each. But for this to work, I need to be able to update
any and all of the topic branches from upstream, and to merge them with
each other -- just like I could with BK. Otherwise they become "dated"
in the time they were first integrated, and it is not convenient to do
simple apples/apples comparisons that are needed to debug and test.

I'm probably a na?ve git user -- but I expect I have a lot of company.
If there is a better way of using the tool to get the job done,
I'm certainly a willing customer with open ears.

thanks,
-Len

2006-01-08 08:17:00

by David Miller

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

From: "Brown, Len" <[email protected]>
Date: Sun, 8 Jan 2006 02:47:30 -0500

> I'm probably a na?ve git user -- but I expect I have a lot of company.
> If there is a better way of using the tool to get the job done,
> I'm certainly a willing customer with open ears.

What I do is simply build a new fresh tree if I feel the urge
to sync with the top of Linus's tree. I use the script below
which I call "git suck". It just sucks the patches out of
one tree and sticks them into another tree. You go:

bash$ cd new-2.6
bash$ git suck ../foo-2.6

It preserves everything except the dates, and it's so incredibly
cheap and fast with GIT.

I know a lot of people react to this kind of usage with "what's the
point of the source control system if you're just messing with patches
in and out of the tree all the time" But as a subsystem maintainer,
you deal with a lot of changes and it's important to get a pristine
clean history when you push things to Linus.

In fact, I do this so much that Linus's tree HEAD often equals my
origin when he pulls.

Merges really suck and I also hate it when the tree gets cluttered
up with them, and Linus is right, ACPI is the worst offender here.

Yes, we can grep the merges out of the shortlog or whatever, but that
merging crap is still physically in the tree.

Just don't do it. Merge into a private branch for testing if you
don't want to rebuild trees like I do, but push the clean tree to
Linus.

#!/bin/sh
#
# Usage: git suck path-to-tree
#
# Pull all patches relative to 'origin' from the tree specified
# and apply them to the current directory tree, keeping all changelog
# and authorship information identical. It will update the dates
# of the changes of course.
(cd $1; git format-patch --mbox origin) || exit 1
for i in $1/*.txt
do
sed 's/\[PATCH\] //' <$i >tmp.patch
git-applymbox -k tmp.patch || exit 1
done

2006-01-08 18:29:44

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

>I know a lot of people react to this kind of usage with "what's the
>point of the source control system if you're just messing with patches
>in and out of the tree all the time" But as a subsystem maintainer,
>you deal with a lot of changes and it's important to get a pristine
>clean history when you push things to Linus.
>
>In fact, I do this so much that Linus's tree HEAD often equals my
>origin when he pulls.
>
>Merges really suck and I also hate it when the tree gets cluttered
>up with them, and Linus is right, ACPI is the worst offender here.
>
>Yes, we can grep the merges out of the shortlog or whatever, but that
>merging crap is still physically in the tree.
>
>Just don't do it. Merge into a private branch for testing if you
>don't want to rebuild trees like I do, but push the clean tree to
>Linus.

Perhaps the tools should try to support what "a lot of people"
expect, rather than making "a lot of people" do extra work
because of the tools?

Call me old fashioned, but I believe that tools are supposed to
make work easier, not harder.

-Len

2006-01-08 19:10:25

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

On Sun, 8 Jan 2006, Brown, Len wrote:
>
> Is it possible for it git, like bk, to simply ignore merge commits in its summary output?

That's not the point. It does: "git log --no-merges" does exactly that.

But fire up "gitk" to watch the history, and see the difference.

> Note that "Auto-update from upstream" is just the place-holder comment
> embedded in the wrapper script in git/Documentation/howto/using-topic-branches.txt

That has absolutely nothing to do with anything. It's not the comment
(which admittedly gives absolutely no information - but why should it,
since the _commit_ itself has no information in it?)

It's like you have empty commits that don't do anything at all, except
that they are worse, because they have two parents.

> I think that Tony's howto above captures two key requirements
> from all kernel maintainers -- which the exception of you --

No. Your commits make it harder for _everybody_ to track the history.

A merge by definition "couples" the history of two branches. That's what a
merge very fundamentally is. It ties two things together. But two things
that don't have any connection to each other _shouldn't_ be tied together.

Just as an example: because of the extra merges, you've made all your
commits dependent on what happened in my tree, with no real reason. So
let's say that somebody reports that something broke in ACPI. Now you
can't just go to the top of the ACPI history and work backwards - you'll
have tied up the two histories so that they are intertwined.

And yes, you can always work around it, but there's just no point. And
none of the other developers seem to need to do it. They do their
development, and then they say "please pull". At that point the two
histories are tied together, but now they are tied together for a
_reason_. It was an intentional synchronization point.

An "automated pull" by definition has no reason. If it works automated,
then the merge has zero semantic meaning.

Linus

2006-01-08 19:19:52

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On 1/9/06, Brown, Len <[email protected]> wrote:
> Perhaps the tools should try to support what "a lot of people"
> expect, rather than making "a lot of people" do extra work
> because of the tools?

I think it does. All the tricky stuff that David and Junio have been
discussing is actually done very transparently by

git-rebase <upstream>

Now, git-rebase uses git-format-patch <options> | git-am <options> so
it sometimes has problems merging. In that case, you can choose to
either resolve the problem (see the doco for how to signal to git-am
that you've resolved a conflict) or to cancel the rebase. If you
choose to cancel the rebase, do

cp .git/refs/heads/{<headname>,<headnamebadrebase>}
cat .git/HEAD_ORIG > .git/refs/heads/<headname>
git-reset --hard
rm -fr .dotest

and you'll be back to where you started. Perhaps this could be rolled
into something like git-rebase --cancel to make it easier, but that's
about it. The toolchain definitely supports it.

cheers,

martin

2006-01-08 19:33:54

by Junio C Hamano

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

Martin Langhoff <[email protected]> writes:

> On 1/9/06, Brown, Len <[email protected]> wrote:
>> Perhaps the tools should try to support what "a lot of people"
>> expect, rather than making "a lot of people" do extra work
>> because of the tools?
>
> I think it does. All the tricky stuff that David and Junio have been
> discussing is actually done very transparently by
>
> git-rebase <upstream>
>
> Now, git-rebase uses git-format-patch <options> | git-am <options> so
> it sometimes has problems merging.

Careful. I do not think rebase works across merges at all.

2006-01-08 19:42:03

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

On Sun, 8 Jan 2006, Brown, Len wrote:
>
> Perhaps the tools should try to support what "a lot of people"
> expect, rather than making "a lot of people" do extra work
> because of the tools?
>
> Call me old fashioned, but I believe that tools are supposed to
> make work easier, not harder.

They DO.

Len, you're doing EXTRA WORK that is pointless.

Just stop doing the automated merges. Problems solved. It really is that
easy. Don't do what David suggests - he does it because he's apparently
_so_ comfortable with things that he prefers to do extra work just to keep
his trees extra clean (I actually would disagree - but git makes that
fairly easy to do, so if you prefer to have as linear a history as
possible, you can do it with git pretty easily).

Now, I'm only complaining about _automated_ merges. If you have a reason
to worry about my tree having clashes with your tree, do a real merge. For
example, in your latest pull, you had a

"pull linus into release branch"

merge, where you merged my v2.6.15 tree. That makes perfect sense.

What I object to is that there were _also_ two automated merges within ten
hours or each other, with absolutely _zero_ development in your tree in
between. Why did you do that in your development tree? By _definition_ you
had done zero development. You just tracked the development in _my_ tree.

In case you wonder, the two commits I'm talking about are:

add5b5ee992e40c9cd8697ea94c223628be162a7
25da0974601fc8096461f3d3f7ca3aab8e79adfb

and neither of them have any reason to be in a development tree. You
didn't develop them.

They are real merges, because you had a trivial patch in your tree
(changing the acpi-devel mailing list address) that I didn't have, so when
you pulled, your end result was thus always different from something I had
(so you did a real "merge", even though it was totally trivial), but the
point is that there is a difference between "the ACPI development tree"
and "the tree that has random ACPI patches and then tracks Linus' tree as
closely as possible".

See?

That's the most egregious example. There's two unnecessary pulls on
December 28 and 29th too (commits 0a5296dc and c1a959d8).

You can do

gitk 0aec63e..f9a204e1

to see exactly what I see when I pulled from you. 11 commits, 5 of which
are just trivial merges that are no development, just tracking _my_ tree.
Of those, one makes sense (tracking a release).

(NOTE NOTE NOTE! It does make sense to track my tree in case you do big
changes and you worry about clashes. Then you would want to synchronize
those big changes with my changes, so that you can resolve any clashes
early. So I'm not saying that tracking trees is always bad: I'm saying
that doing so _unnecessarily_ is bad, because it adds no value, and it
just makes the history harder to read).

Now, most people don't read the history. It gets messy enough quickly
enough that it's hard to read anyway over time. My tree has tons of _real_
merges anyway, since it's by definition the one that is used for most
synchronization, so my tree is always pretty hard to follow.

But my guess is that this probably makes it harder for _you_ to see what
you've done too. If you didn't merge with me, then "git log" would show
just your own changes at the top, and that's likely what you care most
about anyway, no?

Also, if you didn't pull from me, and you decided that you needed to re-do
your tree (let's say that you notice that one of your commits was bad
_before_ you ask me to pull from your tree), then you'd also have an
easier time re-creating your own development without that buggy change,
exactly because _your_ tree wouldn't have my changed mixed up in it.

So your merges likely make git harder to use for you, not easier.

Linus

2006-01-08 19:57:30

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Mon, 9 Jan 2006, Martin Langhoff wrote:
>
> I think it does. All the tricky stuff that David and Junio have been
> discussing is actually done very transparently by
>
> git-rebase <upstream>

Yes, it's fairly easy to do. That said, I would actually discourage it. I
haven't said anything to David, because he is obviously very comfy with
the git usage, and it _does_ result in cleaner trees, so especially since
the networking code ends up being the source of a lot of changes, the
extra cleanup stage that David does might actually be worth it for that
case.

But git is actually designed to have parallel development, and what David
does is to basically artificially linearize it. We merge between us often
enough that it doesn't really end up losing any historical information
(since David can't linearize the stuff that we already merged), but in
_theory_ what David does actually does remove the historical context.

So "git-rebase" is a tool that is designed to allow maintainers to (as the
command says) rebase their own development and re-linearize it, so that
they don't see the real history. It's basically the reverse of what Len is
doing - Len mixes up his history with other peoples history in order to
keep them in sync, while David bassically "re-does" his history to be on
top of mine (to keep it _separate_).

The "git-rebase" means that David will always see the development he has
done/merged as being "on top" of whatever my most recent tree is. It's
actually a bit scary, because if something goes wrong when David re-bases
things, he'll have to clean things up by hand, and git won't help him
much, but hey, it works for him because (a) things seldom go wrong and (b)
he appears so comfortable with the tool that he _can_ fix things up when
they do go wrong.

And yes, git-rebase can be very convenient. It has some problems too
(which is the other reason I don't try to convince other maintainers to
use it): because it re-writes history, a change that _might_ have worked
in its original place in history might no longer work after a rebase if it
depended on something subtle that used to be true but no longer is in the
new place that it has been rebased to.

Which just means that a commit that was tested and found to be working
might suddenly not work any more, which can be very surprising ("But I
didn't change anything!").

On the other hand, this is no different from doing a merge of two
independent streams of development, and getting a new bug that didn't
exist in either of the two, just because they changed the assumptions of
each other (ie not a _mismerge_, but simply two developers changing
something that the other depended on it, and the bug only appears when
both the working trees are merged and the end result no longer works).

So my suggested git usage is to _not_ play games. Neither do too-frequent
merges _nor_ play games with git-rebase.

That said, git-rebase (and associated tools like "git-cherry-pick" etc)
can be a very powerful tool, especially if you've screwed something up,
and want to clean things up. Re-doing history because you realized that a
you did something stupid that you don't want to admit to anybody else.

So trying out git-rebase and git-cherry-pick just in case you decide to
want to use them might be worthwhile. Making it part of your daily routine
like David has done? Somewhat questionable, but hey, it seems to be
working for David, and it does make some things much easier, so..

Linus

2006-01-08 19:57:46

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Sun, 8 Jan 2006, Junio C Hamano wrote:
>
> Careful. I do not think rebase works across merges at all.

Right. You have to do one or the other (rebase your changes to another
tree _or_ merge another tree into your changes), but not mix the two.

Linus

2006-01-08 20:35:50

by David Miller

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

From: Linus Torvalds <[email protected]>
Date: Sun, 8 Jan 2006 11:56:21 -0800 (PST)

> So my suggested git usage is to _not_ play games. Neither do too-frequent
> merges _nor_ play games with git-rebase.
>
> That said, git-rebase (and associated tools like "git-cherry-pick" etc)
> can be a very powerful tool, especially if you've screwed something up,
> and want to clean things up. Re-doing history because you realized that a
> you did something stupid that you don't want to admit to anybody else.
>
> So trying out git-rebase and git-cherry-pick just in case you decide to
> want to use them might be worthwhile. Making it part of your daily routine
> like David has done? Somewhat questionable, but hey, it seems to be
> working for David, and it does make some things much easier, so..

The time at which I do the by-hand rebasing the most are the weeks
leading up to a major release. The reason is to integrate bug fixes
that I know conflict with the 80-odd patches I have queued up for the
next development phase, or that I simply want integrated so that no
_future_ development patches create conflicts.

I think merges with conflicts that need to get resolved by hand create
a lot of noise and useless information and therefore to me they are
pointless. But this is just my opinion. It simply works easier to me
to shuffle the patches in by hand and deal with the rejects one by
one. It's very much akin to how Andrew's -mm tree works.

I think a clean history is worth an extra few minutes of someone's
time. And note that subsystem development is largely linear anyways.

2006-01-08 20:50:29

by Luck, Tony

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

I'll try to update the using-topic-branches document to capture this.
Some of the problem is that it doesn't quite capture what I'm doing
with my test/release branches.

My release branch really is just used as a transfer point to Linus.
I usually[1] don't leave patches sitting in "release" for long enough
that I'll be tempted to merge in from Linus ... once I decide that
some patches are ready to go to Linus I'll update "release" from Linus
(which will be a fast-forward, so no history) merge in the topic
branches, do one final sanity build, push to kernel.org and send
the "please pull" e-mail.

The huge majority of my "automatic update from upstream" merges
go into my test branch ... which never becomes part of the real
history as I never ask Linus to pull from it.

-Tony

[1] Sometimes I goof on this because I forget that I've applied
a trivial patch directly to the release branch without going through
a topic branch. I think I'll fix my update script to check for this case.

2006-01-08 21:21:03

by Luben Tuikov

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

--- Linus Torvalds <[email protected]> wrote:
> So trying out git-rebase and git-cherry-pick just in case you decide to
> want to use them might be worthwhile. Making it part of your daily routine
> like David has done? Somewhat questionable, but hey, it seems to be
> working for David, and it does make some things much easier, so..

How about this usage (branch == tree):

Tree A (your tree)
Tree B (project B, dependent on Tree A)
Tree C (project C, dependent on project B)

(i.e. diff(C-A) = diff(C-B) + diff(B-A))

Your tree is pulled into Tree A as often as your tree
changes and it just fast forwards.

If I want to run project B with your latest tree, then
I resolve/merge from tree A to tree B, compile B
and run it.

If I want to run project C and project B with your
latest tree, I resolve/merge from tree A to tree B
and from tree B to tree C, compile C and run it.

In such cases, are you saying that you'd prefer to
pull from Tree B and Tree C (depending on your needs)?

Another question:
Sometimes, a fix for project B finds its way into
tree C (project C) (since C depended on that fix in B).
Now I'd like to pull that particular fix, identified by
its SHA, into project B, and nothing else, for this I can
use git-cherry-pick, right?

And lastly, is there a tool whereby I can "see" changes
between repos, kind of like git-diff but being able to
give URLs too?

Luben

2006-01-08 23:06:13

by Adrian Bunk

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Sun, Jan 08, 2006 at 11:41:28AM -0800, Linus Torvalds wrote:
>...
> What I object to is that there were _also_ two automated merges within ten
> hours or each other, with absolutely _zero_ development in your tree in
> between. Why did you do that in your development tree? By _definition_ you
> had done zero development. You just tracked the development in _my_ tree.
>...

My impression is that you and Len are talking at different levels.

I can't speak for Len, but let me try to describe a problem in this area
I don't know the solution for:

Consider I want to do the following:
1. update my tree daily from your tree
2. include 10 patches per week into my tree
3. ask you once a month to pull from my tree

How should step 1 be done?

In CVS, I'd do a "cvs update -dP ."
In cogito, the equivalent command seems to be "cg-update".

CVS has no problems if I have changed MAINTAINERS in one place and it
changes daily in your tree in other places, but how do I do the same in
git/cogito without creating the merges you don't want to see?

The solution might be described somewhere in TFM, but this is the class
of problems people like me run into when the goal is simply a git tree
to both track your tree and send changes to you without any interest in
advanced SCM knowledge.

> Linus

cu
Adrian

--

"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed

2006-01-08 23:58:28

by Willy Tarreau

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

Hi Adrian,

On Mon, Jan 09, 2006 at 12:06:11AM +0100, Adrian Bunk wrote:
> On Sun, Jan 08, 2006 at 11:41:28AM -0800, Linus Torvalds wrote:
> >...
> > What I object to is that there were _also_ two automated merges within ten
> > hours or each other, with absolutely _zero_ development in your tree in
> > between. Why did you do that in your development tree? By _definition_ you
> > had done zero development. You just tracked the development in _my_ tree.
> >...
>
> My impression is that you and Len are talking at different levels.
>
> I can't speak for Len, but let me try to describe a problem in this area
> I don't know the solution for:
>
> Consider I want to do the following:
> 1. update my tree daily from your tree
> 2. include 10 patches per week into my tree
> 3. ask you once a month to pull from my tree
>
> How should step 1 be done?

I believe we all have the same problem. The only solution I found for
this was to proceed like David described. I know even have a 'git-patches'
directory next to my git repo to keep the resulting patches after I do a
'git-format-patch --mbox'.

When Linus called it the 'stupid content tracker', he was half right.
In fact, it's more a 'changes tracker' than a 'content tracker'. It
logs everything you and others do, so if you don't want your operations
to appear on others' history, you have to hide them by working on
temporary trees to generate the patches you will use later.

At first I found this very annoying, but finally, it's a way to ensure
that I always have clean and ordered patches. It's not much different
from what I was doing by hand previously, it's just that all git-xxx
operations take much longer time, possibly because of the compression.
On the other hand, its ability to understand mbox saves me some time.

> In CVS, I'd do a "cvs update -dP ."
> In cogito, the equivalent command seems to be "cg-update".
>
> CVS has no problems if I have changed MAINTAINERS in one place and it
> changes daily in your tree in other places, but how do I do the same in
> git/cogito without creating the merges you don't want to see?
>
> The solution might be described somewhere in TFM, but this is the class
> of problems people like me run into when the goal is simply a git tree
> to both track your tree and send changes to you without any interest in
> advanced SCM knowledge.

What sometimes worries me is that some operations seem so much complicated
that there is a high risk of doing the wrong thing, and I'm not certain
that this will reduce the number of mistakes I do in a month, compared
to manual patching. I became a real addict to 'git-reset --hard' ...

> cu
> Adrian

Regards,
Willy

2006-01-09 00:49:12

by Al Viro

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Sun, Jan 08, 2006 at 11:10:20AM -0800, Linus Torvalds wrote:
> That has absolutely nothing to do with anything. It's not the comment
> (which admittedly gives absolutely no information - but why should it,
> since the _commit_ itself has no information in it?)

How do you deal with conflict resolution? That's a genuine question -
I'm not talking about deliberate misuse to hide an attack, just a normal
situation when you have to resolve something like

A:
if (foo)
bar

B:
if (foo & baz)
bar

A':
#ifdef X
if (foo)
bar
...
#endif

merge of A' and B: trivial conflict

and have git pull fail. The obvious way (edit file in question, update-index,
commit) will not only leave zero information about said conflict and actions
needed to deal with it, but will lead to situation when git whatchanged will
not show anything useful. I.e. if conflict turns out to be non-trivial and
ends up being resolved wrong, everyone will have very nasty time trying to
figure out where the breakage had come from when looking at history 6 months
down the road.

Is there any SOP I'm missing here?

Worse (for my use), format-patch on such tree will not give a useful patchset.
You get a series of patches that won't apply to _any_ tree. Even if all
conflicts had been resolved correctly, they still remain there for everyone
trying to apply the patch series, unless you manually rebase it before
format-patch.

And that's a fundamental problem behind all that rebase activity, AFAICS.
It definitely is in my case, and yes, it's fscking inconvenient in a lot
of respects. E.g. I'm using git for resync between build trees on several
boxen. There's a repository holding patchset, plus one clone per build
box. Fixes for build breakage, etc., get done in those clones; after they
are committed there, I pull into master and then pull from other clones
to spread them to other build trees. Works fine, but... Any rebase in
master => instant hell for all clones. I've ended up with the following
layout that kinda-sorta avoids mess:
master:origin: matches upstream
master:topic branch: _not_ rebased until there is a conflict, never get
a pull from anywhere
master:master: gets pulls from topic branches and origin _and_ _nothing_ _else_
master:work: where interaction with build boxen and any edits done in master
repository go. Edits, commits, pulls from master:master, pulls from
build boxen.
buildN:origin == master:work
buildN:work: where work on buildN goes.
When I want to get new stuff (== difference between master:master and
master:work) into the patchset, I cherry-pick from work to topic branches
and re-pull them into master:master until it matches master:work. Then
I pull master:master into master:work to create a point in work history
that marks beginning of new portion of pending stuff. New stuff upstream
is pulled to master:origin -> master:master -> master:work -> build trees.

That works, and gives me merge-free topic branches I can safely format-patch
while keeping master in sync with mainline _and_ also safe for format-patch.
The price is in rather convoluted SOP. And the following piece of fun:
when cherry-pick work->topic, pull topic->master or pull origin->master
gives a conflict, it's time to rebase. Which I do by renaming topic branches
(direct mv in .git/refs/heads), then starting new ones at current origin
and applying old ones to them (cherry + cherry-pick if possible, format-patch
+ applymbox if things get hairy). Then master is recreated as branch from
origin that gets pulls from topic branches, work is branched from it and
build trees get killed and cloned from scratch. It's tolerable since I'm
using ccache on build boxen, so it's _not_ that much of rebuild.

However, that clearly is a killer if any poor sucker (me included) ever
clones from master for any other purpose. And that, BTW, is the main
reason that stops me from moving master to kernel.org right now.

> And yes, you can always work around it, but there's just no point. And
> none of the other developers seem to need to do it. They do their
> development, and then they say "please pull". At that point the two
> histories are tied together, but now they are tied together for a
> _reason_. It was an intentional synchronization point.
>
> An "automated pull" by definition has no reason. If it works automated,
> then the merge has zero semantic meaning.

I'm afraid you are missing a part of picture. There is a bunch of git
uses that handle a heap of foam rather than a long-term branching. I.e.
the tree is tied to mainline closely and most of the stuff in it is
supposed to get flushed into mainline soon after it appears. I.e. the
situation when we have a mergepoint for fixes that _has_ to follow
mainline closely.

I wonder what life would be without merge nodes and with equality nodes
instead. I.e. to merge
O -> A1 -> ..... -> An (=A)
-> B1 -> ..... -> Bm (=B)
would be to create a new branch (C) at Bm, have entire A1...An replayed there,
have B1...Bm replayed in A and then create a node certifying that new head
of A and head of C refer to the same tree. Plus have a way to see which
commits are claimed to be replays of each other. At least that way rebase
would be simply saying that old history is superceded by new one, with
equality node proving that it's OK to do. We would have
O -> M1 -> .... ->Mn for mainline
O -> B1 -> .... -> B for branch post-pull
Mn -> P1 -> ... -> P for merge branch
and B == P as equality node. Old branch would have a bunch of changesets
of its own plus ones from mainline that got there by pulls (including the
last one). And new branch would contain the ports of not-yet-merged ones
to new mainline head, with the same tree as the result and all further
development going on there rather than in the old branch. Oh, well...

2006-01-09 01:14:35

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Sun, 8 Jan 2006, Luben Tuikov wrote:
>
> How about this usage (branch == tree):
>
> Tree A (your tree)
> Tree B (project B, dependent on Tree A)
> Tree C (project C, dependent on project B)
>
> (i.e. diff(C-A) = diff(C-B) + diff(B-A))
>
> Your tree is pulled into Tree A as often as your tree
> changes and it just fast forwards.
>
> If I want to run project B with your latest tree, then
> I resolve/merge from tree A to tree B, compile B
> and run it.
>
> If I want to run project C and project B with your
> latest tree, I resolve/merge from tree A to tree B
> and from tree B to tree C, compile C and run it.

No.

If tree B is based on _some_point_in_ A, then you just test that.

Because development line B is _independent_ of development line A. The
fact that A changes doesn't change B - unless they have some real
dependencies (which we should try to avoid).

So when you update ("fetch" in git parlance) branch A from me, that
shouldn't affect branch B _nor_ branch C in any way. They clearly do not
depend on the new stuff in A, since they do their own independent
development. The fact that they _started_ at some random point during the
development of A doesn't change that fact.

Now, if you want to _test_ the combined "new stuff in branch A and new
stuff in branch B", feel free to do that. But realize that that is _not_
appropriate in either branch A _nor_ branch B.

So you'd be much better off with a separate "test" branch that you test
stuff out in, and you then resolve ("pull" in git parlance) both branch A
and branch B into that test branch.

See? Testing the combination of two branches doesn't actually have
anything to do with either branch.

At some point, you decide that you want to merge what you've done in
branch B. That's a _different_ and independent thing from deciding that
you want to test the combination of two development branches. Clearly,
it's great to test often, but that has nothing to do with releasing a
branch.

> In such cases, are you saying that you'd prefer to
> pull from Tree B and Tree C (depending on your needs)?

I'm saying that mixing up the "let's test the combination" and "let's
merge the two branches" are totally different things and should not be
mixed up.

One is a random event (and then it makes sense to have, for example, a
"automated test branch" that automatically merges every day and tests the
results. I don't think you should expose those random merges to others,
because they actually hinder the readability of the history for _both_
sides.

The other is a _directed_ event. It's the event of saying "branch B" is
now ready to be merged. Usually that's best done by just saying "please
pull now" - ie not by merging branch A into branch B (because that's not
what you actually want, is it? What you want is for the development in
branch B to show up in branch A - so you want branch A to do the pull).

Now, there's a third kind of event, which is again independent of the
other two. It's more of a "let's try to keep the 'topic branch'
development up-to-date with the branch we eventually want to merge the
topic changes into". That's where you can now do two things:

- David often "rebases" all of the changes in his "topic branch" (ie
conceptually "branch B") to the new top-of-head of "branch A". In other
words, he re-writes branch B entirely _as_if_ it was based on the newer
state "branch A". This is what "git rebase" is all about.

- You can just pull from branch A into branch B, as a way to keep branch
B more up-to-date with the work in the "main trunk" or whatever. This
is ok, but it shouldn't be a common event. It should be something that
happens when you (for example) notice during testing that the test
merge no longer works cleanly. Or it might be "It's now been two weeks
since I synchronized, let's just synchronize to be safe".

See? I'm not objecting to topic branches pulling from my tree in general.
It's just that they should have a _reason_. There's never any reason to
pull into a development tree that you haven't done any development in,
just because you also want to use that development tree for testing.

> Another question:
> Sometimes, a fix for project B finds its way into
> tree C (project C) (since C depended on that fix in B).
> Now I'd like to pull that particular fix, identified by
> its SHA, into project B, and nothing else, for this I can
> use git-cherry-pick, right?

That's one way. It's often the best way, especially if it's a really
obvious bugfix. Or you could just fix it in your tree yourself. It will
mean that the two branches have the same fix, but especially if it really
is an identical fix, it won't be a merge problem.

You _can_ just decide to pull branch B into branch C, but that has a real
problem, namely that it inexorably links the two together, so that nobody
can then pull branch C without pulling indirectly branch B at the time
that B->C merge happened. Sometimes that is ok. But it's nice to avoid it
if you can.

But for example, if somebody fixed something in the trunk, and you
actually do need that fix from the trunk for your topic branch
development, then just doing a pull is _fine_. Now we're back to doing a
merge that actually has a perfectly good reason.

IOW, don't cherry-pick to avoid merges when the merge really does make
tons of sense. Merges are good, it's just that _too_ much of a good thing
is bad.

> And lastly, is there a tool whereby I can "see" changes
> between repos, kind of like git-diff but being able to
> give URLs too?

No, all the good tools really are based on fetching (NOT "pulling") the
other branch into your local tree as a separate branch. At that point,
there are tons of wonderful tools you can use.

In other words, say that you want to know what has happened in another
repository, at git://git.kernel.org/xyzzy. You aren't interested in the
stuff that is already part of the trunk, you're just interested in what is
only in that "xyzzy" branch, and how it relates to your code.

What you'd do is

git fetch git://git.kernel.org/xyzzy master:xyzzy-snapshot

which says "fetch the 'master' branch from that xyzzy repository, and call
it 'xyzzy-snapshot' locally.

You can then (for example) fetch the code that is in _my_ tree by doign
the same time (just call that branch 'linus'), and you can now do

gitk linus..xyzzy-snapshot HEAD

which looks strange (you give "gitk" _both_ a range from the "linus"
branch to the "xyzzy-snapshot" _and_ your own HEAD at this time), but what
it basically does is that the "linus.." syntax tells git that you're not
interested in anything that is already in the 'linus' branch.

So the above command line will actually graphically show _both_ your
current HEAD branch _and_ the 'xyzzy-snapshot' branch, in parallel. You
can see how (if at all) they are related to each other, ignoring all the
commits that have already made it into my tree.

(You can also do "linus..HEAD" instead of just HEAD and effectively repeat
the "don't show 'linus' branch any more" twice. It's perfectly equivalent,
of course. You may also want to use the "-d" flag to "gitk" which tells it
to show things in date order, instead of a simplified history order).

Or just do "what has xyzzy-snapshot that I do not have in my HEAD":

git log HEAD..xyzzy-snapshot

(or gitk), or the other way around: what do _I_ have in my HEAD that
hasn't been pushed to xyzzy-snapshot yet:

git log xyzzy-snapshot..HEAD

(or do diffs, "git whatchanged -p", or whatever).

In other words, using a few different branches (you can make them up
dynamically) can be very powerful.

Linus

2006-01-09 03:30:31

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Mon, 9 Jan 2006, Adrian Bunk wrote:
>
> Consider I want to do the following:
> 1. update my tree daily from your tree
> 2. include 10 patches per week into my tree
> 3. ask you once a month to pull from my tree
>
> How should step 1 be done?

I'd do

git fetch linus

to fetch my tree as a branch (obviously, this assumes you've set up a
'.git/remotes/linus' file for the shorthand).

Then, keep your checked-out working-tree that you also do you development
in your 'devel' branch (or whatever).

And then do

git-rebase linus

to rebase your development branch to mine.

THIS is what "rebase" is for. It sounds like what you really want to do is
not have a development branch at all, but you just want to track my tree
and then keep track of a few branches of your own. In other words, you
don't really have a "real" branch - you've got an odd collection of
patches that you really want to carry around on top of _my_ branch. No?

Now, in this model, you're not really using git as a distributed system.
In this model, you're using git to track somebody elses tree, and track a
few patches on top of it, and then "git rebase" is a way to move the base
that you're tracking your patches against forwards..

It's also entirely possible that you may want to look at "stacked git"
(stg), which is really more about a "quilt on top of git" approach. Which
again, may or may not suit your needs better.

Now, the other alternative is to use git as a "real" distributed system,
and then you might keep my "linus" branch around perhaps as a reference
point, but you don't care too much about it. What you care a lot more
about is your "real development" branch, and you simply don't rebase that,
or try to track my branch all that closely. You work in your real
development branch, and that's your bread and butter. You _don't_ merge my
tree every day, because you simply don't care - that's a separate issue.

You might have a totally different directory that you use _just_ to track
my tree, and that is my virgin tree checked out and ready to go to test
what _I_ am doing - totally independently of your tree.

See? Two totally different usage schenarios. In one, you keep a couple of
"odd-ball" patches around (hey, it could be many, but I say a couple just
because the patches aren't a huge deal - they are kind of a small side
project to you, and not the main focus). And you use "git rebase" to move
those patches forward to match the "real" tree, aka mine.

In the other schenario, your development tree is a full-fledged real
branch, with a life of its own, and _not_ slaved to what happens to be
going on in my tree. You may care about my tree for _other_ reasons, but
the two aren't really joined at the hip.

The third schenario is somewhere in between: you do pull from my tree, but
you do it occasionally enough that the merges don't get annoying.

Most people I work with are actually in that gray area. EVERYBODY pulls
some. Nobody tends to do black-and-white either-or schenario. The question
is just how much.

My gut rule: if you have almost as many merges as you have "real work"
commits, you're doing something bad, and you should pull less often
(perhaps by using "rebase", perhaps by just realizing that you don't need
to be tailgating _quite_ that closely).

But 30 "real" commits and 5 merges because the 30 real commits happened
over four weeks time? Sounds fine to me.

Linus

2006-01-09 03:52:45

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Mon, 9 Jan 2006, Al Viro wrote:
>
> How do you deal with conflict resolution? That's a genuine question -
> I'm not talking about deliberate misuse to hide an attack, just a normal
> situation when you have to resolve something like
>
> A:
> if (foo)
> bar
>
> B:
> if (foo & baz)
> bar
>
> A':
> #ifdef X
> if (foo)
> bar
> ...
> #endif
>
> merge of A' and B: trivial conflict

Actually, these days git is pretty good at it. Much better than CVS,
certainly. You can see the "conflict against my old tree", or "conflict
against the remote tree" by using the "--ours" or "--theirs" flag to "git
diff" respectively.

(Or "diff conflict against common base": "git diff --base").

So for your particular example with a trivial base file:

line 1
2
3
if (foo)
bar
6
7
8

and then the changes you had as an example in the A' and B branches, if I
from A' do a "git pull . B", I get:

Trying really trivial in-index merge...
fatal: Merge requires file-level merging
Nope.
Merging HEAD with ad56343c578785b8d932224a8676615e7a3e191f
Merging:
9d619225e3adecee6432a36d67d140e29b0acf62 A' case
ad56343c578785b8d932224a8676615e7a3e191f B: case
found 1 common ancestor(s):
93765ba3f64e9c73438e52683fffa68e5a493df7 Base commit
Auto-merging A
CONFLICT (content): Merge conflict in A

Automatic merge failed; fix up by hand

and then the file contains the contents

line 1
2
3
<<<<<<< HEAD/A
#ifdef X
if (foo)
=======
if (foo && baz)
>>>>>>> ad56343c578785b8d932224a8676615e7a3e191f/A
bar
6
#endif
7
8

ie it will have does a CVS-like merge for me, and I need to fix this up.
However, to _help_ me fix it up, I can now see what the diff is aganst my
original version (A'), with "git diff --ours" (the "--ours" is default, so
it's unnecessary, but just to make it explicit):

* Unmerged path A
diff --git a/A b/A
index 06dd3bc..7334364 100644
--- a/A
+++ b/A
@@ -1,8 +1,12 @@
line 1
2
3
+<<<<<<< HEAD/A
#ifdef X
if (foo)
+=======
+ if (foo && baz)
+>>>>>>> ad56343c578785b8d932224a8676615e7a3e191f/A
bar
6
#endif

which is very helpful especially once I have resolved it. IOW, I just edit
the file and do the trivial resolve, and now I can do a "git diff" again
to make sure that it looks ok:

* Unmerged path A
diff --git a/A b/A
index 06dd3bc..924fc97 100644
--- a/A
+++ b/A
@@ -2,7 +2,7 @@ line 1
2
3
#ifdef X
- if (foo)
+ if (foo && baz)
bar
6
#endif

ahh, looks good, so I just do "git commit A" and that creates the
resolved merge.

> The obvious way (edit file in question, update-index, commit) will not
> only leave zero information about said conflict and actions needed to
> deal with it, but will lead to situation when git whatchanged will not
> show anything useful.

Now, this is a real issue.

The resolve part is pretty easy, but the fact that it's hard to see in
"git-whatchanged" is a limitation of git-whatchanged.

You need to use "gitk", which _does_ know how to show merges as a diff
(and yes, I just checked).

> Is there any SOP I'm missing here?

You're just missing the fact that git-whatchanged (or rather,
"git-diff-tree") isn't smart enough to show merges nicely. It really
_should_. It doesn't. You can choose to show merges with the "-m" flag,
but that will show diffs against each parent, which really isn't what you
want.

I should do the same thing gitk does in git-diff-tree.

> Worse (for my use), format-patch on such tree will not give a useful patchset.
> You get a series of patches that won't apply to _any_ tree.

Now, git-diff-tree _does_ do that. Use the "-m" flag, and choose the tree
you want.

And btw, that works with "git-whatchanged" too. You _can_ pass the "-m"
flag to git-whatchanged, and it will show you each side of the merge
correctly. So it _works_. It's just such a horrible format that by default
it prefers to shut up about merges entirely.

(I don't know of a good three-way diff format. "gitk" can do it, because
gitk can show colors. That's a big deal when you do three-way - or
more-way - diffs).

> And that's a fundamental problem behind all that rebase activity, AFAICS.

You do _not_ want to rebase a merge. It not only won't work, it's against
the whole point of rebasing.

Rebasing is really only a valid operation when you have a few patches OF
YOUR OWN that you want to move up to a new version of somebody elses tree
that you are tracking. You fundamentally _cannot_ rebase if you've done
anything but a linear set of patches. And that has nothing to do with the
patch difficulty - it simply isn't an operation that makes sense.

(Btw, not making sense doesn't mean it might not work. It sometimes might
actually work and do what you _hoped_ it would do, but it's basically by
pure luck, and not because it is a sensible operation. Even stupid
people hit on the right solution every once in a while - not because
they thought about things right, but just because they happened to try
something that worked. The same is true of "git rebase" with merges ;^).

The fundamental reason a rebase doesn't make sense is that if you've done
a merge, it obviously means that some other branch has done development,
and already has the commits that you're trying to rebase. And you CANNOT
rebase for them.

So instead of rebasing across a merge, what you can do is to not do the
merge at all, but instead rebase one of the two branches against the
other. Then you can rebase the result against the thing that you wanted to
rebase them both against. Now you've never rebased a merge - you've just
linearised branches that were linear in themselves against each other.

Basically, a merge ties two branches together. Once you've merged, you
can't make a linear history any more. The merge fundamentally is not
linear.

Linus

2006-01-09 04:34:29

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On 1/9/06, Linus Torvalds <[email protected]> wrote:
> And then do
>
> git-rebase linus
>
> to rebase your development branch to mine.
>
> THIS is what "rebase" is for. It sounds like what you really want to do is
> not have a development branch at all, but you just want to track my tree
> and then keep track of a few branches of your own. In other words, you
> don't really have a "real" branch - you've got an odd collection of
> patches that you really want to carry around on top of _my_ branch. No?

FWIW, I determine whether I should rebase or merge based on

+ Whether the branch/head I maintain is public. For public repos, I
*must* merge carefully as rebase "rewinds" the head and that makes a
mess of any repositor tracking me.

+ Whether the changes on my both sides are significant, and it is
semantically meaningful to have a merge. If either side had just a
couple of minor commits, rebase makes life a lot easier down the path.
If both side clearly saw parallel development, it is more sincere to
merge and let that be recorded.

+ If my attempt to rebase leads to any non-trivial conflicts or
co-dependencies, then I definitely cancel the rebase and merge.

> Now, in this model, you're not really using git as a distributed system.

I'd argue that it is not about distributed or not. It's all in what
you want to record in your history. As such, it is a communication
device -- and I want to make effective use of it. I guess the question
I ask myself is: what will communicate what's happened here most
clearly? What will be useful for people to read? In that context, a
white-lie here and there simplifying the history a bit where it's not
interesting counts as a good thing.

cheers,

martin

2006-01-09 05:53:56

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

>On 1/9/06, Brown, Len <[email protected]> wrote:
>> Perhaps the tools should try to support what "a lot of people"
>> expect, rather than making "a lot of people" do extra work
>> because of the tools?
>
>I think it does. All the tricky stuff that David and Junio have been
>discussing is actually done very transparently by
>
> git-rebase <upstream>
>
>Now, git-rebase uses git-format-patch <options> | git-am <options> so
>it sometimes has problems merging. In that case, you can choose to
>either resolve the problem (see the doco for how to signal to git-am
>that you've resolved a conflict) or to cancel the rebase. If you
>choose to cancel the rebase, do
>
> cp .git/refs/heads/{<headname>,<headnamebadrebase>}
> cat .git/HEAD_ORIG > .git/refs/heads/<headname>
> git-reset --hard
> rm -fr .dotest
>
>and you'll be back to where you started. Perhaps this could be rolled
>into something like git-rebase --cancel to make it easier, but that's
>about it. The toolchain definitely supports it.

This is completely insane.
Do you have any idea what "sometimes has problems merging" means
in practice? It means the tools are really nifty in the trivial
case but worse than worthless when you need them the most.

-Len

2006-01-09 05:55:50

by George Spelvin

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

> And lastly, is there a tool whereby I can "see" changes
> between repos, kind of like git-diff but being able to
> give URLs too?

Write it yourself. It's git-fetch + git-diff.

Or, put another way, if you think you need a special tool for working
with a remote repository, you don't understand git-fetch.

Since git history is immutable, there is no difference between a remote
copy and a local copy. And since fetching is harmless to your local
repository, there's no problem.

If you don't want to copy the entire history, just fetch the tree rather
than the commit. (Does git-fetch do that? It's a subset of its current
effects, so would be an easy enough extension.)

2006-01-09 06:08:46

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On 1/9/06, Brown, Len <[email protected]> wrote:
> This is completely insane.
> Do you have any idea what "sometimes has problems merging" means
> in practice? It means the tools are really nifty in the trivial
> case but worse than worthless when you need them the most.

Len,

all I meant was that you will sometimes see conflicts. And in that
case, you are far better off cancelling the rebase and doing a merge,
where you will have to resolve the conflicts by hand.

git-rebase is for when the potential merge is clearly trivial. In any
other case, you do want a proper merge. But in any case, it is easy to
do

git-fetch <upstream> && git-rebase <upstream>

and if it does anything but a very trivial merge, backtrack and do a merge.

In any case, if I have any suspicion that the merge may not be trivial, I do

git-fetch <upstream> && gitk --since=" 1 month ago" upstream master

before deciding on a course of action. Of course, you can merge all
the time. It's whether people care about a readable/useful history
afterwards.

cheers,

martin

2006-01-09 06:13:57

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Mon, 9 Jan 2006, Martin Langhoff wrote:
>
> and if it does anything but a very trivial merge, backtrack and do a merge.

To be fair, backtracking a "git-rebase" isn't obvious. One of the
downsides of rebasing.

Linus

2006-01-09 06:14:00

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

>Which just means that a commit that was tested and found to be working
>might suddenly not work any more, which can be very surprising ("But I
>didn't change anything!").

This is why I try to follow the top of tree as closely as possible.
While parts of ACPI are well contained, other parts get stomped on constantly.
My two updates in 10 hours were not becaudse of what I did,
it was in response to seeing the flood gates in 2.6.16 open.

Also, it isn't accurate that nothing changed at my end.
I put some trivial patches into my release branch,
udpated the release branch with upstream,
and then pulled the release branch into my test branch
where there are other patches you haven't seen yet.

I pushed the release branch just to 'clear the deck' of
the trivial patches there since there was no reason to delay
them behind all the stuff still on the test branch.

>So trying out git-rebase and git-cherry-pick just in case you
>decide to want to use them might be worthwhile. Making it part of your
>daily routine like David has done? Somewhat questionable, but hey,
>it seems to be working for David, and it does make some things much easier, so..

In the past I have use git-cherry-pick to recover from when a "good" patch
was checked in on top of a "bad" patch, and I wanted to push
the good without the bad.

But the linearization model will not work for me in general.
Branches enable parallel lines of development in git. If that
is thrown out, then we're basically back at quilt.

thanks,
-Len

2006-01-09 06:28:06

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

>From: David S. Miller

>I think merges with conflicts that need to get resolved by hand create
>a lot of noise and useless information and therefore to me they are
>pointless. But this is just my opinion. It simply works easier to me
>to shuffle the patches in by hand and deal with the rejects one by
>one. It's very much akin to how Andrew's -mm tree works.

I guess in this model you can do all your development with quilt,
and the value of git is a high-bandwidth bransport medium to
replace e-mail.

>I think a clean history is worth an extra few minutes of someone's
>time. And note that subsystem development is largely linear anyways.

Maybe true in your neck of the woods, but not true here.

I have more than a dozen topic branches in my tree, and they mature
at different rates.

When a topic branch is in the test tree and and a follow-up patch
is needed, I check out that topic branch and put the patch
exactly in non-linear 3D history where it is meant to live.
When the topic seems fully baked, I can pull the top of the
branch into the release tree.

-Len

2006-01-09 06:46:12

by Junio C Hamano

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

Linus Torvalds <[email protected]> writes:

> To be fair, backtracking a "git-rebase" isn't obvious. One of the
> downsides of rebasing.

I thought "git reset --hard ORIG_HEAD" as usual would do.

2006-01-09 07:35:44

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

Martin,
I appologize for using the phrase "completely insane".
The rebase proposal caught me somewhat off-guard and
I was expressing surprise -- hopefully not taken as insult.

Further, I thank you for your thoughful follow-up.

While I don't expect it to become a routine occurnece for me,
I do see that rebase has utility in some situations.

-Len

2006-01-09 08:06:36

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

Linus,
I think Tony has articulated the work-flow problem that
originally started this thread, as well as the fix.

>I'll try to update the using-topic-branches document to capture this.
>Some of the problem is that it doesn't quite capture what I'm doing
>with my test/release branches.
>
>My release branch really is just used as a transfer point to Linus.
>I usually[1] don't leave patches sitting in "release" for long enough
>that I'll be tempted to merge in from Linus ... once I decide that
>some patches are ready to go to Linus I'll update "release" from Linus
>(which will be a fast-forward, so no history) merge in the topic
>branches, do one final sanity build, push to kernel.org and send
>the "please pull" e-mail.
>
>The huge majority of my "automatic update from upstream" merges
>go into my test branch ... which never becomes part of the real
>history as I never ask Linus to pull from it.
>
>-Tony
>
>[1] Sometimes I goof on this because I forget that I've applied
>a trivial patch directly to the release branch without going through
>a topic branch. I think I'll fix my update script to check
>for this case.

I figured that checking some trivial patches directly into "release"
would be a convenient way to make sure I didn't forget to push them --
as they didn't depend on anything else in my tree. Okay.

To make sure that my test branch (where I generate my consolidated
plain patch, and what Andrew pulls) includes everything, I then pull
"release" into "test". Still good.

But then I decide I need to update my test tree from upstream.
I did this by pulling "linus" into "release", and then pulling
"release" into "test". This creates the book-keeping merge
in "release" that irritates gitk users.

This "flow", BTW, is a habit I picked up from the
"two-phase release strategy" that we used in bk days.
There I'd pull from upstream down into my to-linus tree and then pull
from the to-linus tree into the to-andrew tree.
I expect BK also created a merge cset, but apparently
nobody was looking at the history like they do with gitk today.

So if I simply don't pull from "linus" into a modified
"release" branch then the cluttered history issue goes away.
I should fetch "linus" into "release" right before I merge
the topic branches into "release" and push upstream.
The fetch is a clean fast-forward, and the merges all have
real content.

This will work as long as "release" doesn't get too old
to be pulled upstream without conflicts. Based on past
experience with low latency pulls upstream, I think this will be rare.

Andrew will still get cluttered history in the test tree,
but as he's focused on the content and not the (throw-away) history,
this is surely a non-issue.

So problem #1 is solved, yes?

Going forward...
I'm hopeful that gitk users will not be irritated also
by the liberal use of topic branches. I'm starting to like using
them quite a bit. Yes, it is true that I could cherry-pick
the topics out of their original context to re-manufacture linear
history. But that is extra work. Also, as you poined out,
there is real value in the real history because the context is accurate.
Further, I find that sometimes I need to augment a topic branch
with a follow-up patch. I can checkout the topic branch an plop
the follow-up right on the tip where it logically should live,
and (Tony's) scripts will remind me when the branch is not fully
pulled into test or release -- so it will never get misplaced.

In the case where a topic branch is a single commit, gitk users
will see both the original commit, as well as the merge commit
back into "release".

-Len

2006-01-09 10:11:22

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On 1/9/06, Brown, Len <[email protected]> wrote:
> I appologize for using the phrase "completely insane".
> The rebase proposal caught me somewhat off-guard and
> I was expressing surprise -- hopefully not taken as insult.
>
> Further, I thank you for your thoughful follow-up.

No worries... and no offence taken! In a sense we are still exploring
possible/desirable workflows and what the missing pieces are. And yes,
some thing don't quite make sense from the outside, perhaps because
they just don't or because we arent' explaining them very well.

In a sense, we do have a bit of a challenge explaining what how all
the parts fit together, even to bk old timers it seems.

> While I don't expect it to become a routine occurnece for me,
> I do see that rebase has utility in some situations.

As long as you've got enough tools to use a workflow that fits you,
it's all good.

cheers,

martin

2006-01-09 12:31:57

by Johannes Schindelin

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

Hi,

On Mon, 9 Jan 2006, Martin Langhoff wrote:

> In a sense we are still exploring possible/desirable workflows and what
> the missing pieces are. And yes, some thing don't quite make sense from
> the outside, perhaps because they just don't or because we arent'
> explaining them very well.

Maybe what is needed here is this:

T1 - T2 .. Tn .. Tp
\ \ \
\ M1 M2
\ / /
B1 .. Bm .. Bo

where T1..Tp are the upstream commits, B1..Bo are the local commits, and
M1.. are the test merges just to make sure nothing breaks?

As long as the Mx commits resolve automatically, no need for an explicit
merge in the Bx commits, since a pull from B into T will just recreate an
Mx as next commit in T.

Kind of "throw away merge".

Ciao,
Dscho

2006-01-09 16:48:16

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

On Mon, 9 Jan 2006, Brown, Len wrote:
> >
> >The huge majority of my "automatic update from upstream" merges
> >go into my test branch ... which never becomes part of the real
> >history as I never ask Linus to pull from it.
> >
> >-Tony
> >
> >[1] Sometimes I goof on this because I forget that I've applied
> >a trivial patch directly to the release branch without going through
> >a topic branch. I think I'll fix my update script to check
> >for this case.
>
> I figured that checking some trivial patches directly into "release"
> would be a convenient way to make sure I didn't forget to push them --
> as they didn't depend on anything else in my tree. Okay.

One thing we could do is to make it easier to apply a patch to a
_non_current_ branch.

In other words, let's say that we want to encourage the separation of a
"development branch" and a "testing and use" branch (which I'd definitely
personally like to encourage people to do).

And one way to do that might be to teach "git-apply" to apply patches to a
non-active branch, and then you keep the "testing and use" branch as your
_checked_out_ branch (and it's going to be really dirty), but when you
actually apply patches you could do that to the "development" branch with
something like

git-apply -b development < patch-file

(Now, of course, that's only if you apply somebody elses patch - if you
actually do development _yourself_, you'd either have to check out the
development branch and do it there, or you'd move the patch you have in
your "ugly" checked-out testing branch into the development branch with

git diff | git-apply -b development

or something similar..)

Then you could always do "git pull . development" to pull in the
development stuff into your working branch - keeping the development
branch clean all the time.

Do you think that kind of workflow would be more palatable to you? It
shouldn't be /that/ hard to make git-apply branch-aware... (It was part of
my original plan, but it is more work than just using the working
directory, so I never finished the thought).

> I'm hopeful that gitk users will not be irritated also
> by the liberal use of topic branches.

"gitk" is actually pretty good at showing multiple branches. Try doing a

gitk --all -d

and you'll see all the topic branches in date order. The "-d" isn't
strictly necessary, and to some degree makes the output messier by
interleaving the commits from different branches, so you may not want to
do it, but it is sometimes nice to see the "relative dates" of individual
commits rather than the denser format that gitk defaults to.

> In the case where a topic branch is a single commit, gitk users
> will see both the original commit, as well as the merge commit
> back into "release".

Yes, topic branches will always imply more commits, but I think they are
of the "nice" kind.

I definitely encourage people to use git as a distributed concurrent
development system ratehr than the "collection of patches" thing. Quilt is
much better at the collection of patches.

So I'd encourage topic branches - even within something like ACPI, you
might have separate topics ("interpreter" branch vs "x86" branch vs
"generic-acpi" branch).

And yes, that will make history sometimes messier too, and it will cause
more merges, but the difference there is that the merges will be
meaningful (ie merging the "acpi interpreter" branch into the generic ACPI
branch suddenly has _meaning_, even if there only ends up being a couple
of commits per merge).

Ok?

Linus

2006-01-09 16:57:17

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

On Mon, 9 Jan 2006, Linus Torvalds wrote:
>
> One thing we could do is to make it easier to apply a patch to a
> _non_current_ branch.
> [ ... ]
> Do you think that kind of workflow would be more palatable to you? It
> shouldn't be /that/ hard to make git-apply branch-aware... (It was part of
> my original plan, but it is more work than just using the working
> directory, so I never finished the thought).

Btw, this is true in a bigger sense: the things "git" does have largely
been driven by user needs. Initially mainly mine, but things like
"git-rebase" were from people who wanted to work as "sub-maintainers" (eg
Junio before he became the head honcho for git itself).

But if there are workflow problems, let's try to fix them. The "apply
patches directly to another branch" suggestion may not be sane (maybe it's
too confusing to apply a patch and not actually see it in the working
tree), but workflow suggestions in general are appreciated.

We've made switching branches about as efficient as it can be (but if the
differences are huge, the cost of re-writing the working directory is
never going to be low). But switching branches has the "confusion factor"
(ie you forget which branch you're on, and apply a patch to your working
branch instead of your development branch), so maybe there are other ways
of doing the same thing that might be sensible..

So send suggestions to the git lists. Maybe they're insane and can't be
done, but while I designed git to work with _my_ case (ie mostly merging
tons of different trees and then having occasional big batches of
patches), it's certainly _supposed_ to support other maintainers too..

Linus

2006-01-09 20:06:56

by Junio C Hamano

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

Linus Torvalds <[email protected]> writes:

> One thing we could do is to make it easier to apply a patch to a
> _non_current_ branch.
>...
> And one way to do that might be to teach "git-apply" to apply patches to a
> non-active branch,...
>
> git diff | git-apply -b development
>
> or something similar..)
>
> Then you could always do "git pull . development" to pull in the
> development stuff into your working branch - keeping the development
> branch clean all the time.

I had to do something like that last night, when I hacked on
gitweb. gitweb as shipped does not work for anybody but kay
(e.g. it has /home/kay hardcoded in it). So I did:

$ git clone git://git.kernel.org/pub/scm/git/gitweb gitweb
$ cd gitweb
$ git checkout -b custom
$ edit gitweb.cgi ;# adjust /home/kay -> somewhere else etc.
$ git commit -a -m "customization for junio's home"

Then I started preparing a proposed fix for Kay:

$ git checkout -b symref master
$ edit gitweb.cgi
$ git commit -a -s -m "make it work on symref repository"

Now the thing is that I cannot test symref branch as is. I
deliberately omitted the change necessary to make the upstream
work on my local machine from that branch, because I want to
keep my home-machine customization separate from what I will
eventually feed Kay. So I do a throwaway test branch:

$ git checkout -b test master
$ git pull . custom symref ;# an octopus ;-)
# I could have done two separate pulls, custom then symref.

The interesting part starts here. Inevitably, I find bugs and
bugs and bugs in the test branch, and I fix them in the working
tree, without committing. Eventually things starts working.
I did not commit here in the test branch, because the symref
branch is where I intend to keep this set of changes. So
instead, I did this:

$ git diff HEAD >P.diff
$ git checkout -f symref
$ git reset --soft HEAD^
$ git apply P.diff
$ git commit -a -C ORIG_HEAD

Usually I strongly discourage people to use "checkout -f"
because it will leave files that are in the current branch but
not in the new branch behind in the working tree. Here I used
"checkout -f symref" because I knew this is a one-file project.

Instead of fixing the symref commit in place like this, I could
have committed P.diff as a separate "fixup" commit on top of the
symref branch, in which case the above sequence would have been:

$ git diff HEAD >P.diff
$ git checkout -f symref
$ git apply P.diff
$ git commit -a -m 'fixup bugs in the previous.'

but I did not --- it would have been more disgusting than
honest.

And after that, the usual format-patch:

$ git format-patch origin..symref

In either case, this *was* cumbersome. And I did it twice for
two independent topics. Admittedly, these topic branches were
both single-commit topics, and in real life your subsystem
maintainers must be facing bigger mess than this toy experience
of mine, but the principle is the same.

I think there are a couple of ways to improve what I had to do.
I'll think aloud here. The fictitious transcripts all start
after I got things working in the test branch working tree, with
a clean index file (i.e. changes are in the working tree only).

1. Make a commit in the "test" branch, and then cherry-pick the
commit back to the topic branch:

$ git commit -a -m "Fix symref fix"
$ git checkout symref
$ git cherry-pick -r test

2. Fix "git checkout <branch>" so that it does a reasonable thing
even when a dirty path is different in current HEAD and
destination branch. Then I could:

$ git checkout symref ;# this would not work in the current git
# it would die like this:
# $ git checkout symref
# fatal: Entry 'gitweb.cgi' not uptodate. Cannot merge.
$ git diff ;# just to make sure inevitable automated merge
# did the right thing
$ git commit -a -m "Fix symref fix"
# I could collapse them into one instead, like this:
# $ git reset --soft HEAD^
# $ git commit -a -C ORIG_HEAD

To retest (possibly with latest from Kay), we can rebuild the
test branch from scratch since it is by definition a throwaway
branch and never is exposed to public:

$ git fetch origin
$ git checkout test
$ git reset --head origin
$ git pull . custom symref

Obviously I prefer to have #2 work well, but #1 would work today.

I am not sure if making "git-apply" to take different branch is
a sane approach. It might make sense to teach git-applymbox and
git-am about branches, though. So is teaching git-merge about
merging into different branch.

2006-01-09 22:51:48

by Luben Tuikov

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

--- Linus Torvalds <[email protected]> wrote:
> But if there are workflow problems, let's try to fix them. The "apply
> patches directly to another branch" suggestion may not be sane (maybe it's
> too confusing to apply a patch and not actually see it in the working
> tree), but workflow suggestions in general are appreciated.

This is sensible, thank you.

A very general workflow I've seen people use is more/less as
I outlined in my previous email:

tree A (linus' or trunk)
Project B (Tree B)
Project C (Tree C, depending on stuff in Project B)

Now this could be how the "managers" see things, but development,
could've "cloned" from Tree B and Tree C further, as is often
customary to have a a) per user tree, or b) per bug tree.

So pull/merge/fetch/whatever follows Tree A->B->C.

It is sensible to have another tree say, called something
like "for_linus" or "upstream" or "product" which includes
what has accumulated in C from B and in B from A, (eq diff(C-A)).
I.e. a "push" tree. So that I can tell you, "hey,
pull/fetch/merge/whatever the current verb en vogue is, from
here to get latest xyz".

What I also wanted to mention is that Tree B undeniably
depends on the _latest_ state of Tree A, since Project B
uses API/behaviour of the code in Tree A, so one cannot just
say they are independent. Similarly for Tree C/Project C,
is dependent on B, and dependent on A.

Also sometimes a bugfix in C, prompts a bugfix in A,
so that the bugfix in A doesn't apply unless the bugfix in C.
(To get things more complicated.)

I think this is more/less the most easier to see, understand and
follow workflow approach, which is also the case for other SCMs.

What are the commands to follow to make everyone happy when
pulling from such a development process?

FWIW, "git diff A C | send to Linus" would get you the
"no merge messages/ancestors I want to see" idea, if I understand
this thread correctly.

> We've made switching branches about as efficient as it can be (but if the
> differences are huge, the cost of re-writing the working directory is
> never going to be low). But switching branches has the "confusion factor"
> (ie you forget which branch you're on, and apply a patch to your working
> branch instead of your development branch), so maybe there are other ways
> of doing the same thing that might be sensible..

Yes. Ever since I started used git, I never used branch
switching, but I do have git branches and I do use git branching.

I basically have a branch per directory, whereby the object db
is shared as is remotes/refs/etc, HEAD and index are not shared
of course.

This allows me to do a simple and fast "cd" to change/go to a
different branch, since they are in different directories.
So the time I wait to switch branches is the time the filesystem
takes to do a "cd".

This also allows me to build/test/patch/work on branches
simultaneously.

Thank you,
Luben

2006-01-09 23:09:31

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

On Mon, 9 Jan 2006, Luben Tuikov wrote:
>
> Yes. Ever since I started used git, I never used branch
> switching, but I do have git branches and I do use git branching.
>
> I basically have a branch per directory, whereby the object db
> is shared as is remotes/refs/etc, HEAD and index are not shared
> of course.
>
> This allows me to do a simple and fast "cd" to change/go to a
> different branch, since they are in different directories.
> So the time I wait to switch branches is the time the filesystem
> takes to do a "cd".
>
> This also allows me to build/test/patch/work on branches
> simultaneously.

Yes. It has many advantages, and it's the approach I pushed pretty hard
originally, but the "many branches in the same tree" approach seems to
have become the more common one. Using many branches in the same tree is
definitely the better approach for _distribution_, but that doesn't
necessarily mean that it's the better one for development.

For example, you can have a git distribution tree with 20 different
branches on kernel.org, but do development in 20 different trees with just
one branch active - and when you do a "git push" to push out your branch
in your development tree, it just updates that one branch on the
distribution site.

So git certainly supports that kind of behaviour, but nobody I know
actually does it that way (not even me, but since I tend to just merge
other peoples code, I don't actually have multiple branches: I create
temporary branches for one-off things, but don't maintain them that way).

Linus

2006-01-09 23:34:41

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On 1/10/06, Linus Torvalds <[email protected]> wrote:
> Using many branches in the same tree is
> definitely the better approach for _distribution_, but that doesn't
> necessarily mean that it's the better one for development.
(...)
> So git certainly supports that kind of behaviour, but nobody I know
> actually does it that way

Hrm! We do. http://locke.catalyst.net.nz/gitweb?p=moodle.git;a=heads
shows a lot of heads that share 99% of the code. The repo is ~90MB --
and we check each head out with cogito, develop and push. It is a
shared team repo, using git+ssh and sticky gid and umask 002.

Works pretty well I have to add. The only odd thing is that the
fastest way to actually start working on a new branch is to ssh on to
the server and cp moodle.git/refs/heads/{foo,bar} and then cg-clone
that bar branch away. Perhaps I should code up an 'cg-branch-add
--in-server' patch.

regards,

martin

2006-01-10 02:50:28

[permalink] [raw]

Subject: RE: git pull on Linux/ACPI release tree

On Mon, 9 Jan 2006, Luben Tuikov wrote:
>
> A very general workflow I've seen people use is more/less as
> I outlined in my previous email:
>
> tree A (linus' or trunk)
> Project B (Tree B)
> Project C (Tree C, depending on stuff in Project B)
>
> Now this could be how the "managers" see things, but development,
> could've "cloned" from Tree B and Tree C further, as is often
> customary to have a a) per user tree, or b) per bug tree.
>
> So pull/merge/fetch/whatever follows Tree A->B->C.
>
> It is sensible to have another tree say, called something
> like "for_linus" or "upstream" or "product" which includes
> what has accumulated in C from B and in B from A, (eq diff(C-A)).
> I.e. a "push" tree. So that I can tell you, "hey,
> pull/fetch/merge/whatever the current verb en vogue is, from
> here to get latest xyz".
>
> What I also wanted to mention is that Tree B undeniably
> depends on the _latest_ state of Tree A, since Project B
> uses API/behaviour of the code in Tree A, so one cannot just
> say they are independent. Similarly for Tree C/Project C,
> is dependent on B, and dependent on A.

Note that in the case where the _latest_ state of the tre you are tracking
really matters, then doing a "git pull" is absolutely and unquestionably
the right thing to do.

So if people thought that I don't want to have sub-maintainers pulling
from my tree _at_all_, then that was a mis-communication. I don't in any
way require a linear history, and criss-cross merges are supported
perfectly well by git, and even encouraged in those situations.

After all, if tree B starts using features that are new to tree A, then
the merge from A->B is required for functionality, and the synchronization
is a fundamental part of the history of development. In that cases, the
history complexity of the resulting tree is a result of real development
complexity.

Now, obviously, for various reasons we want to avoid having those kinds of
linkages as much as possible. We like to have develpment of different
subsystems as independent as possible, not because it makes for a "more
readable history", but because it makes it a lot easier to debug - if we
have three independent features/development trees, they can be debugged
independently too, while any linkages inevitably also mean that any bugs
end up being interlinked..

Linus

2006-01-10 06:34:04

by Kyle Moffett

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Jan 09, 2006, at 21:50, Linus Torvalds wrote:
> if we have three independent features/development trees, they can
> be debugged independently too, while any linkages inevitably also
> mean that any bugs end up being interlinked..

One example:

If I have ACPI, netdev, and swsusp trees change between an older
version and a newer one, and my net driver starts breaking during
suspend, I would be happiest debugging with the following set of
patches/trees (Heavily simplified):

^
|
[5]
|
broken
^ ^ ^
[2] [3] [4]
/ | \
netdev3 acpi3 swsusp3
^ ^ ^
| | |
netdev2 acpi2 swsusp2
^ ^ ^
| | |
netdev1 acpi1 swsusp1
^ ^ ^
\ | /
\ | /
\ | /
\|/
|
[1]
|
works

If the old version [1] works and the new one [5] doesn't, then I can
immediately test [2], [3], and [4]. If one of those doesn't work,
I've identified the problematic patchset and cut the debugging by
2/3. If they all work, then we know precisely that it's the
interactions between them, which also makes debugging a lot easier.

Cheers,
Kyle Moffett

--
There are two ways of constructing a software design. One way is to
make it so simple that there are obviously no deficiencies. And the
other way is to make it so complicated that there are no obvious
deficiencies. The first method is far more difficult.
-- C.A.R. Hoare

2006-01-10 06:38:12

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On 1/10/06, Kyle Moffett <[email protected]> wrote:
> If they all work, then we know precisely that it's the
> interactions between them, which also makes debugging a lot easier.

The more complex your tree structure is, the more the interactions are
likely to be part of the problem. Is git-bisect not useful in this
scenario?

cheers,

martin

2006-01-10 15:32:31

by Alex Riesen

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On 1/9/06, Junio C Hamano <[email protected]> wrote:
> 2. Fix "git checkout <branch>" so that it does a reasonable thing
> even when a dirty path is different in current HEAD and
> destination branch. Then I could:
>
> $ git checkout symref ;# this would not work in the current git
> # it would die like this:
> # $ git checkout symref
> # fatal: Entry 'gitweb.cgi' not uptodate. Cannot merge.

That is actually very interesting. I already wished sometimes to be
able to switch branches with a dirty working directory (and usually
ended up with git diff+checkout+apply).
Even if it results in a merge and conflict markers in files it looks
like a very practical idea!

> $ git diff ;# just to make sure inevitable automated merge
> # did the right thing
> $ git commit -a -m "Fix symref fix"
> # I could collapse them into one instead, like this:
> # $ git reset --soft HEAD^
> # $ git commit -a -C ORIG_HEAD

2006-01-10 15:32:01

by Catalin Marinas

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

"Brown, Len" wrote:
>>I _really_ wish you wouldn't have those automatic merges.
>>
>>Why do you do them? They add nothing but ugly and unnecessary
>>history, and in this pull, I think almost exactly half of the
>>commits were just these empty merges.
>
> Is it possible for it git, like bk, to simply ignore merge commits
> in its summary output?

As Junio suggested, you can have a look at StGIT
(http://www.procode.org/stgit/) for a different workflow. There is a
tutorial both on the web and in the doc/ directory but, anyway, it is
pretty similar to Quilt only that the patches are GIT commits.

In principle, you keep all the patches in a stack whose base is the
HEAD of Linus' kernel. You can indefinitely modify/push/pop the
patches and, once you are happy with the state of the stack, ask Linus
to pull using standard GIT commands (or mail them with 'stg
mail'). You can afterwards pull the latest changes from Linus using
'stg pull'. This operation pops the patches you have, advances the
base of the stack (so no "merge" message) and pushes your patches
back. Since pushing is done with a three-way merge, it detects whether
there are any upstream modifications to your patches (if not, all the
patches should become empty and safely removed from the stack).

You can also have a branch for upstream merges only and you can easily
cherry-pick patches or commits from other branches. This is quite
useful if you want to continue the work on your development branch
until Linus merges your patches.

--
Catalin

2006-01-10 18:06:57

by Kyle Moffett

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Jan 10, 2006, at 01:38, Martin Langhoff wrote:
> On 1/10/06, Kyle Moffett <[email protected]> wrote:
>> If they all work, then we know precisely that it's the
>> interactions between them, which also makes debugging a lot easier.
>
> The more complex your tree structure is, the more the interactions
> are likely to be part of the problem. Is git-bisect not useful in
> this scenario?

IIRC git-bisect just does an outright linearization of the whole tree
anyways, which makes git-bisect work everywhere, even in the presence
of difficult cross-merges. On the other hand, if you are git-
bisecting ACPI changes (perhaps due to some ACPI breakage), and ACPI
has 10 pulls from mainline, you _also_ have to wade through the
bisection of any other changes that occurred in mainline, even if
they're totally irrelevant. This is why it's useful to only pull
mainline into your tree (EX: ACPI) when you functionally depend on
changes there (as Linus so eloquently expounded upon).

Cheers,
Kyle Moffett

--
Q: Why do programmers confuse Halloween and Christmas?
A: Because OCT 31 == DEC 25.

2006-01-10 18:28:22

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Tue, 10 Jan 2006, Kyle Moffett wrote:
>
> On Jan 10, 2006, at 01:38, Martin Langhoff wrote:
> >
> > The more complex your tree structure is, the more the interactions are
> > likely to be part of the problem. Is git-bisect not useful in this scenario?
>
> IIRC git-bisect just does an outright linearization of the whole tree anyways,
> which makes git-bisect work everywhere, even in the presence of difficult
> cross-merges.

It's not really a linearization - at no time does git-bisect _order_ the
commits. After all, no linear order actually exists.

Instead, it really cuts the tree up into successively smaller parts.

Think of it as doing a binary search in a 2-dimensional surface - you
can't linearize the plane, but you can decide to test first one half of
the surface, and then depending on whether it was there, you can halve
that surface etc..

> On the other hand, if you are git-bisecting ACPI changes
> (perhaps due to some ACPI breakage), and ACPI has 10 pulls from mainline, you
> _also_ have to wade through the bisection of any other changes that occurred
> in mainline, even if they're totally irrelevant.

Yes. Although if you _know_ that the problem happened in a specific file
or specific subdirectory, you can actually tell "git bisect" to only
bother with changes to that file/directory/set-of-directories to speed up
the search.

IOW, if you absolutely know that it's ACPI-related, you can do something
like

git bisect start drivers/acpi arch/i386/kernel/acpi

to tell the bisect code that it should totally ignore anything that
doesn't touch those two directories.

However, if it turns out that you were wrong (and the ACPI breakage was
brought on by something that changed something else), "git bisect" will
just get confused and report the wrong commit, so this is really something
you should be careful with (and verify the end result by checking that
undoing that _particular_ commit really fixes things).

And yes, "git bisect" _will_ work with bugs that depend on two branches of
a merge: it will point to the merge commit itself as being the problem.
Now, at that point you really are screwed, and you'll have to figure out
why both branches work, but the combination of them do not.

Maybe it's as simple as just a merge done wrong (bad manual fixups), but
maybe it's a perfectly executed merge that just happens to have one branch
changing the assumptions that the other branch depended on.

Happily, that is not very common. I know people are using "git bisect",
and I don't think anybody has ever reported it so far. It will happen
eventually, but I'd actually expect it to be much more common that "git
bisect" will hit other - worse - problems, like bugs that "come and go",
and that a simple bisection simply cannot find because they aren't
totally repeatable.

Linus

2006-01-10 18:45:22

by Johannes Schindelin

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

Hi,

On Tue, 10 Jan 2006, Linus Torvalds wrote:

>
> On Tue, 10 Jan 2006, Kyle Moffett wrote:
> >
> > On Jan 10, 2006, at 01:38, Martin Langhoff wrote:
> > >
> > > The more complex your tree structure is, the more the interactions are
> > > likely to be part of the problem. Is git-bisect not useful in this scenario?
> >
> > IIRC git-bisect just does an outright linearization of the whole tree anyways,
> > which makes git-bisect work everywhere, even in the presence of difficult
> > cross-merges.
>
> It's not really a linearization - at no time does git-bisect _order_ the
> commits. After all, no linear order actually exists.
>
> Instead, it really cuts the tree up into successively smaller parts.
>
> Think of it as doing a binary search in a 2-dimensional surface - you
> can't linearize the plane, but you can decide to test first one half of
> the surface, and then depending on whether it was there, you can halve
> that surface etc..

How?

If you bisect, you test a commit. If the commit is bad, you assume *all*
commits before that as bad. If it is good, you assume *all* commits after
that as good.

Now, if you have a 2-dimensional surface, you don't have a *point*, but
typically a *line* separating good from bad.

Further, the comparison with 2 dimensions is particularly bad. You
*have* partially linear development lines, it got *nothing* to do with
an area. The commits still make up a *list*, and it depends how you
*order* that list for bisect. (And don't tell me they are not ordered:
they are.)

If you order the commits by date, you don't get anything meaningful point
before which it is bad, and after which it is good.

So, how is bisect supposed to work if you don't have one straight
development line from bad to good?

Ciao,
Dscho

2006-01-10 19:04:04

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Tue, 10 Jan 2006, Johannes Schindelin wrote:
> >
> > Think of it as doing a binary search in a 2-dimensional surface - you
> > can't linearize the plane, but you can decide to test first one half of
> > the surface, and then depending on whether it was there, you can halve
> > that surface etc..
>
> How?
>
> If you bisect, you test a commit. If the commit is bad, you assume *all*
> commits before that as bad. If it is good, you assume *all* commits after
> that as good.

No, that's not how bisect works at all.

It's true that if a commit is bad, then all the commits _reachable_ from
that commit are considered bad.

And it's true that if a commit is good, then all commits that _reach_ that
commit are considered good.

But that doesn't mean that there is an ordering. The commits that fall
into the camp of being "neither good nor bad" are _not_ ordered. There are
commits in there that are not directly reachable from the good commit.

> Now, if you have a 2-dimensional surface, you don't have a *point*, but
> typically a *line* separating good from bad.

Exactly.

And a git graph is not really a two-dimensional surface, but exactly was
with a 2-dimensional surface, it is _not_ enough to have a *point* to
separate the good from bad.

You need to have a _set of points_ to separate the good from the bad. You
can think of it as a line that bisects the surface: if you were to print
out the development graph, the set of points literally _do_ form a virtual
line across the development surface.

(Actually, you can't in general print out the development graph on a
2-dimensional paper without having development lines that cross each
other, but you could actually do it in three dimensions, where the
"boundary" between good and bad is actually a 2-dimensional surface in
3-dimensional space).

But to describe the surface of "known good", you actually just need a list
of known good commits, and the "commits reachable from those commits"
_becomes_ the surface.

> Further, the comparison with 2 dimensions is particularly bad.

No it is not. It's a very good comparison.

In a linearized model (one-dimensional, fully ordered set), the only thing
you need for bisection is two points: the beginning and the end.

In the git model, you need _many_ points to describe the area being
bisected. Exactly the same way as if you were to bisect a 2-dimensional
surface.

Now, the git history is _not_ really a two-dimensional surface, so it's
just an analogy, not an exact identity. But from a visualization
standpoint, it's a good way to think of each "git bisect" as adding a
_line_ on the surface rather than a point on a linear line.

> So, how is bisect supposed to work if you don't have one straight
> development line from bad to good?

Read the code.

I'm pretty proud of it. It's simple, and it's obvious once you think about
it, but it is pretty novel as far as I know. BK certainly had nothing
similar, not have I heard of anythign else that does it. Git _might_ be
the first thing that has ever done it, although it's simple enough that I
wouldn't be surprised if others have too.

Linus

2006-01-10 19:30:13

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Tue, 10 Jan 2006, Linus Torvalds wrote:
>
> Now, the git history is _not_ really a two-dimensional surface, so it's
> just an analogy, not an exact identity. But from a visualization
> standpoint, it's a good way to think of each "git bisect" as adding a
> _line_ on the surface rather than a point on a linear line.

Actually, the way I think of it is akin to the "light cones" in physics. A
point in space-time doesn't define a fully ordered "before and after": but
it _does_ describe a "light cone" which tells you what is reachable from
that point, and what that point reaches. Within those cones, that
particular point ("commit") has a strict ordering.

And exactly as in physics, in git there's a lot of space that is _not_
ordered by that commit. And the way to bisect is basically to find the
right points in "git space" to create the right "light cone" that you
find the point where the git space that is reachable from that commit has
the same volume as the git space that isn't reachable.

And maybe that makes more sense to you (if you're into physics), or maybe
it makes less sense to you.

Now, since we always search the "git space" in the cone that is defined by
"reachable from the bad commit, but not reachable from any good commit",
the way we handle "bad" and "good" is actually not a mirror-image. If we
fine a new _bad_ commit, we know that it was reachable from the old bad
commit, and thus the old bad commit is now uninteresting: the new bad
commit forms a "past light cone" that is a strict subset of the old one,
so we can totally discard the old bad commit from any future
consideration. It doesn't tell us anything new.

In contrast, if we find a new _good_ commit, the "past light cone" (aka
"set of commits reachable from it") is -not- necessarily a proper superset
of the previous set of good commits, so when we find a good commit, we
still need to carry the _other_ good commits around, and the "known good"
universe is the _union_ of all the "good commit past lightcones".

Then the "unknown space" is the set difference of the "past lightcone of
the bad commit" and of this "union of past lightcones of good commits".
It's the space that is reachable from the known-bad commit, but not
reachable from any known-good commit.

So this means that when doing bisection, what we want to do is find the
point in git space that has _new_ "reachability" within that unknown space
that is as close to half that volume as space as possible. And that's
exactly what "git-rev-list --bisect" calculates.

So every time, we try to either move the "known bad" light-cone down in
time in the unknown space, _or_ we add a new "known good" light-cone. In
either case, the "unknown git space" keeps shrinking by half each time.

("by half" is not exact, because git space is not only quanticized, it
also has a rather strange "distance function". In other words, we're
talking about a rather strange space. The good news is that the space is
small enough that we can just enumerate every quantum and simply
calculate the volume it defines in that space. IOW, we do a very
brute-force thing, and it works fine).

Linus

2006-01-10 20:19:27

by Adrian Bunk

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On Sun, Jan 08, 2006 at 07:26:50PM -0800, Linus Torvalds wrote:
>...
> THIS is what "rebase" is for. It sounds like what you really want to do is
> not have a development branch at all, but you just want to track my tree
> and then keep track of a few branches of your own. In other words, you
> don't really have a "real" branch - you've got an odd collection of
> patches that you really want to carry around on top of _my_ branch. No?

Yes.

> Now, in this model, you're not really using git as a distributed system.
> In this model, you're using git to track somebody elses tree, and track a
> few patches on top of it, and then "git rebase" is a way to move the base
> that you're tracking your patches against forwards..

I am using the workaround of carrying the patches in a mail folder,
applying them in a batch, and not pulling from your tree between
applying a batch of patches and you pulling from my tree.

> It's also entirely possible that you may want to look at "stacked git"
> (stg), which is really more about a "quilt on top of git" approach. Which
> again, may or may not suit your needs better.
>...

After a quick look, stg seems to be an interesting project that might
suit my needs.

I'd say the main problem is that git with several other projects like
cogito and stg on top of it allow many different workflows. But finding
the one that suits one's needs without doing something in a wrong way
is non-trivial.

It might help if someone could write some kind of "Git for dummies" that
focusses only on the usage and advantages/disadvantages of user
interfaces like cogito, stg and (H)GCT and restricts the discussion of
git internals to a short paragraph in the introduction.

> Linus

cu
Adrian

--

"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed

2006-01-10 20:33:13

[permalink] [raw]

Subject: Re: git pull on Linux/ACPI release tree

On 1/11/06, Adrian Bunk <[email protected]> wrote:
> I am using the workaround of carrying the patches in a mail folder,
> applying them in a batch, and not pulling from your tree between
> applying a batch of patches and you pulling from my tree.

In that case, there's a mostly automated way of doing that if you read
the last couple lines of git-rebase, using something along the lines
of

git-format-patch <yours> <linus> | git-am -3 -k

> I'd say the main problem is that git with several other projects like
> cogito and stg on top of it allow many different workflows. But finding
> the one that suits one's needs without doing something in a wrong way
> is non-trivial.

You are right about that, but much of the space (of what workflows are
interesting) is still being explored, and git and the porcelains
reacting to people's interests. So it's still a moving target. A fast
moving target.

cheers,

martin

2006-01-10 20:36:32