2008-03-02 15:49:59

by Steve French

[permalink] [raw]
Subject: kernel.org git tree corrupt?

In preparation for a merge request I do
cg-diff -m -r origin: | diffstat -p1
(git diff -m -r origin: shows the same problem) and
cg-log -m -r origin:
from my kernel.org cifs-2.6.git tree against the linux-2.6.git which
should only show a small set of changes (typically only affecting one
directory in the kernel source), but instead now is showing thousands
of changesets with changed lines all over the tree. The origin is
/pub/scm/linux/kernel/git/torvalds/linux-2.6.git
and the only operation done on cifs-2.6.git is git pull (to merge in
changes from linux-2.6.git) and applying (patch and cg-commit) small
patches (to the fs/cifs directory typically). Even after doing git
pull again (and also git-fsck) that did not seem to help as there are
still thousands of changesets showing up when there should be fewer
than 30. Was the linux-2.6.git rebuilt in the last four weeks with
different changeset numbers? Something seems to have drastically
changed in the kernel.org tree in the last few weeks.

Any idea how to get around this ... short of going through all patches
to fs/cifs saving the uncommitted ones off one by one by hand,
recloning the tree and reapplying them one by one?
Doing "cg-log -m -r origin: fs/cifs" shows changesets that are already
in linux-2.6.git (e.g. the vfs-wide changes to add path_put
1d957f9bf87da74f420424d16ece005202bbebd3 and
4ac9137858e08a19f29feac4e1f4df7c268b0ba5 )

The oldest patch which affects fs/cifs which incorrectly shows up as
not in mainline (ie shows up with cg-diff but is actually merged into
both mainline and my tree, and submitted and merged by someone else)
is eebd2aa355692afaf9906f62118620f1a1c19dbb dated 2/4/2008 so I assume
that the problem started around then.

Ideas for another solution short of recloning and sorting through all
of the last month or two of patches that affect this directory by
hand?


--
Thanks,

Steve


2008-03-02 21:28:10

by Linus Torvalds

[permalink] [raw]
Subject: Re: kernel.org git tree corrupt?



On Sun, 2 Mar 2008, Steve French wrote:
>
> In preparation for a merge request I do
> cg-diff -m -r origin: | diffstat -p1
> (git diff -m -r origin: shows the same problem) and
> cg-log -m -r origin:

First off, you really should lay off the cogito thing, it's pretty much
guaranteed that any cogito usage will just be harder and less likely to be
correct than just usign native git (and almost nobody will be able to help
you any more - it's not like it's been maintained for the last year).

It *sounds* like your "origin" isn't what you expect it to be.

> Was the linux-2.6.git rebuilt in the last four weeks with different
> changeset numbers? Something seems to have drastically changed in the
> kernel.org tree in the last few weeks.

Nope, nothing has been rebuilt, but there's been another merge window, so
there's certainly been drastic changes.

> Doing "cg-log -m -r origin: fs/cifs" shows changesets that are already
> in linux-2.6.git (e.g. the vfs-wide changes to add path_put
> 1d957f9bf87da74f420424d16ece005202bbebd3 and
> 4ac9137858e08a19f29feac4e1f4df7c268b0ba5 )

In that case, your "origin" simply isn't updated, but points somewhere
long back in history. I can only assume that cogito has done something
wrong, like not been able to handle packed refs or something, and you have
an updated tree but "origin" pointing to way back in the history.

> Ideas for another solution short of recloning and sorting through all
> of the last month or two of patches that affect this directory by
> hand?

Do this:

- get rid of cogito (or at least make sure that it doesn't mess up the
following sequence)

- look at your .git/config file to make sure that the remote branches are
sane. It should have something like this (and if it doesn't, add it):

[remote "origin"]
url = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
fetch = +refs/heads/*:refs/remotes/origin/*

[branch "master"]
remote = origin
merge = refs/heads/master

- look at your "origin" branch, and make sure it's a *remote* branch, not
the old-style local one. Ie it should *not* show up when you do a
plain

git branch

but it *should* show up (as both "origin/HEAD" and "origin/master")
when you do

git branch -a

and if it's a local branch (very possible, since you're using cogito
that may not even understand remote branches), then just blow it away
entirely with "git branch -D origin".

- do "git fetch" to make sure that the remote branches are up-to-date

- now do "git log origin" an it should show something recent like

commit 038f2f725503b55ab76cfd2645915a85594710fe
Merge: 8d07a67... 855c603...
Author: Linus Torvalds <[email protected]>
Date: Sun Mar 2 12:38:17 2008 -0800

Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ieee1394/linux139
...

- and if it does, now you should be all good, and you can dothe normal
"gitk origin.." and see the commits that are in your tree but not in
mine.

Now your setup should be ok again. I *suspect* that what has happened is
that you got a local "origin" branch - probably through cogito - that hid
the remote one.

Linus

2008-03-02 21:40:30

by Martin Langhoff

[permalink] [raw]
Subject: Re: kernel.org git tree corrupt?

On Mon, Mar 3, 2008 at 10:16 AM, Linus Torvalds
<[email protected]> wrote:
> In that case, your "origin" simply isn't updated, but points somewhere
> long back in history. I can only assume that cogito has done something
> wrong, like not been able to handle packed refs or something, and you have
> an updated tree but "origin" pointing to way back in the history.

Strange - cogito over git protocol will use git itself to peek at the
references. Steve, what is your git version?

> > Ideas for another solution short of recloning and sorting through all
> > of the last month or two of patches that affect this directory by
> > hand?

Linus' plan assumes you have a .git/config file. I don't think a
cogito-based checkout follows such modern conventions - here's an
alternative plan:

- make sure you have a current git

- get a fresh clone of linux-2.6, and in there

# tell it about your old checkout
$ git remote add oldlinux /path/to/your/older/linux/checkout/.git
$ git fetch oldlinux
# visualise what's in there
$ gitk origin/master oldlinux/master

# you may want to merge your old branch
$ git merge oldlinux/master

hth,


martin

2008-03-02 22:02:46

by Steve French

[permalink] [raw]
Subject: Re: kernel.org git tree corrupt?

On Sun, Mar 2, 2008 at 3:40 PM, Martin Langhoff
<[email protected]> wrote:
> On Mon, Mar 3, 2008 at 10:16 AM, Linus Torvalds
> <[email protected]> wrote:
> > In that case, your "origin" simply isn't updated, but points somewhere
> > long back in history. I can only assume that cogito has done something
> > wrong, like not been able to handle packed refs or something, and you have
> > an updated tree but "origin" pointing to way back in the history.
>
> Strange - cogito over git protocol will use git itself to peek at the
> references. Steve, what is your git version?

This is master.kernel.org so git version is 1.5.4.3

--
Thanks,

Steve

2008-03-03 03:43:52

by Steve French

[permalink] [raw]
Subject: Re: kernel.org git tree corrupt?

On Sun, Mar 2, 2008 at 3:16 PM, Linus Torvalds
<[email protected]> wrote:
>
> On Sun, 2 Mar 2008, Steve French wrote:
> >
> > In preparation for a merge request I do
> > cg-diff -m -r origin: | diffstat -p1
> > (git diff -m -r origin: shows the same problem) and
> > cg-log -m -r origin:
>
> First off, you really should lay off the cogito thing
no problem

> - look at your .git/config file to make sure that the remote branches are
> sane. It should have something like this (and if it doesn't, add it):
>
> [remote "origin"]
> url = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
> fetch = +refs/heads/*:refs/remotes/origin/*
>
> [branch "master"]
> remote = origin
> merge = refs/heads/master
.git/config now includes

[core]
repositoryformatversion = 0
filemode = true
bare = false
logallrefupdates = true
[remote "origin"]
url = /pub/scm/linux/kernel/git/torvalds/linux-2.6.git
fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
remote = origin
merge = refs/heads/master


> - look at your "origin" branch, and make sure it's a *remote* branch, not
> the old-style local one. Ie it should *not* show up when you do a
> plain
>
> git branch
now only shows "* master"

> but it *should* show up (as both "origin/HEAD" and "origin/master")
> when you do
>
> git branch -a
Now "git branch -a" shows
* master
origin/master

It is missing "origin/HEAD"
> - now do "git log origin" an it should show something recent
git log origin and git log origin/master both return
"ambiguous argument 'origin': unknown revision"

I must be misunderstanding the syntax/manpage of git-log but "git diff
-m -r origin/master"
now works fine and displays exactly what I expect so things have improved.



> Now your setup should be ok again. I *suspect* that what has happened is
> that you got a local "origin" branch - probably through cogito - that hid
> the remote one.
This seems plausible and


--
Thanks,

Steve

2008-03-03 16:22:22

by Linus Torvalds

[permalink] [raw]
Subject: Re: kernel.org git tree corrupt?



On Sun, 2 Mar 2008, Steve French wrote:
>
> > - look at your "origin" branch, and make sure it's a *remote* branch, not
> > the old-style local one. Ie it should *not* show up when you do a
> > plain
> >
> > git branch
> now only shows "* master"
>
> > but it *should* show up (as both "origin/HEAD" and "origin/master")
> > when you do
> >
> > git branch -a
> Now "git branch -a" shows
> * master
> origin/master
>
> It is missing "origin/HEAD"

Ahh, yeah, my bad. The origin/HEAD thing will be created if you use either
clone or "git remote add -m master" to create the remote. But when I asked
you to do it by just editing the config file, you now have to create that
HEAD pointer manually too.

You can do

git symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/master

to create the thing (it just creates a symbolic ref from the origin/HEAD
remote ref to the origin/master one, so now git will know that when you
talk about 'origin', it is supposed to just use the master branch of
that remote).

> > - now do "git log origin" an it should show something recent
> git log origin and git log origin/master both return
> "ambiguous argument 'origin': unknown revision"

Well, origin/master should have worked, but the fact that plain "origin"
didn't work is due to exactly the lack of HEAD file for that remote branch
(for your kernel tree, there's only one remote branch, so it may be
"obvious" that origin must be talking about the master branch, but if
there are multiple branches at the origin it's not obvious *which* branch
should be considered the default one, which is why we use HEAD)

> I must be misunderstanding the syntax/manpage of git-log but "git diff
> -m -r origin/master"
> now works fine and displays exactly what I expect so things have improved.

Well, if "origin/master" works in that situation, then "origin/master"
should have worked in "git log" too, so maybe you mistyped?

Anyway, with the HEAD link added for the remote 'origin', you should be
able to use plain 'origin' like you're used to.

Linus

2008-03-04 04:42:47

by Steve French

[permalink] [raw]
Subject: Re: kernel.org git tree corrupt?

On Mon, Mar 3, 2008 at 10:21 AM, Linus Torvalds
<[email protected]> wrote:
>
> On Sun, 2 Mar 2008, Steve French wrote:
> > Now "git branch -a" shows
> > * master
> > origin/master
> >
> > It is missing "origin/HEAD"
>
> Ahh, yeah, my bad. The origin/HEAD thing will be created if you use either
> clone or "git remote add -m master" to create the remote. But when I asked
> you to do it by just editing the config file, you now have to create that
> HEAD pointer manually too.
>
> You can do
>
> git symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/master
>
> to create the thing (it just creates a symbolic ref from the origin/HEAD
Worked.

Shaggy pointed me to the git-request-pull script which is
easier than the previous scripts I used and now works fine on
the cifs-2.6.git tree so will send merge request.


--
Thanks,

Steve