2002-03-21 06:14:43

by Martin Blais

[permalink] [raw]
Subject: xxdiff as a visual diff tool (shameless plug)

Hello gang.

(I always thought you guys must be visual-diff hungry, with the
large amount patching-by-hand that you must do, and I knew I'd
send this email one day, just never got to it. I've witnessed
the latest KT with the discussion about SCM systems, and in
particular, the discussion about the graphical merge tools, so
now is the time or never.)

I was wondering if you've tried my tool xxdiff (shameless
plug), it has the following features:

* 2-way and 3-way file comparisons, with LOTS of conveniences
for merging and policing (selecting hunks, reordering, fast
"save as merged" button);

* highlighting of horizontal diffs (once you try that for
merge reviews, you can never go back); multiple horizontal
diff hunks are supported as well, i.e. if you added two
words on a line, sufficiently separated, it highlights the
two words separately;

* can show the resulting merged file from the selections (in
a separate window, currently dev. feature is a classic
3-pane view);

* can "unmerge" CVS conflicts and display them as if they
were two files;

* uses whatever external diff program you prefer (I used to
use cleardiff which sometimes does a better does than GNU
diff)

* fully customizable with an .xxdiffrc file

It depends only on troltech's qt, so you don't need a desktop
environment to use it. I've introduced it at my workplace and
at least 50 people using it daily in a commercial development
environment for more than 2 years now, people using it mostly
for merge reviews and merging, so it has become fairly
stable. And quite many users have been replacing their vendor's
visual diff with it.

Check it out at http://xxdiff.sourceforge.net

(I actually bothered writing documentation as well)

It does not do edits at this point because I felt everyone has
their own strong preferences for editing files. However, I may
in the future support very simple editing of hunks. For now,
the recommended way to edit is to leave some hunks unselected
and those will be output in the merged file with markers
similar to CVS conflicts. You can then edit the files. Also,
what I often do is leave it running, then edit one of the
original files and then press C-r, and it redoes the diff.


I'd be delighted to hear any ideas on features you think are
missing, incorrect or needed for integration with BitKeeper and
otherwise, and also what you would like to see in it in the
future (I could prioritize development). Check out the TODO
file for details (from the web site).

With the large amount of parallel development going on in the
kernel, some of you guys must be experts at merging patches,
and I'm sure you'll come up with tons of cool ideas. I'm
actively supporting and developing this tool, so I will
consider suggestions seriously, although I have limited amount
of nighttime to develop this.


I was already contemplating adding support for patch input
(mentioned in KT) and output (I think I might be able to hack
this quickly in the meantime with a wrapper script).


> Andrew said that tkdiff was already quite good at this, and
> might be something to merge into the project. But he added,
> "The problem I find is that I often want to take (file1+patch)
> -> file2, when I don't have file1. But merge tools want to take
> (file1|file2) -> file3. I haven't seen a graphical tool which
> helps you to wiggle a patch into a file."

I actually wanted to implement it exactly like this in xxdiff,
and I think it may not be too hard. Something like

"xxdiff --patch file1 < patch"

and it would display as two files, and allow you to save merged
results. That was the plan (read more below).

I wanted to spawn a patch command on it and recuperate the
output and then run diff and display that, so one could use and
alternate patch program as well, and I would reuse patch's
heuristics automatically (e.g. i don't have to code it myself).


> Neil Brown agreed with this, and said, "I would like a tool
> (actually an emacs mode) that would show me exactly why a patch
> fails, and allow me to edit bits until it fits, and then apply
> it. I assume that is what you mean by "wiggle a patch into a
> file"." Pavel Machek also thought this would be a rerally great
> thing.

The part I haven't figured out a nice solution for yet, is for
the failed hunks... how to display them sensibly. I was
thinking of using the 3-file display with the third file
showing only the failed hunks in the approximate location where
they were supposed to be. Not sure how to do this yet. Any
ideas welcome.

Cheers,



--
M.

PS. Please CC replies to me, I don't follow the list itself,
just the KT summaries.


2002-03-21 10:43:34

by Tim Waugh

[permalink] [raw]
Subject: Re: xxdiff as a visual diff tool (shameless plug)

On Thu, Mar 21, 2002 at 01:13:32AM -0500, Martin Blais wrote:

> It does not do edits at this point because I felt everyone has
> their own strong preferences for editing files.

Incidentally, on a related subject: editdiff (from patchutils) can do
simple hunk editing. Use emacs or vi to hand-edit a patch, and it
will fix up offsets and counts for you.

Tim.
*/

2002-03-22 10:14:36

by Pavel Machek

[permalink] [raw]
Subject: Re: xxdiff as a visual diff tool (shameless plug)

Hi!

> I actually wanted to implement it exactly like this in xxdiff,
> and I think it may not be too hard. Something like
>
> "xxdiff --patch file1 < patch"
>
> and it would display as two files, and allow you to save merged
> results. That was the plan (read more below).
>
> I wanted to spawn a patch command on it and recuperate the
> output and then run diff and display that, so one could use and
> alternate patch program as well, and I would reuse patch's
> heuristics automatically (e.g. i don't have to code it myself).

... but ...

> > Neil Brown agreed with this, and said, "I would like a tool
> > (actually an emacs mode) that would show me exactly why a patch
> > fails, and allow me to edit bits until it fits, and then apply
> > it. I assume that is what you mean by "wiggle a patch into a
> > file"." Pavel Machek also thought this would be a rerally great
> > thing.
>
> The part I haven't figured out a nice solution for yet, is for
> the failed hunks... how to display them sensibly. I was
> thinking of using the 3-file display with the third file
> showing only the failed hunks in the approximate location where
> they were supposed to be. Not sure how to do this yet. Any
> ideas welcome.

It would be great to somehow split patches before feeding them to the
patch. If you have one big hunk, and it fails because of one letter
added somewhere in file, it is *big pain* to find/kill offending
letter.

Pavel
--
(about SSSCA) "I don't say this lightly. However, I really think that the U.S.
no longer is classifiable as a democracy, but rather as a plutocracy." --hpa

2002-03-22 10:30:08

by Andrew Morton

[permalink] [raw]
Subject: Re: xxdiff as a visual diff tool (shameless plug)

Pavel Machek wrote:
>
> It would be great to somehow split patches before feeding them to the
> patch. If you have one big hunk, and it fails because of one letter
> added somewhere in file, it is *big pain* to find/kill offending
> letter.

yup. It's a *lot* easier if the diff was generated with `diff -1',
because that makes many more hunks.

There is sufficient information in a normal diff to turn it into
a `diff -1' diff, I think. All it needs is for Tim^Wsomeone to code it.

-

2002-03-22 18:24:50

by Martin Blais

[permalink] [raw]
Subject: Re: xxdiff as a visual diff tool (shameless plug)

On Friday 22 March 2002 04:27, Pavel Machek wrote:
> Hi!
>
> > I actually wanted to implement it exactly like this in xxdiff,
> > and I think it may not be too hard. Something like
> >
> > "xxdiff --patch file1 < patch"
> >
> > and it would display as two files, and allow you to save merged
> > results. That was the plan (read more below).
> >
> > I wanted to spawn a patch command on it and recuperate the
> > output and then run diff and display that, so one could use and
> > alternate patch program as well, and I would reuse patch's
> > heuristics automatically (e.g. i don't have to code it myself).
>
> ... but ...
>
> > > Neil Brown agreed with this, and said, "I would like a tool
> > > (actually an emacs mode) that would show me exactly why a patch
> > > fails, and allow me to edit bits until it fits, and then apply
> > > it. I assume that is what you mean by "wiggle a patch into a
> > > file"." Pavel Machek also thought this would be a rerally great
> > > thing.
> >
> > The part I haven't figured out a nice solution for yet, is for
> > the failed hunks... how to display them sensibly. I was
> > thinking of using the 3-file display with the third file
> > showing only the failed hunks in the approximate location where
> > they were supposed to be. Not sure how to do this yet. Any
> > ideas welcome.
>
> It would be great to somehow split patches before feeding them to the
> patch. If you have one big hunk, and it fails because of one letter
> added somewhere in file, it is *big pain* to find/kill offending
> letter.

i though patch did that by itself, by attempting to merge per-hunk, and that
it saved all the failed merges in a reject file (file.rej). or perhaps
i'm missing something. isn't it the case?

(note that i don't have much experience using patch itself)

2002-03-22 18:30:02

by Martin Blais

[permalink] [raw]
Subject: Re: xxdiff as a visual diff tool (shameless plug)

On Friday 22 March 2002 04:27, Pavel Machek wrote:
> Hi!
>
> It would be great to somehow split patches before feeding them to the
> patch. If you have one big hunk, and it fails because of one letter
> added somewhere in file, it is *big pain* to find/kill offending
> letter.

oops.. sorry Pavel, never mind previous email, i got it now (brain is slowly
booting this morning).

that seems more like a patch problem/improvement request. i wouldn't do the
patch myself... however, with the rejected hunks problem, i wonder if it is
at all possible to avoid implementing patch functionality in the diffing tool
itself.

2002-03-22 21:44:49

by Pavel Machek

[permalink] [raw]
Subject: Re: xxdiff as a visual diff tool (shameless plug)

Hi!

> > It would be great to somehow split patches before feeding them to the
> > patch. If you have one big hunk, and it fails because of one letter
> > added somewhere in file, it is *big pain* to find/kill offending
> > letter.
>
> oops.. sorry Pavel, never mind previous email, i got it now (brain is slowly
> booting this morning).

Problem is that sometimes even one hunk is too much.

> that seems more like a patch problem/improvement request. i wouldn't do the
> patch myself... however, with the rejected hunks problem, i wonder if it is
> at all possible to avoid implementing patch functionality in the diffing tool
> itself.

Question is how to do it in patch. Even one *long line* can be too
much, and then your horizontal highlight would come very handy.

Pavel
--
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

2002-03-23 20:48:33

by Martin Blais

[permalink] [raw]
Subject: Re: xxdiff as a visual diff tool (shameless plug)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Friday 22 March 2002 16:44, Pavel Machek wrote:
> Hi!
> > that seems more like a patch problem/improvement request. i wouldn't do
> > the patch myself... however, with the rejected hunks problem, i wonder if
> > it is at all possible to avoid implementing patch functionality in the
> > diffing tool itself.
>
> Question is how to do it in patch. Even one *long line* can be too
> much, and then your horizontal highlight would come very handy.

actually, thinking more about it, having to use patch is not really a problem
for the BK use case discussed on this list, as BK most likely can provide the
file on the main branch and the common ancestor between the developer's
branch and the main branch. one then just needs to spawn a 3-way xxdiff on
those three files (with automatic selection of non-conflictual hunks).
that's what i do with Clearcase, where all the file versions are available
directly in the filesystem. CVS however, requires that i first fetch the file
to a temporary copy and there is no way to figure out the common ancestor
where the last update to the main branch occured (unless you hack it into a
tag or something).

my point is, if you're using scm, you don't need to use patches at all, so
there is no real need for a tool to help "wiggle patches in", in that
context.
-----BEGIN PGP SIGNATURE-----
Comment: For info see http://www.gnupg.org

iEYEARECAAYFAjyc6coACgkQq2PmC9F3Xx3LMACeJbrnBgPsFxSGuXlL8PdCdlmm
z2wAn2qNiFC6sgpz4yQZKMGV1DzIeayx
=rB+O
-----END PGP SIGNATURE-----

2002-03-23 23:39:42

by Pavel Machek

[permalink] [raw]
Subject: Re: xxdiff as a visual diff tool (shameless plug)

Hi!

> > > that seems more like a patch problem/improvement request. i wouldn't do
> > > the patch myself... however, with the rejected hunks problem, i wonder if
> > > it is at all possible to avoid implementing patch functionality in the
> > > diffing tool itself.
> >
> > Question is how to do it in patch. Even one *long line* can be too
> > much, and then your horizontal highlight would come very handy.
>
> actually, thinking more about it, having to use patch is not really a problem
> for the BK use case discussed on this list, as BK most likely can
> provide the

I'm not using bitkeeper. RMS told me not to use it, Larry told me not
to use it, Davem told me not to use it... So I'm not using it ;-).

So it still would be nice to have some scripts for automatic script
applying.

And yes, I usually do have common ancestor (namely clean version of
kernel I'm trying to patch.)

Pavel
--
Casualities in World Trade Center: ~3k dead inside the building,
cryptography in U.S.A. and free speech in Czech Republic.

2002-03-25 06:55:30

by Paul Mackerras

[permalink] [raw]
Subject: Re: xxdiff as a visual diff tool (shameless plug)

Martin Blais writes:

> I was wondering if you've tried my tool xxdiff (shameless
> plug), it has the following features:

While we're in shameless plug mode, anyone who is working with two or
more kernel trees should have a look at dirdiff. The great thing
about dirdiff is that it compares from 2 to 5 directory trees and
shows you which files differ between the trees in a concise manner.
You can then select one of the files and display the differences for
that file between any two of the versions. You can also select
multiple files and then copy them from one tree to another.

I use this when Linus or Marcelo puts out a new patch. Using dirdiff
I can get a quick view of which files have been changed, added or
deleted. I can then quickly copy over anything where I don't need to
look too closely (e.g. stuff under fs or arch/ia64, for instance) and
have a close look at what changes have been made in other areas
(e.g. under arch/ppc, arch/i386 or mm).

In fact, I find a 3-way dirdiff useful in that situation - comparing
the previous Linus/Marcelo tree, the current Linus/Marcelo tree and my
tree. Different states of the file (such as modified in my tree and
unchanged in the Linus/Marcelo trees, unmodified in my tree and
modified in the latest Linus/Marcelo release, modified in both, etc.)
show up as different color patterns which are easily recognized.

Dirdiff displays an array of colored squares with one column per
directory tree and one line per file. For a given file, versions that
are the same get the same color. Each group of identical files gets
given a color ranging from green (newest) to red (oldest) based on the
most recent modification time of the files in the group. (In fact the
only reason why dirdiff is limited to 5 trees is that I thought that
any more than 5 colors in the green - red spectrum would not be
sufficiently visually distinct.)

Dirdiff also has features to make it easy to use in conjunction with
Bitkeeper or CVS, for example you can get it to ignore differences in
BK or CVS tags when deciding whether two files go in the same group.

The differences get displayed in a separate window as a diff -u but
with different colored backgrounds instead of - or + at the beginning
of the line. You can then do a merge; each line of difference has a
little checkbox, and if you check the checkbox, that line will be
changed in the merged file. The merged file is brought up in a window
which allows basic editing (insert, delete, cut/copy/paste, find)
before you either save the merged file or discard the merge.

Dirdiff is available from ftp://samba.org/pub/paulus/dirdiff-1.5.tar.gz
and it's also a debian package (although I hear the debian version is
a little out of date).

Paul.