LinuxLists.cc - 2.6.19 -mm merge plans

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Andrew Morton wrote:
> kerneldoc-error-on-ata_piixc.patch
> libata-add-40pin-short-cable-support-honour-drive.patch
> libata-add-40pin-short-cable-support-honour-drive-fix.patch
> 1-of-2-jmicron-driver.patch
> 1-of-2-jmicron-driver-fix.patch
> 2-of-2-jmicron-driver-plumbing-and-quirk.patch
> 2-of-2-jmicron-driver-plumbing-and-quirk-cleanup.patch
> non-libata-driver-for-jmicron-devices.patch
> via-pata-controller-xfer-fixes.patch
> via-pata-controller-xfer-fixes-fix.patch
> via-sata-oops-on-init.patch
>
> ATA stuff. I am hopelessly confused regarding which patches pertain to
> sata, which to pata and which to legacy IDE. It's a matter of weeding
> through all of these and finding an appropriate route to get them merged.

Should be quite easy, as its all based on path.

drivers/{ata,scsi} -> me
drivers/ide -> not me (Alan?)

Jeff

2006-09-20 21:38:33

by Badari Pulavarty

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Wed, 2006-09-20 at 13:54 -0700, Andrew Morton wrote:

Sorry, I didn't get back to this earlier. I have been chasing
one more ext2 & ext3 regression (this time with random reads).
Anyway ..

> add-address_space_operationsbatch_write.patch
> add-address_space_operationsbatch_write-fix.patch
> pass-io-size-to-batch_write-address-space-operation.patch
>
> These add a new address_space operation. For reiser4, with potential for
> use by other filesystems.
>
> Problem is, 2.6.18 has a significant writev() performace regression on NFS
> and probably on other filesystems. Because 2.6.18 does
> prepare_write/commit_write for each iovec segment. We want to go back to
> copying mulitple iovec segments within a single prepare_write/commit_write.
>
> Plus there's still the possible deadlock in our standard write() function
> (thw thing which fault_in_pages_readable() tries to prevent).
>
> All of this should be fixed.

What needs to be fixed here ?

Thanks,
Badari

2006-09-20 21:54:00

by Jiri Kosina

[permalink] [raw]

Subject: USB: fix autosuspend-autoresume with CONFIGRe: 2.6.19 -mm merge plans

On Wed, 20 Sep 2006, Andrew Morton wrote:

> fix-gregkh-usb-usbcore-add-autosuspend-autoresume-infrastructure.patch
> gregkh-usb-usbcore-add-autosuspend-autoresume-infrastructure-2.patch

Hi Andrew,

a few days ago I submitted a patch [1] to autosupend-autoresume
infrastructure (and Alan Stern submitted a similar patch a few hours later
[2]).

Without this one-liner, all kernels compiled without CONFIG_USB_SUSPEND
will be unable to perform more than one suspend/resume cycle, which is
quite annoying. Would you please reconsider pushing these together with
other autosuspend/autoresume infrastructure fixes?

Thanks.

[1] http://lkml.org/lkml/2006/9/18/290
[2] http://lkml.org/lkml/2006/9/19/93

--
Jiri Kosina

2006-09-20 21:55:41

by Trond Myklebust

[permalink] [raw]

Subject: Re: Autofs4 breakage (was 2.6.19 -mm merge plans)

On Wed, 2006-09-20 at 13:54 -0700, Andrew Morton wrote:

> add-newline-to-nfs-dprintk.patch
> fs-nfs-make-code-static.patch
>
> NFS queue -> Trond.
>
> The NFS git tree breaks autofs4 submounts. Still.

I still suspect that is due to a misconfigured selinux setup on your
machine. If autofs4 expects to be able to do mkdir() on your NFS
partition (something which in itself is wrong), then selinux should be
configured to allow it to do so.

Anyhow, does reverting the patch

http://kernel.org/git/gitweb.cgi?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=a634904a7de0d3a0bc606f608007a34e8c05bfee;hp=ddeff520f02b92128132c282c350fa72afffb84a

'fix' the issue for you?

Cheers,
Trond

2006-09-20 22:10:13

[permalink] [raw]

Subject: Re: USB: fix autosuspend-autoresume with CONFIGRe: 2.6.19 -mm merge plans

On Wed, 20 Sep 2006 23:53:23 +0200 (CEST)
Jiri Kosina <[email protected]> wrote:

> On Wed, 20 Sep 2006, Andrew Morton wrote:
>
> > fix-gregkh-usb-usbcore-add-autosuspend-autoresume-infrastructure.patch
> > gregkh-usb-usbcore-add-autosuspend-autoresume-infrastructure-2.patch
>
> Hi Andrew,
>
> a few days ago I submitted a patch [1] to autosupend-autoresume
> infrastructure (and Alan Stern submitted a similar patch a few hours later
> [2]).
>
> Without this one-liner, all kernels compiled without CONFIG_USB_SUSPEND
> will be unable to perform more than one suspend/resume cycle, which is
> quite annoying. Would you please reconsider pushing these together with
> other autosuspend/autoresume infrastructure fixes?
>
> Thanks.
>
> [1] http://lkml.org/lkml/2006/9/18/290
> [2] http://lkml.org/lkml/2006/9/19/93

I expect the appropriate fixes will automagically appear in Greg's tree, to
be picked up in next -mm. Perhaps they already have appeared - Alan?

2006-09-20 22:13:52

[permalink] [raw]

Subject: Re: Autofs4 breakage (was 2.6.19 -mm merge plans)

On Wed, 20 Sep 2006 17:55:33 -0400
Trond Myklebust <[email protected]> wrote:

> On Wed, 2006-09-20 at 13:54 -0700, Andrew Morton wrote:
>
> > add-newline-to-nfs-dprintk.patch
> > fs-nfs-make-code-static.patch
> >
> > NFS queue -> Trond.
> >
> > The NFS git tree breaks autofs4 submounts. Still.
>
> I still suspect that is due to a misconfigured selinux setup on your
> machine.

"still"? I don't recall being told that. Perhaps I was asleep.

It's an up-to-date-a-few-weeks-ago FC5 machine. So if I'm busted then lots
of people are.

> If autofs4 expects to be able to do mkdir() on your NFS
> partition (something which in itself is wrong), then selinux should be
> configured to allow it to do so.
>
> Anyhow, does reverting the patch
>
> http://kernel.org/git/gitweb.cgi?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=a634904a7de0d3a0bc606f608007a34e8c05bfee;hp=ddeff520f02b92128132c282c350fa72afffb84a
>
> 'fix' the issue for you?
>

I'll take a look this evening.

2006-09-20 22:41:18

by Greg KH

[permalink] [raw]

Subject: Re: USB: fix autosuspend-autoresume with CONFIGRe: 2.6.19 -mm merge plans

On Wed, Sep 20, 2006 at 03:09:18PM -0700, Andrew Morton wrote:
> On Wed, 20 Sep 2006 23:53:23 +0200 (CEST)
> Jiri Kosina <[email protected]> wrote:
>
> > On Wed, 20 Sep 2006, Andrew Morton wrote:
> >
> > > fix-gregkh-usb-usbcore-add-autosuspend-autoresume-infrastructure.patch
> > > gregkh-usb-usbcore-add-autosuspend-autoresume-infrastructure-2.patch
> >
> > Hi Andrew,
> >
> > a few days ago I submitted a patch [1] to autosupend-autoresume
> > infrastructure (and Alan Stern submitted a similar patch a few hours later
> > [2]).
> >
> > Without this one-liner, all kernels compiled without CONFIG_USB_SUSPEND
> > will be unable to perform more than one suspend/resume cycle, which is
> > quite annoying. Would you please reconsider pushing these together with
> > other autosuspend/autoresume infrastructure fixes?
> >
> > Thanks.
> >
> > [1] http://lkml.org/lkml/2006/9/18/290
> > [2] http://lkml.org/lkml/2006/9/19/93
>
> I expect the appropriate fixes will automagically appear in Greg's tree, to
> be picked up in next -mm. Perhaps they already have appeared - Alan?

I just added the fix to my tree, am catching up on my pending patch
queue right now...

thanks,

greg k-h

2006-09-21 00:10:50

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

> introduce-config_zone_dma.patch
> optional-zone_dma-in-the-vm.patch
> optional-zone_dma-in-the-vm-tidy.patch
> optional-zone_dma-for-i386.patch
> optional-zone_dma-for-x86_64.patch
> optional-zone_dma-for-ia64.patch
> remove-zone_dma-remains-from-parisc.patch
> remove-zone_dma-remains-from-sh-sh64.patch

Would it not make sense to define what ZONE_DMA actually means
consistently before trying to change it? The current mess across
different architectures seems like a disaster area to me.

What DOES requesting ZONE_DMA from a driver actually mean?

M.

2006-09-21 00:23:04

[permalink] [raw]

Subject: ZONE_DMA (was: Re: 2.6.19 -mm merge plans)

(Subject rewritten, developer cc'ed, thwap delivered)

On Wed, 20 Sep 2006 17:09:57 -0700
"Martin J. Bligh" <[email protected]> wrote:

>
> > introduce-config_zone_dma.patch
> > optional-zone_dma-in-the-vm.patch
> > optional-zone_dma-in-the-vm-tidy.patch
> > optional-zone_dma-for-i386.patch
> > optional-zone_dma-for-x86_64.patch
> > optional-zone_dma-for-ia64.patch
> > remove-zone_dma-remains-from-parisc.patch
> > remove-zone_dma-remains-from-sh-sh64.patch
>
> Would it not make sense to define what ZONE_DMA actually means
> consistently before trying to change it? The current mess across
> different architectures seems like a disaster area to me.
>
> What DOES requesting ZONE_DMA from a driver actually mean?
>

AFAIK it means "floppy disks" ;)

My concern about these patches is that they'll only be useful for
self-compiled kernels, because distros will be forced to enable ZONE_DMA
for evermore anyway.

If that's correct then perhaps we should drop these patches, because they
will serve to weaken ongoing testing.

2006-09-21 00:32:06

[permalink] [raw]

Subject: Re: ZONE_DMA (was: Re: 2.6.19 -mm merge plans)

On Wed, 20 Sep 2006, Andrew Morton wrote:

> > Would it not make sense to define what ZONE_DMA actually means
> > consistently before trying to change it? The current mess across
> > different architectures seems like a disaster area to me.
> >
> > What DOES requesting ZONE_DMA from a driver actually mean?

ZONE_DMA is a memory area that is needed by an arch for devices that
cannot do DMA to all of memory. The high boundary is set by
MAX_DMA_ADDRESS.

> My concern about these patches is that they'll only be useful for
> self-compiled kernels, because distros will be forced to enable ZONE_DMA
> for evermore anyway.

We already have 4 arches now that do not need ZONE_DMA at all.

ZONE_DMA does not have a bright future with IOMMUs and other things
around. None of my system uses ZONE_DMA and I have a variety of them.

And yes if we do not have this facility in the kernel then distros cannot
pick it up. At least on IA64 I know that hardware from the major vendors
has not been needing ZONE_DMA for a while now.

Also ZONE_DMA is a very bad concept. Multiple drivers may have different
address requirements. What we need is some way for a driver to tell the
kernel what the required range of addresses is. If a device is only
capable of handling 30 valid address bits then we may have to use
ZONE_DMA and only allow the use of the lower 16MB. It would be better to
develop a different mechanism.

2006-09-21 00:40:25

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Wed, 20 Sep 2006, Martin J. Bligh wrote:

> Would it not make sense to define what ZONE_DMA actually means
> consistently before trying to change it? The current mess across
> different architectures seems like a disaster area to me.

Actually the desaster is cleaned up by this patch. A couple of
architectures that were wrongly using ZONE_DMA now use ZONE_NORMAL.

2006-09-21 00:40:20

by Alan

[permalink] [raw]

Subject: Re: ZONE_DMA (was: Re: 2.6.19 -mm merge plans)

Ar Mer, 2006-09-20 am 17:31 -0700, ysgrifennodd Christoph Lameter:
> ZONE_DMA does not have a bright future with IOMMUs and other things
> around. None of my system uses ZONE_DMA and I have a variety of them.

IOMMUs don't always help. They IOMMU aperture on AMD64 for example is
too high to help devices with below 32bit limits.

2006-09-21 00:51:12

[permalink] [raw]

Subject: Re: ZONE_DMA

Andrew Morton wrote:
> (Subject rewritten, developer cc'ed, thwap delivered)
>
> On Wed, 20 Sep 2006 17:09:57 -0700
> "Martin J. Bligh" <[email protected]> wrote:
>
>>> introduce-config_zone_dma.patch
>>> optional-zone_dma-in-the-vm.patch
>>> optional-zone_dma-in-the-vm-tidy.patch
>>> optional-zone_dma-for-i386.patch
>>> optional-zone_dma-for-x86_64.patch
>>> optional-zone_dma-for-ia64.patch
>>> remove-zone_dma-remains-from-parisc.patch
>>> remove-zone_dma-remains-from-sh-sh64.patch
>> Would it not make sense to define what ZONE_DMA actually means
>> consistently before trying to change it? The current mess across
>> different architectures seems like a disaster area to me.
>>
>> What DOES requesting ZONE_DMA from a driver actually mean?
>>
>
> AFAIK it means "floppy disks" ;)

That's the problem - it doesn't mean that at all, except on ia32.
It means totally different things on different architectures. IIRC,
on PPC64, it's all of memory.

Having something that's used in generic code that means random
things on different arches just seems like a recipe for disaster
to me.

> My concern about these patches is that they'll only be useful for
> self-compiled kernels, because distros will be forced to enable ZONE_DMA
> for evermore anyway.
>
> If that's correct then perhaps we should drop these patches, because they
> will serve to weaken ongoing testing.

Well, I think we actually need to define what it means before we
mess with it any more. It's not Christoph's fault that it's broken.
But messing with something that's pretty much undefined will likely
have undefined consequences.

Christoph Lameter wrote:
> Actually the desaster is cleaned up by this patch. A couple of
> architectures that were wrongly using ZONE_DMA now use ZONE_NORMAL.

OK ... but requesting ZONE_DMA means what? DMAable for which device?
Is it always a floppy disk? on some platforms a PCI card? And how
is the VM meant to know what the device is capable of anyway?

Having an arch-specific definition of the limit is arbitrary and
useless, is it not? The limit is imposed by the device and its
driver, we're not communicating it into any sensible way into the
VM code, AFAICS. Unless we're pretending we never call it from
generic code, which seems woefully unlikely to me.

Are we redefining ZONE_DMA to always be 16MB limit across all
architectures? At least that'd be consistent.

M.

2006-09-21 01:10:39

[permalink] [raw]

Subject: Re: ZONE_DMA

On Wed, 20 Sep 2006, Martin J. Bligh wrote:

> Having something that's used in generic code that means random
> things on different arches just seems like a recipe for disaster
> to me.

ZONE_DMA is only used as ZONE_NORMAL if the architecture does not
need ZONE_NORMAL because all of memory is reachable via DMA.

> OK ... but requesting ZONE_DMA means what? DMAable for which device?
> Is it always a floppy disk? on some platforms a PCI card? And how
> is the VM meant to know what the device is capable of anyway?

I already explained that twice to you. I think we all agree that the
situation could be better.

> Having an arch-specific definition of the limit is arbitrary and
> useless, is it not? The limit is imposed by the device and its
> driver, we're not communicating it into any sensible way into the
> VM code, AFAICS. Unless we're pretending we never call it from
> generic code, which seems woefully unlikely to me.

Its bad but its not useless. See how various arches use it.

> Are we redefining ZONE_DMA to always be 16MB limit across all
> architectures? At least that'd be consistent.

That wont work because many architectures use different limits. Maybe you
should once in a while have a look at the sources.

The patchset leaves the semantics of ZONE_DMA as memory beyond
MAX_DMA_ADDRESS as is. Nothing should break. It only allows us to opt out
of this scheme if we do not need it.

2006-09-21 01:24:49

[permalink] [raw]

Subject: Re: ZONE_DMA

Christoph Lameter wrote:
> On Wed, 20 Sep 2006, Martin J. Bligh wrote:
>
>> Having something that's used in generic code that means random
>> things on different arches just seems like a recipe for disaster
>> to me.
>
> ZONE_DMA is only used as ZONE_NORMAL if the architecture does not
> need ZONE_NORMAL because all of memory is reachable via DMA.

That's still inconsistent because it doesn't say DMA for *which*
device.

>> OK ... but requesting ZONE_DMA means what? DMAable for which device?
>> Is it always a floppy disk? on some platforms a PCI card? And how
>> is the VM meant to know what the device is capable of anyway?
>
> I already explained that twice to you.

We seem to be miscommunicating ... you did indeed give a technically
correct definition. But in practice, AFAICS, it's useless. The requestor
has no idea what the arch has implemented, if it's a driver from
arch-independent code.

> I think we all agree that the situation could be better.

Indeed, that would seem to cause little dispute.

>> Having an arch-specific definition of the limit is arbitrary and
>> useless, is it not? The limit is imposed by the device and its
>> driver, we're not communicating it into any sensible way into the
>> VM code, AFAICS. Unless we're pretending we never call it from
>> generic code, which seems woefully unlikely to me.
>
> Its bad but its not useless. See how various arches use it.
>
>> Are we redefining ZONE_DMA to always be 16MB limit across all
>> architectures? At least that'd be consistent.
>
> That wont work because many architectures use different limits. Maybe you
> should once in a while have a look at the sources.

I'm perfectly well aware that it's inconsistent, that's my whole point.
However, by some chance of history, it's sort of vaguely working. I
think it's dangerous to mess with it rather than fixing it properly.

AFAICS, the correct way to do this is have the requestor pass a memory
bound into the allocator, and have the arch figure out which zones
are applicable.

> Actually the desaster is cleaned up by this patch. A couple of
> architectures that were wrongly using ZONE_DMA now use ZONE_NORMAL.

Odd that the PPC64 maintainers didn't seem to know about this.
Perhaps it might be a good idea to talk to them before doing this?

M.

2006-09-21 02:29:46

by john stultz

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans (NTP changes)

On Wed, 2006-09-20 at 13:54 -0700, Andrew Morton wrote:
> A wander through the -mm patch queue, along with some commentary on my
> intentions.
>
> When replying to this email, please rewrite the Subject: to something
> appropriate. Please also attempt to cc the appropriate developer(s).
>
> ntp-move-all-the-ntp-related-code-to-ntpc.patch
> ntp-move-all-the-ntp-related-code-to-ntpc-fix.patch
> ntp-add-ntp_update_frequency.patch
> ntp-add-ntp_update_frequency-fix.patch
> ntp-add-time_adj-to-tick-length.patch
> ntp-add-time_freq-to-tick-length.patch
> ntp-prescale-time_offset.patch
> ntp-add-time_adjust-to-tick-length.patch
> ntp-remove-time_tolerance.patch
> ntp-convert-time_freq-to-nsec-value.patch
> ntp-convert-to-the-ntp4-reference-model.patch
> ntp-cleanup-defines-and-comments.patch
> kernel-time-ntpc-possible-cleanups.patch
> kill-wall_jiffies.patch
>
> Will merge.

No objections here, but I wanted to put forth some caution as I've seen
some odd NTP behavior with the full NTP patchset on my laptop (either it
does not converge or it just converges *much* more slowly then without).
Unfortunately I've not been able to collect solid enough data to analyze
the issue (really, each run should go for atleast a full day and always
run on the same network).

However, in trying to narrow it down, the
ntp-add-time_adj-to-tick-length patch is where the behavior seems to
change the most. Again, no solid data, so I could be seeing ghosts, but
I wanted to mention it.

I'll try to put aside some time to run a few longer tests and see if I
can get some clear results.

thanks
-john

2006-09-21 02:40:36

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans (NTP changes)

On Wed, 20 Sep 2006 19:28:51 -0700
john stultz <[email protected]> wrote:

> On Wed, 2006-09-20 at 13:54 -0700, Andrew Morton wrote:
> > A wander through the -mm patch queue, along with some commentary on my
> > intentions.
> >
> > When replying to this email, please rewrite the Subject: to something
> > appropriate. Please also attempt to cc the appropriate developer(s).
> >
> > ntp-move-all-the-ntp-related-code-to-ntpc.patch
> > ntp-move-all-the-ntp-related-code-to-ntpc-fix.patch
> > ntp-add-ntp_update_frequency.patch
> > ntp-add-ntp_update_frequency-fix.patch
> > ntp-add-time_adj-to-tick-length.patch
> > ntp-add-time_freq-to-tick-length.patch
> > ntp-prescale-time_offset.patch
> > ntp-add-time_adjust-to-tick-length.patch
> > ntp-remove-time_tolerance.patch
> > ntp-convert-time_freq-to-nsec-value.patch
> > ntp-convert-to-the-ntp4-reference-model.patch
> > ntp-cleanup-defines-and-comments.patch
> > kernel-time-ntpc-possible-cleanups.patch
> > kill-wall_jiffies.patch
> >
> > Will merge.
>
> No objections here, but I wanted to put forth some caution as I've seen
> some odd NTP behavior with the full NTP patchset on my laptop (either it
> does not converge or it just converges *much* more slowly then without).
> Unfortunately I've not been able to collect solid enough data to analyze
> the issue (really, each run should go for atleast a full day and always
> run on the same network).
>
> However, in trying to narrow it down, the
> ntp-add-time_adj-to-tick-length patch is where the behavior seems to
> change the most. Again, no solid data, so I could be seeing ghosts, but
> I wanted to mention it.
>
> I'll try to put aside some time to run a few longer tests and see if I
> can get some clear results.
>

OK, thanks. Won't merge.

2006-09-21 03:39:52

[permalink] [raw]

Subject: Re: Autofs4 breakage (was 2.6.19 -mm merge plans)

On Wed, 20 Sep 2006 17:55:33 -0400
Trond Myklebust <[email protected]> wrote:

> On Wed, 2006-09-20 at 13:54 -0700, Andrew Morton wrote:
>
> > add-newline-to-nfs-dprintk.patch
> > fs-nfs-make-code-static.patch
> >
> > NFS queue -> Trond.
> >
> > The NFS git tree breaks autofs4 submounts. Still.
>
> I still suspect that is due to a misconfigured selinux setup on your
> machine. If autofs4 expects to be able to do mkdir() on your NFS
> partition (something which in itself is wrong), then selinux should be
> configured to allow it to do so.
>
> Anyhow, does reverting the patch
>
> http://kernel.org/git/gitweb.cgi?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=a634904a7de0d3a0bc606f608007a34e8c05bfee;hp=ddeff520f02b92128132c282c350fa72afffb84a
>
> 'fix' the issue for you?
>

yes.

2006-09-21 04:22:29

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Andrew Morton wrote:
> A wander through the -mm patch queue, along with some commentary on my
> intentions.
>
>
> When replying to this email, please rewrite the Subject: to something
> appropriate. Please also attempt to cc the appropriate developer(s).
>
>
> There are quite a lot of patches here which belong in subsystem trees.
> I'll patchbomb the relevant maintainers soon. Could I pleeeeeze ask that
> they either merge the patches or solidly nack them (with reasons)? Don't
> just ignore it all and leave me hanging onto this stuff for ever. Thanks.

I know this is probably heresy, but what would happen if we didn't merge
all that stuff at once, and then committed to having a real 4-week cycle?

The cycles seem to be stretching out again, and I don't really think
it's worth it to hold up the entire kernel for every single piddly
little regression to get fixed. We'll _never_ be perfect, even if we
weren't slackers.

Jeff

2006-09-21 05:07:53

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, 21 Sep 2006 00:22:26 -0400
Jeff Garzik <[email protected]> wrote:

> Andrew Morton wrote:
> > A wander through the -mm patch queue, along with some commentary on my
> > intentions.
> >
> >
> > When replying to this email, please rewrite the Subject: to something
> > appropriate. Please also attempt to cc the appropriate developer(s).
> >
> >
> > There are quite a lot of patches here which belong in subsystem trees.
> > I'll patchbomb the relevant maintainers soon. Could I pleeeeeze ask that
> > they either merge the patches or solidly nack them (with reasons)? Don't
> > just ignore it all and leave me hanging onto this stuff for ever. Thanks.
>
> I know this is probably heresy, but what would happen if we didn't merge
> all that stuff at once, and then committed to having a real 4-week cycle?

(where'd 4 weeks come from?)

Why would a shorter cycle be better? What are we trying to achieve?

> The cycles seem to be stretching out again, and I don't really think
> it's worth it to hold up the entire kernel for every single piddly
> little regression to get fixed. We'll _never_ be perfect, even if we
> weren't slackers.
>

People seem to treat the stabilisation period as a wonderful quiet time in
which to run off and develop new features, rather than participating in the
stabilisation. This has the following effects:

1: release cycles get longer

2: the kernel has more bugs

3: we put new features into the kernel faster than we otherwise would
(see 2:, above).

Furthermore, in this period we have 60-odd disjoint trees which are based
on a relatively-slowly-changing mainline. This makes people think they are
free to go berzerk, leaving me bemusedly wondering why there are VFS and
NFS changes in the OCFS2 tree, SATA changes in the powerpc tree, SATA
changes in the scsi tree, configfs changes in the GFS2 tree,
every-goddam-thing changes in the driver tree, MM changes in the parisc
tree, etc, etc, etc.

If you think that shortening the release cycle will cause people to be more
disciplined in their changes, to spend less time going berzerk and to spend
more time working with our users and testers on known bugs then I'm all
ears.

So... it again comes down to "what are we trying to achieve"? Me, I'd
like to see people spending less time developing whizzy new things and more
time fixing bugs, tuning performance, etc. That would fix the lengthy
release cycle problem automatically.

What do _you_ want to achieve by making changes?

And a question. The current batch of git trees has:

2611 files changed, 295643 insertions(+), 130150 deletions(-)

How much of this has been suitably reviewed?

2006-09-21 05:23:40

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Wed, 20 Sep 2006, Andrew Morton wrote:
>
> Why would a shorter cycle be better? What are we trying to achieve?

I don't think a shorter cycle is necessarily better, but I think we could
try having a more "directed" cycle, and perhaps merge certain specific
things rather than everything.

That would possibly _cause_ a shorter cycle, if only because the problems
are hopefully more focused from the fact that we merged with a certain
focus.

Of course, it would likely just frustrate the people who didn't get
merged, and would need to wait for the next cycle. So it might be a net
negative, even if we'd bring individual cycles in a bit.

> > The cycles seem to be stretching out again, and I don't really think
> > it's worth it to hold up the entire kernel for every single piddly
> > little regression to get fixed. We'll _never_ be perfect, even if we
> > weren't slackers.

I think that's true. 2.6.18 got delayed partly due to me beign away, but I
also think that it then got delayed too much afterwards too, just because
I felt a bit nervous about having been away ;)

So it definitely stretched out too much.

Whether there is a lot we can do about it, I dunno. In many ways, the real
issue is simply that we have a lot of changes. And people are _never_ as
interested in the testing part as they were in writing new code..

Linus

2006-09-21 06:36:59

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Andrew Morton wrote:
> If you think that shortening the release cycle will cause people to be more
> disciplined in their changes, to spend less time going berzerk and to spend
> more time working with our users and testers on known bugs then I'm all
> ears.

Honestly, I do think it would be positive. It would shorten the
feedback loop, and get more changes out to testers.

It would also decrease the pressure of the 60+ trees trying to get
everything in, because they know the next release is 3-4 months away.
It would be _much_ easier to say "break the generic device stuff in
2.6.20 not 2.6.19, please" if we knew 2.6.20 wasn't going to be a 2007
release.

Jeff

2006-09-21 06:48:35

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, 21 Sep 2006 02:36:55 -0400
Jeff Garzik <[email protected]> wrote:

> Andrew Morton wrote:
> > If you think that shortening the release cycle will cause people to be more
> > disciplined in their changes, to spend less time going berzerk and to spend
> > more time working with our users and testers on known bugs then I'm all
> > ears.
>
> Honestly, I do think it would be positive. It would shorten the
> feedback loop, and get more changes out to testers.
>
> It would also decrease the pressure of the 60+ trees trying to get
> everything in, because they know the next release is 3-4 months away.
> It would be _much_ easier to say "break the generic device stuff in
> 2.6.20 not 2.6.19, please" if we knew 2.6.20 wasn't going to be a 2007
> release.
>

Well, it might be worth trying. But there's a practical problem: how do we
get there when there's so much work pending? If we skip some people's
trees then they'll get sore, and it's not obvious that it'll help much, as
the various trees are fairly unrelated (ie: parallelisable).

I guess the most practical way is to incrementally shorten the cycles.

<rerererepeating self>

I do think that any process change we make should send the signal "slow
down, be more careful, test and review it more carefully". Or at least,
"try to make sure it compiles".

A compulsory Reviewed-by: would wedge things up nicely ;)

2006-09-21 08:50:19

by Alan

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Ar Mer, 2006-09-20 am 22:23 -0700, ysgrifennodd Linus Torvalds:
>
> On Wed, 20 Sep 2006, Andrew Morton wrote:
> >
> > Why would a shorter cycle be better? What are we trying to achieve?
>
> I don't think a shorter cycle is necessarily better, but I think we could
> try having a more "directed" cycle, and perhaps merge certain specific
> things rather than everything.

Works for me. We do need to keep pushing drivers each cycle (and we need
faster cycles just to keep up with the chipset people) but a situation
where people are told to keep those driver updates working with the old
core code would be fine (ie as 2.4 sometimes was) for some of the cycles
when they are not the goal.

Alan

2006-09-21 08:53:26

by Alan

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

> If you think that shortening the release cycle will cause people to be more
> disciplined in their changes, to spend less time going berzerk and to spend
> more time working with our users and testers on known bugs then I'm all
> ears.

A suggestion from the department of evil ideas: Call even cycles
development odd ones stabilizing. Nothing gets into an odd one without a
review and linux-kernel signoff/ack ?

Alan

2006-09-21 10:24:31

by Roman Zippel

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans (NTP changes)

Hi,

On Wed, 20 Sep 2006, john stultz wrote:

> No objections here, but I wanted to put forth some caution as I've seen
> some odd NTP behavior with the full NTP patchset on my laptop (either it
> does not converge or it just converges *much* more slowly then without).
> Unfortunately I've not been able to collect solid enough data to analyze
> the issue (really, each run should go for atleast a full day and always
> run on the same network).

grumble...
As I said before it's expected that the initial covergence is slower and I
need the data over multiple days to really say something about it.
There has been really a lot of time for doing this...

bye, Roman

2006-09-21 10:58:06

by Jan Engelhardt

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

>> If you think that shortening the release cycle will cause people to be more
>> disciplined in their changes, to spend less time going berzerk and to spend
>> more time working with our users and testers on known bugs then I'm all
>> ears.
>
>A suggestion from the department of evil ideas: Call even cycles
>development odd ones stabilizing. Nothing gets into an odd one without a
>review and linux-kernel signoff/ack ?

Why, just open 2.7.0. Harhar!

Jan Engelhardt
--

2006-09-21 11:07:41

by Rafael J. Wysocki

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thursday, 21 September 2006 08:48, Andrew Morton wrote:
> On Thu, 21 Sep 2006 02:36:55 -0400
> Jeff Garzik <[email protected]> wrote:
>
> > Andrew Morton wrote:
> > > If you think that shortening the release cycle will cause people to be more
> > > disciplined in their changes, to spend less time going berzerk and to spend
> > > more time working with our users and testers on known bugs then I'm all
> > > ears.
> >
> > Honestly, I do think it would be positive. It would shorten the
> > feedback loop, and get more changes out to testers.
> >
> > It would also decrease the pressure of the 60+ trees trying to get
> > everything in, because they know the next release is 3-4 months away.
> > It would be _much_ easier to say "break the generic device stuff in
> > 2.6.20 not 2.6.19, please" if we knew 2.6.20 wasn't going to be a 2007
> > release.
> >
>
> Well, it might be worth trying. But there's a practical problem: how do we
> get there when there's so much work pending? If we skip some people's
> trees then they'll get sore, and it's not obvious that it'll help much, as
> the various trees are fairly unrelated (ie: parallelisable).
>
> I guess the most practical way is to incrementally shorten the cycles.
>
>
> <rerererepeating self>
>
> I do think that any process change we make should send the signal "slow
> down, be more careful, test and review it more carefully". Or at least,
> "try to make sure it compiles".
>
> A compulsory Reviewed-by: would wedge things up nicely ;)

Well, I think this need not help. Like when some USB-related changes that
had been reviewed and even tested happened to break ohci-hcd because they
had only been tested on uhci ...

IMHO every change should appear in at least three consecutive -mm kernels
without causing any problems before it's allowed to go to the mainline.

Greetings,
Rafael

--
You never change things by fighting the existing reality.
R. Buckminster Fuller

2006-09-21 13:22:23

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

* Andrew Morton <[email protected]> wrote:

> ntp-move-all-the-ntp-related-code-to-ntpc.patch
> ntp-move-all-the-ntp-related-code-to-ntpc-fix.patch
> ntp-add-ntp_update_frequency.patch
> ntp-add-ntp_update_frequency-fix.patch
> ntp-add-time_adj-to-tick-length.patch
> ntp-add-time_freq-to-tick-length.patch
> ntp-prescale-time_offset.patch
> ntp-add-time_adjust-to-tick-length.patch
> ntp-remove-time_tolerance.patch
> ntp-convert-time_freq-to-nsec-value.patch
> ntp-convert-to-the-ntp4-reference-model.patch
> ntp-cleanup-defines-and-comments.patch
> kernel-time-ntpc-possible-cleanups.patch
> kill-wall_jiffies.patch
>
> Will merge.

would be nice to merge the -hrt queue that goes right ontop this queue.
Even if HIGH_RES_TIMERS is "default n" in the beginning. That gives us
high-res timers and dynticks which are both very important features to
certain classes of users/devices.

The current queue is at:

http://tglx.de/projects/hrtimers/2.6.18/

Hm?

Ingo

2006-09-21 14:04:23

by Michael Hanselmann

[permalink] [raw]

Subject: Re: Apple Motion Sensor (was: 2.6.19 -mm merge plans)

On Wed, Sep 20, 2006 at 01:54:38PM -0700, Andrew Morton wrote:
> apple-motion-sensor-driver-2.patch
> apple-motion-sensor-driver-2-fixes-update.patch
> apple-motion-sensor-driver-kconfig-fix.patch

Please wait with those. We're currently trying to come up with a generic
class for all motion sensor drivers in the kernel. This will change the
userland interface again.

Thanks,
Michael

2006-09-21 15:26:28

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, 21 Sep 2006, Alan Cox wrote:
>
> A suggestion from the department of evil ideas: Call even cycles
> development odd ones stabilizing. Nothing gets into an odd one without a
> review and linux-kernel signoff/ack ?

I don't think that's an evil idea, and in fact we've discussed it before.
I personally like it - right now we tend to have that "interminable series
of -rc<n>" (where <n> is 3..) before release, and I'd almost personally
prefer to just have a rule that is more along the lines of

- 2.6.<odd> is "the big initial merges with all the obvious fixes to make
it all work" (ie roughly the current -rc2 or perhaps -rc3).

- 2.6.<even> is "no big merges, just careful fixes" (ie the current "real
release")

Each would be ~3 weeks, leaving us with effectively the same real release
schedule, just a naming change.

That said, I think Andrew was of the opinion that it doesn't really _fix_
anything, and he may well be right. What's the point of the odd release,
if the weekly snapshots after that are supposed to be strictly better than
it anyway?

So I think I may like it just because it _seems_ to combine the good
features of both the old naming scheme and the current one, but I suspect
Andrew may be right in that it doesn't _really_ change anything, deep
down.

Dunno.

Linus

2006-09-21 15:54:34

by Jan Engelhardt

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Sep 21 2006 08:25, Linus Torvalds wrote:
>
> - 2.6.<odd> is "the big initial merges with all the obvious fixes to make
> it all work" (ie roughly the current -rc2 or perhaps -rc3).
>
> - 2.6.<even> is "no big merges, just careful fixes" (ie the current "real
> release")

That sounds interesting. To me this looks like a careful approach at
more or less marking a release "this contains new stuff" and "this does
not", like 2.<odd>.x and 2.<even>.x, respectively, but of course without
the code divergence that happened between 2.4 and 2.5.

Will there be a -stable branch for 2.6.<odd>s, or will they be limited
to 2.6.<even>, just like there is no -stable for -rcs?

>Each would be ~3 weeks, leaving us with effectively the same real release
>schedule, just a naming change.

More or less, yes. Let's assume a "good release" (one with a fair number
of -rcs), and compare both approches (hope your MUA does tabs=8):

Week/Current Current style Week/Proposal Your proposal
+0 2.6.18 +0 2.6.18
+2 2.6.19-rc1 - none
+3 2.6.19-rc2 +3 2.6.19
+4 2.6.19-rc3 - none
+5 2.6.19 +6 2.6.20
+7 2.6.20-rc1 - none
+8 2.6.20-rc2 +9 2.6.21
+9 2.6.20-rc3 - none
+10 2.6.21 +12 2.6.22
(+1 between each -rc is purely assumptional)

Though this is a purely dry mathematical table. We did not always stick
to a "-rc3 at most" rule, but gone up to like -rc7 recently. With the
new versioning scheme, no such thing seems likely to be happening
(excluding of course external influences like people travelling).
IOW, the schedule gets more organized. I like that idea.

(Based upon the assumption that 1 week passes between each -rc
release, there would, with the new proposal, 'only' be 243 weeks to hit
2.6.99 instead of 405. That is, version numbers go up faster :)

Jan Engelhardt
--

2006-09-21 15:58:44

[permalink] [raw]

Subject: Re: ZONE_DMA

On Wed, 20 Sep 2006, Martin J. Bligh wrote:

> > ZONE_DMA is only used as ZONE_NORMAL if the architecture does not need
> > ZONE_NORMAL because all of memory is reachable via DMA.
>
> That's still inconsistent because it doesn't say DMA for *which*
> device.

Thats the way ZONE_DMA works right now and AFAIK the only way forward is
to make it optional and then introduce another way of allocating memory
for a device. The migrate away from it. The first step is to allow people
who do not need ZONE_DMA to opt out.

> > That wont work because many architectures use different limits. Maybe you
> > should once in a while have a look at the sources.
>
> I'm perfectly well aware that it's inconsistent, that's my whole point.
> However, by some chance of history, it's sort of vaguely working. I
> think it's dangerous to mess with it rather than fixing it properly.

I think you are spreading FUD. The existing scheme has been working for a
long time. Come up with something concrete. I am not
changing the definition of ZONE_DMA.

> AFAICS, the correct way to do this is have the requestor pass a memory
> bound into the allocator, and have the arch figure out which zones
> are applicable.

Exactly. But you cannot do that with ZONE_DMA __GFP_DMA. We likely need a
new page allocator API for that.

> > Actually the desaster is cleaned up by this patch. A couple of architectures
> > that were wrongly using ZONE_DMA now use ZONE_NORMAL.
>
> Odd that the PPC64 maintainers didn't seem to know about this.
> Perhaps it might be a good idea to talk to them before doing this?

Maybe they have not been reading linux-arch? It was discussed among arch
maintainers and 4 arches have switched off ZONE_DMA for good.

2006-09-21 16:00:19

by James Bottomley

[permalink] [raw]

Subject: Re: ZONE_DMA (was: Re: 2.6.19 -mm merge plans)

On Thu, 2006-09-21 at 02:03 +0100, Alan Cox wrote:
> IOMMUs don't always help. They IOMMU aperture on AMD64 for example is
> too high to help devices with below 32bit limits.

That's because it's not an IOMMU; it's a GART. A true IOMMU separates
the machine physical and bus physical address spaces ... a GART merely
remaps a hole in physical address space.

James

2006-09-21 16:57:19

[permalink] [raw]

Subject: Re: ZONE_DMA

Christoph Lameter wrote:
> On Wed, 20 Sep 2006, Martin J. Bligh wrote:
>
>
>>>ZONE_DMA is only used as ZONE_NORMAL if the architecture does not need
>>>ZONE_NORMAL because all of memory is reachable via DMA.
>>
>>That's still inconsistent because it doesn't say DMA for *which*
>>device.
>
>
> Thats the way ZONE_DMA works right now and AFAIK the only way forward is
> to make it optional and then introduce another way of allocating memory
> for a device. The migrate away from it. The first step is to allow people
> who do not need ZONE_DMA to opt out.

OK. Let's leave aside the issue for a second of whether ZONE_DMA should
be configurable or not (ie your patch), and just worry about how this
works in practice going forwards in the short term. Don't get me wrong,
I'd love to kill ZONE_DMA, or at least the 16MB way it's implemented in
i386 right now.

I presume the fallback order for everything is still
HIGHMEM -> NORMAL -> DMA, and nobody is proposing changing that.
(ignoring DMA32 to keep thing simpler).

If a device driver wants "DMAable" memory, and thus does a ZONE_DMA
allocation, and we've moved all its memory from ZONE_DMA to ZONE_NORMAL
(as I think you're proposing doing for PPC64 (and ia64?)), then the
allocation will fail.

So are we saying that no driver code should be calling with GFP_DMA
(a quick grep turns up 148 instances under driver/), that if they do
they should only work on specific architectures (some instances were
s390-only drivers)? If so, should we not be removing the definiton of
GFP_DMA itself if ZONE_DMA is config'ed out, so that it fails at
compile time, rather than runtime?

>>AFAICS, the correct way to do this is have the requestor pass a memory
>>bound into the allocator, and have the arch figure out which zones
>>are applicable.
>
> Exactly. But you cannot do that with ZONE_DMA __GFP_DMA. We likely need a
> new page allocator API for that.

Glad we're agreed on that, at least.

M.

2006-09-21 17:35:17

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, 21 Sep 2006 15:14:33 +0200
Ingo Molnar <[email protected]> wrote:

>
> * Andrew Morton <[email protected]> wrote:
>
> > ntp-move-all-the-ntp-related-code-to-ntpc.patch
> > ntp-move-all-the-ntp-related-code-to-ntpc-fix.patch
> > ntp-add-ntp_update_frequency.patch
> > ntp-add-ntp_update_frequency-fix.patch
> > ntp-add-time_adj-to-tick-length.patch
> > ntp-add-time_freq-to-tick-length.patch
> > ntp-prescale-time_offset.patch
> > ntp-add-time_adjust-to-tick-length.patch
> > ntp-remove-time_tolerance.patch
> > ntp-convert-time_freq-to-nsec-value.patch
> > ntp-convert-to-the-ntp4-reference-model.patch
> > ntp-cleanup-defines-and-comments.patch
> > kernel-time-ntpc-possible-cleanups.patch
> > kill-wall_jiffies.patch
> >
> > Will merge.
>
> would be nice to merge the -hrt queue that goes right ontop this queue.
> Even if HIGH_RES_TIMERS is "default n" in the beginning. That gives us
> high-res timers and dynticks which are both very important features to
> certain classes of users/devices.
>
> The current queue is at:
>
> http://tglx.de/projects/hrtimers/2.6.18/
>

I've never even looked at this work. Just add it to the pipeline I guess.
We can merge it once it passes review by Roman ;)

But the possible problems with the NTP patches which John has possibly
observed were news to me - it's not clear what the situation is there.

2006-09-21 17:54:54

[permalink] [raw]

Subject: Re: ZONE_DMA

On Thu, 21 Sep 2006, Martin Bligh wrote:

> I presume the fallback order for everything is still
> HIGHMEM -> NORMAL -> DMA, and nobody is proposing changing that.
> (ignoring DMA32 to keep thing simpler).

It would help if you would actually look at the code instead of presuming.
No changes are made in that area.

> If a device driver wants "DMAable" memory, and thus does a ZONE_DMA
> allocation, and we've moved all its memory from ZONE_DMA to ZONE_NORMAL
> (as I think you're proposing doing for PPC64 (and ia64?)), then the
> allocation will fail.

I would not presume proposing anything for PPC64 and left everything the
way it is. The arch people can control if they want ZONE DMA or not and
the default is to leave things as is. If PPC64 wants to go ZONE_DMAless
then the arch code needs to be modified not refer to ZONE_DMA anymore and
if that works then we can switch CONFIG_ZONE_DMA off for PPC64. See the
patches in mm for examples how other arches have done it.

> So are we saying that no driver code should be calling with GFP_DMA
> (a quick grep turns up 148 instances under driver/), that if they do
> they should only work on specific architectures (some instances were
> s390-only drivers)? If so, should we not be removing the definiton of
> GFP_DMA itself if ZONE_DMA is config'ed out, so that it fails at
> compile time, rather than runtime?

This was covered at length before. Removing all GFP_DMA references would
require extensive #ifdefs. The limited patch in mm is only neutering
GFP_DMA for arches that do not need it. If an arch has removed its definition of
CONFIG_ZONE_DMA then __GFP_DMA will be ignored in the page allocator.

> > > AFAICS, the correct way to do this is have the requestor pass a memory
> > > bound into the allocator, and have the arch figure out which zones
> > > are applicable.
> >
> > Exactly. But you cannot do that with ZONE_DMA __GFP_DMA. We likely need a
> > new page allocator API for that.
>
> Glad we're agreed on that, at least.

I think we agree on a lot more. Hopefully when we meet at lunch today we
can sync some more.

2006-09-21 18:00:18

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, 21 Sep 2006 08:25:55 -0700 (PDT)
Linus Torvalds <[email protected]> wrote:

>
>
> On Thu, 21 Sep 2006, Alan Cox wrote:
> >
> > A suggestion from the department of evil ideas: Call even cycles
> > development odd ones stabilizing. Nothing gets into an odd one without a
> > review and linux-kernel signoff/ack ?
>
> I don't think that's an evil idea, and in fact we've discussed it before.
> I personally like it - right now we tend to have that "interminable series
> of -rc<n>" (where <n> is 3..) before release, and I'd almost personally
> prefer to just have a rule that is more along the lines of
>
> - 2.6.<odd> is "the big initial merges with all the obvious fixes to make
> it all work" (ie roughly the current -rc2 or perhaps -rc3).
>
> - 2.6.<even> is "no big merges, just careful fixes" (ie the current "real
> release")
>
> Each would be ~3 weeks, leaving us with effectively the same real release
> schedule, just a naming change.
>
> That said, I think Andrew was of the opinion that it doesn't really _fix_
> anything, and he may well be right. What's the point of the odd release,
> if the weekly snapshots after that are supposed to be strictly better than
> it anyway?
>
> So I think I may like it just because it _seems_ to combine the good
> features of both the old naming scheme and the current one, but I suspect
> Andrew may be right in that it doesn't _really_ change anything, deep
> down.
>

Again, before we can implement anything we should describe what problem we are
actually trying to solve here.

Jeff: "I want faster release cycles because <no reason given>"

Me: "I want less bugs"

Anyone else?

2006-09-21 18:20:08

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Andrew Morton wrote:
> Jeff: "I want faster release cycles because <no reason given>"

All the standard goodness that "release early, release often" provides.

* Avoiding the achingly long wait where huge amounts of changes pile up,
then go in. It should be OBVIOUS that
merge 10,000 changes. global test. repeat
is worse than
merge 1,000 changes. global test. repeat.

I think it's patently unfair to complain about bugs and regressions,
then limit developers to 3-4 test points [mainline releases] per year.

* Faster release cycles means code doesn't spend a quarter of the year
in limbo before users test it and give good feedback.

* Code stands a better chance of getting more review.

* Regressions are perceived to be fixed more quickly, if the fix
requires more than just 1-2 lines going to [email protected].

* Submitters don't have to wait for a quarter of a year in order for
their submissions to hit a mainline release.

With this last release, I just didn't see the value at all to going all
the way to -rc7. There weren't huge numbers of testers screaming about
-rc1 and -rc2. It just seemed like we delayed for no good reason other
than a blind hope that the passage of time would fix bugs.

Jeff

2006-09-21 18:22:57

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, 21 Sep 2006, Andrew Morton wrote:
>
> Again, before we can implement anything we should describe what problem we are
> actually trying to solve here.
>
> Jeff: "I want faster release cycles because <no reason given>"
>
> Me: "I want less bugs"
>
> Anyone else?

Me: "I want peoples expectations to line up".

(That, btw, is totally independent of this particulay issue - I just like
people to know what to expect..)

One of the things that I think the current model has excelled at is how it
really changed peoples behaviour, simply because they knew and understood
the rules.

I think the "big merges in the first two weeks, and a -rc1 after, and no
new code after that" rule has been working because it brought everybody in
on the same page.

I actually expected people to dislike arbitrary rules more than they do,
but I've come to believe that people _like_ having rules that they have to
obey, as long as it's not a big pain for them. In other words, arbitrary
rules are not actually disliked at all, people actually _like_ them,
because suddenly there's less need for making unnecessary judgement
decisions.

As an example: I thought I'd get a lot of back-lash on the whole sign-off
procedure. Instead, we're basically signing off everything, and having a
few simple rules ended up making it just easier to forward stuff, and we
haven't had any of the discussions about who gets to be attributed as an
author since the sign-off was introduced. That was a totally unexpected
bonus, as far as I was concerned.

The same goes for my anal efforts at trying to make people use a specific
format for sending patches, and sending the "please pull" messages. I'm
not hearing any grumbling about it at all, and in fact I'm getting the
distinct feeling that people like knowing exactly what format to use,
because they didn't really care themselves, and it turns out that having
any rule - even if it's fairly arbitrary - seems to be better than not
having a rule at all.

So I think that a "odd release"/"even release" rule that clarifies what a
certain mid-point in the release cycle actually _means_, even if it
doesn't necessarily add anything else, might be a good thing. It just
solidifies peoples expectations about where we are in a release cycle.

If we make an arbitrary rule to go with the release cycle ("leading up to
the even cycle, you need to get an ack from somebody that actually tested
the fix") that could actually be a good thing, for this reason.

I dunno. Maybe the only arbitrary rule ends up being that an "odd release"
would become a good place for people to try, knowing that we expect bug
reports from them. Right now -rc1 might be _too_ scary, even if it ends up
being exactly that: the only difference is really not about technology,
but about what peoples expectations are.

If we could instill a culture of "if you aren't a developer, but you just
want to help out, try the odd releases", that in itself might be worth the
naming change. If it would allow a group of people who might not feel
comfortable about reporting problems with a "-rc" to feel like they are
_expected_ to report a problem with an odd release, then that would be a
good thing, no?

I'm just throwing this out as an idea. I'm not going to really argue very
strongly for it. It might have horrible downsides too, for all I know, and
we might get people who didn't get the memo on "even vs odd releases"
being really unhappy about somethign they perceive to be a buggy release.

Linus

2006-09-21 18:33:44

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Linus Torvalds wrote:
> One of the things that I think the current model has excelled at is how it
> really changed peoples behaviour, simply because they knew and understood
> the rules.
>
> I think the "big merges in the first two weeks, and a -rc1 after, and no
> new code after that" rule has been working because it brought everybody in
> on the same page.

I definitely agree with all that.

I simply argue that, the more time that passes between releases, the
MORE BUGS that appear in the next release.

After -rc1, you reach a point of diminishing returns where users don't
re-test Release Candidates, developers move on to new code rather than
fix bugs, and we all move into a limbo where 2.6.X-rcY doesn't see much
activity, but the huge "merge snowball" in -mm builds and builds and builds.

As an aside, if a release is getting held up by some key bugs or
regressions, I think it's more than fair for Andrew to loudly shame said
developers into action. "The following nincompoops are holding up the
release: Jeff Garzik [bug #1222, #3391], Greg KH [bug #9987, #4418], ..."

Jeff

2006-09-21 18:56:06

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, 21 Sep 2006 14:33:41 -0400
Jeff Garzik <[email protected]> wrote:

> I think it's more than fair for Andrew to loudly shame said
> developers into action. "The following nincompoops are holding up the
> release: Jeff Garzik [bug #1222, #3391], Greg KH [bug #9987, #4418], ..."

I don't have the bandwidth to do that work, alas. I know how to do it, and
have tried to do it, but a) it's dull and b) even I get tired of whining at
people all the time.

The good news is that google is prepared to hire someone to sit next to me
and do it, but I haven't got off my ass and written the job description
yet.

2006-09-21 19:46:13

by Adrian Bunk

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, Sep 21, 2006 at 02:33:41PM -0400, Jeff Garzik wrote:
> Linus Torvalds wrote:
> >One of the things that I think the current model has excelled at is how it
> >really changed peoples behaviour, simply because they knew and understood
> >the rules.
> >
> >I think the "big merges in the first two weeks, and a -rc1 after, and no
> >new code after that" rule has been working because it brought everybody in
> >on the same page.
>
>
> I definitely agree with all that.
>
> I simply argue that, the more time that passes between releases, the
> MORE BUGS that appear in the next release.
>
> After -rc1, you reach a point of diminishing returns where users don't
> re-test Release Candidates, developers move on to new code rather than
> fix bugs, and we all move into a limbo where 2.6.X-rcY doesn't see much
> activity, but the huge "merge snowball" in -mm builds and builds and builds.

This "there is not enough testing by users" is simply not true:

Even the kernel Bugzilla that contains only a small subset of all bug
reports contains 78 (sic) open bugs reported against 2.6.18-rc kernels [1].

Not all of them are regressions, but this shows that users are testing
-rc kernels and reporting bugs.

> As an aside, if a release is getting held up by some key bugs or
> regressions, I think it's more than fair for Andrew to loudly shame said
> developers into action. "The following nincompoops are holding up the
> release: Jeff Garzik [bug #1222, #3391], Greg KH [bug #9987, #4418], ..."
>
> Jeff

cu
Adrian

[1] including bugs reported against 2.6.18-rc?-mm? and 2.6.18-rc?-git*

--

"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed

2006-09-21 20:37:22

by Diego Calleja

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

El Thu, 21 Sep 2006 21:46:04 +0200,
Adrian Bunk <[email protected]> escribi?:

> Even the kernel Bugzilla that contains only a small subset of all bug
> reports contains 78 (sic) open bugs reported against 2.6.18-rc kernels [1].

I suspect that not many people is subscribed to the bugzilla mailing list,
not surprising since the URLs doesn't seem to be in the tree :)

After fixing my english, I wonder if the following patch could be applied...

Signed-off-by: Diego Calleja <[email protected]>

--- 2.6/Documentation/HOWTO.OLD 2006-09-21 22:14:06.000000000 +0200
+++ 2.6/Documentation/HOWTO 2006-09-21 22:36:17.000000000 +0200
@@ -374,6 +374,26 @@ of information is needed by the kernel d
problem.

+Managing bug reports
+--------------------
+
+One of the best ways to put into practice your hacking skills is by fixing
+bugs reported by other people. Not only you will help to make the kernel
+more stable, you'll learn to fix real world problems and you will improve
+your skills, and other developers will be aware of your presence. Fixing
+bugs is one of the best ways to get merits between other developers, because
+not many people like wasting time fixing other people's bugs.
+
+To work in the already reported bug reports, go to http://bugzilla.kernel.org.
+If you want to be advised of the future bug reports, you can subscribe to the
+bugme-new mailing list (only new bug reports are mailed here) or to the
+bugme-janitor mailing list (every change in the bugzilla is mailed here)
+
+ http://lists.osdl.org/mailman/listinfo/bugme-new
+ http://lists.osdl.org/mailman/listinfo/bugme-janitors
+
+
+
Mailing lists
-------------

2006-09-21 20:38:28

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Adrian Bunk wrote:
> Not all of them are regressions, but this shows that users are testing
> -rc kernels and reporting bugs.

I never claimed otherwise. But every release turns up plenty of bugs
that are otherwise uncaught, because its inevitable that a much larger
set of users tests the full releases.

Jeff

2006-09-21 20:49:37

by Adrian Bunk

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, Sep 21, 2006 at 04:38:21PM -0400, Jeff Garzik wrote:
> Adrian Bunk wrote:
> >Not all of them are regressions, but this shows that users are testing
> >-rc kernels and reporting bugs.
>
>
> I never claimed otherwise. But every release turns up plenty of bugs
> that are otherwise uncaught, because its inevitable that a much larger
> set of users tests the full releases.

My point is that we don't lack testers and bug reports - we lack people
fixing bugs.

> Jeff

cu
Adrian

--

"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed

2006-09-21 21:29:34

by Bill Davidsen

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

,Linus Torvalds wrote:
>
> On Thu, 21 Sep 2006, Andrew Morton wrote:
>> Again, before we can implement anything we should describe what problem we are
>> actually trying to solve here.
>>
>> Jeff: "I want faster release cycles because <no reason given>"
>>
>> Me: "I want less bugs"
>>
>> Anyone else?
>
> Me: "I want peoples expectations to line up".
>
> (That, btw, is totally independent of this particulay issue - I just like
> people to know what to expect..)
>
> One of the things that I think the current model has excelled at is how it
> really changed peoples behaviour, simply because they knew and understood
> the rules.
>
> I think the "big merges in the first two weeks, and a -rc1 after, and no
> new code after that" rule has been working because it brought everybody in
> on the same page.
>
> I actually expected people to dislike arbitrary rules more than they do,
> but I've come to believe that people _like_ having rules that they have to
> obey, as long as it's not a big pain for them. In other words, arbitrary
> rules are not actually disliked at all, people actually _like_ them,
> because suddenly there's less need for making unnecessary judgement
> decisions.
>
> As an example: I thought I'd get a lot of back-lash on the whole sign-off
> procedure. Instead, we're basically signing off everything, and having a
> few simple rules ended up making it just easier to forward stuff, and we
> haven't had any of the discussions about who gets to be attributed as an
> author since the sign-off was introduced. That was a totally unexpected
> bonus, as far as I was concerned.
>
> The same goes for my anal efforts at trying to make people use a specific
> format for sending patches, and sending the "please pull" messages. I'm
> not hearing any grumbling about it at all, and in fact I'm getting the
> distinct feeling that people like knowing exactly what format to use,
> because they didn't really care themselves, and it turns out that having
> any rule - even if it's fairly arbitrary - seems to be better than not
> having a rule at all.
>
> So I think that a "odd release"/"even release" rule that clarifies what a
> certain mid-point in the release cycle actually _means_, even if it
> doesn't necessarily add anything else, might be a good thing. It just
> solidifies peoples expectations about where we are in a release cycle.
>
> If we make an arbitrary rule to go with the release cycle ("leading up to
> the even cycle, you need to get an ack from somebody that actually tested
> the fix") that could actually be a good thing, for this reason.
>
> I dunno. Maybe the only arbitrary rule ends up being that an "odd release"
> would become a good place for people to try, knowing that we expect bug
> reports from them. Right now -rc1 might be _too_ scary, even if it ends up
> being exactly that: the only difference is really not about technology,
> but about what peoples expectations are.
>
> If we could instill a culture of "if you aren't a developer, but you just
> want to help out, try the odd releases", that in itself might be worth the
> naming change. If it would allow a group of people who might not feel
> comfortable about reporting problems with a "-rc" to feel like they are
> _expected_ to report a problem with an odd release, then that would be a
> good thing, no?
>
> I'm just throwing this out as an idea. I'm not going to really argue very
> strongly for it. It might have horrible downsides too, for all I know, and
> we might get people who didn't get the memo on "even vs odd releases"
> being really unhappy about somethign they perceive to be a buggy release.
>
> Linus
I think it would help if you went back to using meaningful names for
releases, because 2.6.19-test1 is pretty clearly a test release even to
people who can't figure out if a number is odd or even. Then after
people stop reporting show stoppers, change to rc numbers, where rc
versions are actually candidates for release without known major bugs.

Test and rc have pretty clear meanings, anyone who puts a test release
on a production machine can't complain they didn't know, and people who
want to wait for the "should not eat your data" stage will be more
willing to try a release.

Automating testing: someone could write a simple web form to report test
results (like sign off, sort of). CPU, distribution, network, display,
filesystems used, that sort of thing. So I say FC4, Radeon-x, HT on X86,
multiple GigE, port blocking and forwarding, SNAT, TV bttv, DVD burn,
smbfs, ext3. When N people have reported no issues with a feature, you
consider it tested. If no one tests something like UML, cryptoloop, nbd,
Reiser4, or UFS, then you poke the list to try stuff. Or when a change
goes in, mark it "needs testing" on some web site. In other words
improve developer <=> tester communication without using a lot of
someone's time.

I know we have a test tracking system, but I don't see "testers needed"
messages on the list, and I'm not sure you get or use feedback from any
test tracking to identify untested features in kernels.

--
Bill Davidsen <[email protected]>
Obscure bug of 2004: BASH BUFFER OVERFLOW - if bash is being run by a
normal user and is setuid root, with the "vi" line edit mode selected,
and the character set is "big5," an off-by-one errors occurs during
wildcard (glob) expansion.

2006-09-21 21:33:34

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Bill Davidsen wrote:
> I think it would help if you went back to using meaningful names for
> releases, because 2.6.19-test1 is pretty clearly a test release even to
> people who can't figure out if a number is odd or even. Then after
> people stop reporting show stoppers, change to rc numbers, where rc
> versions are actually candidates for release without known major bugs.

Actually, considering our group of developers, I think "-rc" has been
remarkably successful at staying on the "bug fixes only" theme.

Jeff

2006-09-21 21:34:34

by Randy Dunlap

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, 21 Sep 2006 22:37:10 +0200 Diego Calleja wrote:

> El Thu, 21 Sep 2006 21:46:04 +0200,
> Adrian Bunk <[email protected]> escribi?:
>
> > Even the kernel Bugzilla that contains only a small subset of all bug
> > reports contains 78 (sic) open bugs reported against 2.6.18-rc kernels [1].
>
> I suspect that not many people is subscribed to the bugzilla mailing list,
> not surprising since the URLs doesn't seem to be in the tree :)
>
> After fixing my english, I wonder if the following patch could be applied...
>
> Signed-off-by: Diego Calleja <[email protected]>
>
> --- 2.6/Documentation/HOWTO.OLD 2006-09-21 22:14:06.000000000 +0200
> +++ 2.6/Documentation/HOWTO 2006-09-21 22:36:17.000000000 +0200
> @@ -374,6 +374,26 @@ of information is needed by the kernel d
> problem.
>
>
> +Managing bug reports
> +--------------------
> +
> +One of the best ways to put into practice your hacking skills is by fixing
> +bugs reported by other people. Not only you will help to make the kernel
> +more stable, you'll learn to fix real world problems and you will improve
> +your skills, and other developers will be aware of your presence. Fixing
> +bugs is one of the best ways to get merits between other developers, because

s/between/among/ :)

Acked-by: Randy Dunlap <[email protected]>

> +not many people like wasting time fixing other people's bugs.
> +
> +To work in the already reported bug reports, go to http://bugzilla.kernel.org.
> +If you want to be advised of the future bug reports, you can subscribe to the
> +bugme-new mailing list (only new bug reports are mailed here) or to the
> +bugme-janitor mailing list (every change in the bugzilla is mailed here)
> +
> + http://lists.osdl.org/mailman/listinfo/bugme-new
> + http://lists.osdl.org/mailman/listinfo/bugme-janitors
> +
> +
> +
> Mailing lists
> -------------
>
> -

---
~Randy

2006-09-21 21:51:55

by David Miller

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

From: Jeff Garzik <[email protected]>
Date: Thu, 21 Sep 2006 17:33:27 -0400

> Bill Davidsen wrote:
> > I think it would help if you went back to using meaningful names for
> > releases, because 2.6.19-test1 is pretty clearly a test release even to
> > people who can't figure out if a number is odd or even. Then after
> > people stop reporting show stoppers, change to rc numbers, where rc
> > versions are actually candidates for release without known major bugs.
>
> Actually, considering our group of developers, I think "-rc" has been
> remarkably successful at staying on the "bug fixes only" theme.

I agree.

But even on that note I would love to have a release cycle where I
didn't merge any new features and could work entirely on the bugs
that never get worked on.

Sure, I'll still be merging new features into my "N + 1" tree.
But my pure interactions with Linus's tree can focus entirely
on bug fixing, and I really want an environment in which to
concentrate on that exclusively.

I think the even/odd idea is great, personally. And if this
makes some people have to wait a little bit longer for their
favorite feature to get merged, that's tough. :-)

I truly think we need to move towards a more stability minded
mode and back off on the new features just a bit.

2006-09-21 22:08:08

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, Sep 21, 2006 at 02:52:08PM -0700, David Miller wrote:

> But even on that note I would love to have a release cycle where I
> didn't merge any new features and could work entirely on the bugs
> that never get worked on.

Would certainly be nice, even if we didn't do it every-other, but
once every half dozen or so releases. I've been looking over the
osdl bugzilla recently (Ironically in a form of escapism from
the Fedora bugzilla). There's a ton of really old reports in there
that could be mopped up with a targetted bugfixing release.
Right now, with so many open bugs, it's difficult to get a real
picture of where the problem areas are because there's so much
crap in there. (Fedora's bugzilla is actually going through the
same problem right now too sadly, at least in part because
the last few releases have taken so damned long to come out, and
the -stable releases whilst an improvement, haven't gone far
enough to fixing a lot of issues users are seeing[*]).

> Sure, I'll still be merging new features into my "N + 1" tree.
> But my pure interactions with Linus's tree can focus entirely
> on bug fixing, and I really want an environment in which to
> concentrate on that exclusively.

There's nothing actually stopping you from enforcing this rule
in the trees you maintain though. You could do this for networking
in .19 without having a mandate from Linus that the kernel as
a whole is going to do the same.

Not that networking is an area that sees that many regressions
compared to other subsystems IMO. What's your secret? :)

> I think the even/odd idea is great, personally. And if this
> makes some people have to wait a little bit longer for their
> favorite feature to get merged, that's tough. :-)

My concern is that people will 'sit out' the even stage, and
just accumulate stuff in a single tree they dump once when
every odd release opens up.

We already have some subsystems that do once-per-release merges,
and then let fixes build up in their out-of-tree SCM for months
until the next window. It won't necessarily get worse, but unless
everyone is participating in the odd/even rules, we won't get
the benefits that it would offer.

Dave

[*] I'm not demeaning Greg & Chris' work here at all, they've
been doing a stellar job, but I think we could use more people
going through the changelogs looking for stuff that needs
backporting.

2006-09-21 22:44:05

by David Miller

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

From: Dave Jones <[email protected]>
Date: Thu, 21 Sep 2006 18:05:39 -0400

> On Thu, Sep 21, 2006 at 02:52:08PM -0700, David Miller wrote:
>
> > I think the even/odd idea is great, personally. And if this
> > makes some people have to wait a little bit longer for their
> > favorite feature to get merged, that's tough. :-)
>
> My concern is that people will 'sit out' the even stage, and
> just accumulate stuff in a single tree they dump once when
> every odd release opens up.

At least they would be dumping on top of "mostly working".
I kind of like that. It breeds more confidence into the
tree having been working before the dump took place, thus
making the isolation of cause much easier.

> We already have some subsystems that do once-per-release merges,
> and then let fixes build up in their out-of-tree SCM for months
> until the next window. It won't necessarily get worse, but unless
> everyone is participating in the odd/even rules, we won't get
> the benefits that it would offer.

Having odd/even rules kind of adds legitimacy to the per-tree folks
doing the same. This avoids situations like "why is XXX being an
asshole with his tree, when there are other trees merging new
features this round?". Having buy-in from everyone is very useful
and gets folks in the correct mindset.

2006-09-21 23:16:04

[permalink] [raw]

Subject: Re: ZONE_DMA

>>If a device driver wants "DMAable" memory, and thus does a ZONE_DMA
>>allocation, and we've moved all its memory from ZONE_DMA to ZONE_NORMAL
>>(as I think you're proposing doing for PPC64 (and ia64?)), then the
>>allocation will fail.
>
> I would not presume proposing anything for PPC64 and left everything the
> way it is. The arch people can control if they want ZONE DMA or not and
> the default is to leave things as is. If PPC64 wants to go ZONE_DMAless
> then the arch code needs to be modified not refer to ZONE_DMA anymore and
> if that works then we can switch CONFIG_ZONE_DMA off for PPC64. See the
> patches in mm for examples how other arches have done it.

Sorry, there was some confusion then, maybe I misunderstood one of your
earlier emails.

>>So are we saying that no driver code should be calling with GFP_DMA
>>(a quick grep turns up 148 instances under driver/), that if they do
>>they should only work on specific architectures (some instances were
>>s390-only drivers)? If so, should we not be removing the definiton of
>>GFP_DMA itself if ZONE_DMA is config'ed out, so that it fails at
>>compile time, rather than runtime?
>
>
> This was covered at length before. Removing all GFP_DMA references would
> require extensive #ifdefs. The limited patch in mm is only neutering
> GFP_DMA for arches that do not need it. If an arch has removed its definition of
> CONFIG_ZONE_DMA then __GFP_DMA will be ignored in the page allocator.

Just ignoring GFP_DMA in the allocator seems like a horrible violation
to me. GFP_DMA means "give me memory from ZONE_DMA". We're both well
aware that the whole concept of ZONE_DMA doesn't make much sense, but
still, that's what it does.

So if you just put all of memory in ZONE_DMA for your particular
machine, and bumped the DMA limit up to infinity, we wouldn't need
any of these patches, right? Which would also match what the other
arches do for this (eg PPC64).

Seems like a much simpler way of solving the problem to me. Or do you
think that won't work for some reason?

M.

2006-09-21 23:25:30

by Trond Myklebust

[permalink] [raw]

Subject: Re: Autofs4 breakage (was 2.6.19 -mm merge plans)

On Wed, 2006-09-20 at 20:39 -0700, Andrew Morton wrote:
> On Wed, 20 Sep 2006 17:55:33 -0400
> Trond Myklebust <[email protected]> wrote:
>
> > On Wed, 2006-09-20 at 13:54 -0700, Andrew Morton wrote:
> >
> > > add-newline-to-nfs-dprintk.patch
> > > fs-nfs-make-code-static.patch
> > >
> > > NFS queue -> Trond.
> > >
> > > The NFS git tree breaks autofs4 submounts. Still.
> >
> > I still suspect that is due to a misconfigured selinux setup on your
> > machine. If autofs4 expects to be able to do mkdir() on your NFS
> > partition (something which in itself is wrong), then selinux should be
> > configured to allow it to do so.
> >
> > Anyhow, does reverting the patch
> >
> > http://kernel.org/git/gitweb.cgi?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=a634904a7de0d3a0bc606f608007a34e8c05bfee;hp=ddeff520f02b92128132c282c350fa72afffb84a
> >
> > 'fix' the issue for you?
> >
>
> yes.

Hmm... but you aren't seeing it with a clean 2.6.18 kernel?

Cheers,
Trond

2006-09-21 23:46:40

by Wu Fengguang

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Wed, Sep 20, 2006 at 01:54:38PM -0700, Andrew Morton wrote:
>
> The readahead code is complex, I'm unconvinced that it has a lot of benefit
> and Wu has gone quiet. Will drop.

Sorry, I've been putting efforts to meet the deadline of the google
SoC project "Rapid linux desktop startup through pre-caching", which
still can not be called success for now. And there's my pending paper
work...

I should be able to come back and concentrate on the readahead patch
after one month, whether it be dropped for now. I guess it can be
further improved in complexity and performance. It also needs a good
document for the overall design and benefits. And sure the
performance numbers :-)

Regards,
Fengguang Wu

2006-09-21 23:59:22

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, 22 Sep 2006 07:46:37 +0800
Fengguang Wu <[email protected]> wrote:

> On Wed, Sep 20, 2006 at 01:54:38PM -0700, Andrew Morton wrote:
> >
> > The readahead code is complex, I'm unconvinced that it has a lot of benefit
> > and Wu has gone quiet. Will drop.
>
> Sorry, I've been putting efforts to meet the deadline of the google
> SoC project "Rapid linux desktop startup through pre-caching", which
> still can not be called success for now. And there's my pending paper
> work...

Oh, OK.

That's a neat thing to be working on.

> I should be able to come back and concentrate on the readahead patch
> after one month, whether it be dropped for now. I guess it can be
> further improved in complexity and performance. It also needs a good
> document for the overall design and benefits. And sure the
> performance numbers :-)

I'll hang onto the patches then - they're causing little maintenance
trouble for me.

2006-09-22 00:59:21

by Bill Davidsen

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Jeff Garzik wrote:
> Bill Davidsen wrote:
>> I think it would help if you went back to using meaningful names for
>> releases, because 2.6.19-test1 is pretty clearly a test release even
>> to people who can't figure out if a number is odd or even. Then after
>> people stop reporting show stoppers, change to rc numbers, where rc
>> versions are actually candidates for release without known major bugs.
>
>
> Actually, considering our group of developers, I think "-rc" has been
> remarkably successful at staying on the "bug fixes only" theme.
>
Perhaps I misread what Linus said, the issue I was suggesting be
addressed was one of clarity to the testers, not the developers. The
releases identified as test would be for evaluation, while the ones
identified as rc would really be candidates with no "fix before next
version" bugs known. I would think that between test releases some bugs
could be fixed, but new features could be added. That would encourage
more active testing without overly slowing the development process.

Having used that for a long time for 2.2 and 2.4 I think there's quite a
track record of that nomenclature being clear to the users.

--
Bill Davidsen <[email protected]>
Obscure bug of 2004: BASH BUFFER OVERFLOW - if bash is being run by a
normal user and is setuid root, with the "vi" line edit mode selected,
and the character set is "big5," an off-by-one errors occurs during
wildcard (glob) expansion.

2006-09-22 02:00:30

by George Spelvin

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

>> A suggestion from the department of evil ideas: Call even cycles
>> development odd ones stabilizing. Nothing gets into an odd one without a
>> review and linux-kernel signoff/ack ?
>
> I don't think that's an evil idea, and in fact we've discussed it before.
> I personally like it - right now we tend to have that "interminable series
> of -rc<n>" (where <n> is 3..) before release, and I'd almost personally
> prefer to just have a rule that is more along the lines of
>
> - 2.6.<odd> is "the big initial merges with all the obvious fixes to make
> it all work" (ie roughly the current -rc2 or perhaps -rc3).
>
> - 2.6.<even> is "no big merges, just careful fixes" (ie the current "real
> release")
>
> Each would be ~3 weeks, leaving us with effectively the same real release
> schedule, just a naming change.
>
> That said, I think Andrew was of the opinion that it doesn't really _fix_
> anything, and he may well be right. What's the point of the odd release,
> if the weekly snapshots after that are supposed to be strictly better than
> it anyway?
>
> So I think I may like it just because it _seems_ to combine the good
> features of both the old naming scheme and the current one, but I suspect
> Andrew may be right in that it doesn't _really_ change anything, deep
> down.

Not meaning to re-open old wounds, but this is exactly why people
complained about dropping the "-pre" kernels. That was a good way to
distinguish between "the features are in, but so are some obvious bugs;
read the changelog" and, in -rc, "this has no *known* bugs, so please
test and report".

If you want to just go back to that (2.6.<odd>-rcX would be -pre, and
2.6.<odd> would be -rc1), I don't think people would need it explained
to them. And the kernel.org mirror system already understands it.

Although, as you say, its not clear that any of this politically
correct renaming is actually changing anything.

If someone really wants to affect the bug count, figure out a way to
auto-generate a "bugs and regressions in mainline, by contributor" score
and publicize that. As the various distributed computing projects have
found, people can be competitive about the silliest things if you give
them a score to measure themselves by.

Since kudos would flow from the author of a bug to the finder as well
as the fixer, it could simultaneously encourage testers.

Quantifying all of this hand-waving is, of course, a non-trivial
project (how do you compare a printk spelling fix to a silent
IDE data corruption bug?), but it would be very worthwhile.

2006-09-22 02:59:15

[permalink] [raw]

Subject: Re: ZONE_DMA

On Thu, 21 Sep 2006, Martin Bligh wrote:

> Just ignoring GFP_DMA in the allocator seems like a horrible violation
> to me. GFP_DMA means "give me memory from ZONE_DMA". We're both well
> aware that the whole concept of ZONE_DMA doesn't make much sense, but
> still, that's what it does.

We agreed that the definition of ZONE_DMA is not consistent across
architectures.

The concept of ZONE_DMA makes only sense in terms of an architectures
definition if a memory boundary (MAX_DMA_ADDRESS) exists for special DMA
devices not able to reach all of memory. If we do not have ZONE_DMA then the
architecture has to remove the definition of CONFIG_ZONE_DMA
and with that action told us that it is allowable to ignore GFP_DMA since
all devices can do DMA to all of memory (and all of memory is memory
without a border which is of course in ZONE_NORMAL).

GFP_DMA like GFP_HIGHMEM and GFP_DMA32 means give me memory from the zone
if its there. If not (the arch has no such memory) we fall back to ZONE_NORMAL.

This is fully consistent with established uses.

> So if you just put all of memory in ZONE_DMA for your particular
> machine, and bumped the DMA limit up to infinity, we wouldn't need
> any of these patches, right? Which would also match what the other
> arches do for this (eg PPC64).

That would mean abusing ZONE_DMA for a purpose it was not intended for.
ZOME_DMA is used to partition memory for a DMA not for covering all of
memory. That works yes but it shows a misunderstanding of the purpose for
which ZONE_DMA was created.

Also if you would do that then ZONE_NORMAL would be empty and you would
not be able to reach the goal of a system with a single zone. The slab
allocator gets thoroughly confused and waste pages allocating
memory in different slabs for ZONE_NORMAL and ZONE_DMA but they end up in
the same ZONE_DMA. Various other bits and pieces of the VM behave in
strange way but it works mostly. Seems that you got lucky but this should
be fixed.

ZONE_NORMAL is DMAable. GFP_DMA has never meant this is for DMA but it has
always meant this is for a special restricted DMA zone. That is also why
you have GFP_DMA32. Both GFP_DMA and GFP_DMA32 select special restricted
memory areas for handicapped DMA devices that are not able to reach all of
memory. Neither should cover all of memory.

2006-09-22 06:33:28

by Andi Kleen

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Andrew Morton <[email protected]> writes:
>
> Jeff: "I want faster release cycles because <no reason given>"

I think i would also prefer somewhat faster cycles, simple to get
more regular full scale testing.

-Andi

2006-09-22 08:35:52

by Russell King

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, Sep 21, 2006 at 06:05:39PM -0400, Dave Jones wrote:
> We already have some subsystems that do once-per-release merges,
> and then let fixes build up in their out-of-tree SCM for months
> until the next window. It won't necessarily get worse, but unless
> everyone is participating in the odd/even rules, we won't get
> the benefits that it would offer.

I'm heading in that direction (once-per-release merges) actually.

On one hand, I'm credited with the ARM architecture being one of the
best maintained embedded architectures in the kernel tree. On the
other hand, that appears to be winding Linus up due to the regular
merge requests, which were happening maybe once or twice a week.

Linus seems to be of the opinion that, if anyone can't wait a number
of months for their patch to get into mainline, then they shouldn't
be involved in this game. The content of the tree which that comment
was made at contained (imho) just bug fixes.

At the moment, we're up to 528kB (initial commit Aug 21st) of IOP3xx
and S3C24xx machine updates, and various other developments. As for
the other trees, MMC (9kB since Aug 27) and serial (20kB since Aug 30)
but neither have been looked at for a while, certainly not post 2.6.18.
I'm not even responding to mail about these because I haven't been even
thinking about them yet.

As far as -mm getting these, I have asked Andrew to pull this tree in
the past, but whenever I rebase the trees (eg, when 2.6.18 comes out)
and fix up the rejects, Andrew seems to have a hard time coping. I
guess Andrew finds it too difficult to handle my devel branches.

Where I go from here I'm not sure - I'm running out of ideas for
correct "Care and Operation of (my) Linus Torvalds", except becoming
one of the Bad People who only merge _lots_ of changes once in a blue
moon.

So, what I'm going to be doing this cycle is essentially sitting on
stuff for quite some time and not really caring about where in the
release cycle mainline actually is. (Anyone remember Linus moaning
at various people for doing exactly this? Eg, ALSA people?) It
pains me to do this because it's obviously not the _correct_ thing
to do, but I don't see any other way of keeping Linus happy. And this
does mean giving up all hope of getting anything in mainline.

As far as my future, I will be handing MMC off to Pierre Ossman during
this cycle (there are other reasons for doing this which Pierre has been
aware of for some time.)

I'll also be dropping my serial tree entirely - I have no idea who could
stand in for serial, so there's going to be no real "hand over" for that.
I do have some outstanding in-progress changes which aren't really ready,
but those will probably end up in /dev/null (in much the same way that my
in-progress changes for PCMCIA ended up in a similar place when I handed
that tree over.)

So, it's going to mean that the only thing I'm going to be caring about
post-2.6.19 is ARM again.

--
Russell King
Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/
maintainer of: 2.6 Serial core

2006-09-22 09:24:35

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Wed, 2006-09-20 at 13:54 -0700, Andrew Morton wrote:
>
> mtd-maps-ixp4xx-partition-parsing.patch
> fix-the-unlock-addr-lookup-bug-in-mtd-jedec-probe.patch
> mtd-printk-format-warning.patch
> fs-jffs2-jffs2_fs_ih-removal-of-old-code.patch
> drivers-mtd-nand-au1550ndc-removal-of-old-code.patch
>
> MTD queue -> dwmw2

Merged, with the exception of the unlock addr one which I'm still not
sure about -- about to investigate harder.

Also merged are
pci-mtd-switch-to-pci_get_device-and-do-ref-counting.patch and
avr32-mtd-unlock-flash-if-necessary.patch

--
dwmw2

2006-09-22 10:10:29

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, 2006-09-22 at 10:24 +0100, David Woodhouse wrote:
> On Wed, 2006-09-20 at 13:54 -0700, Andrew Morton wrote:
> >
> > mtd-maps-ixp4xx-partition-parsing.patch
> > fix-the-unlock-addr-lookup-bug-in-mtd-jedec-probe.patch
> > mtd-printk-format-warning.patch
> > fs-jffs2-jffs2_fs_ih-removal-of-old-code.patch
> > drivers-mtd-nand-au1550ndc-removal-of-old-code.patch
> >
> > MTD queue -> dwmw2
>
> Merged, with the exception of the unlock addr one which I'm still not
> sure about -- about to investigate harder.

I just reverted Randy's printk format 'fix', since rq->flags _is_ an
unsigned long, so changing from %ld to %d actually _introduces_ a
warning.

Randy, that's the second time I recall recently that I've ended up
reverting a printk format fix from you -- what are you doing? How did
you manage to get the warning you reported:

drivers/mtd/mtd_blkdevs.c:72: warning: long int format, different type arg (arg 2)

--
dwmw2

2006-09-22 10:46:03

by Jan Engelhardt

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

>> > mtd-maps-ixp4xx-partition-parsing.patch
>> > fix-the-unlock-addr-lookup-bug-in-mtd-jedec-probe.patch
>> > mtd-printk-format-warning.patch
>> > fs-jffs2-jffs2_fs_ih-removal-of-old-code.patch
>> > drivers-mtd-nand-au1550ndc-removal-of-old-code.patch
>> >
>> > MTD queue -> dwmw2
>>
>> Merged, with the exception of the unlock addr one which I'm still not
>> sure about -- about to investigate harder.
>
>I just reverted Randy's printk format 'fix', since rq->flags _is_ an
>unsigned long, so changing from %ld to %d actually _introduces_ a
>warning.

If it is an unsigned long, it should neither be %ld nor %d, but %lu.

Jan Engelhardt
--

2006-09-22 11:21:06

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, 2006-09-22 at 12:42 +0200, Jan Engelhardt wrote:
> If it is an unsigned long, it should neither be %ld nor %d, but %lu.

It'll never be negative.

--
dwmw2

2006-09-22 12:33:35

by Haavard Skinnemoen

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans: AVR32

On Wed, 20 Sep 2006 13:54:38 -0700
Andrew Morton <[email protected]> wrote:

> AVR32 architecture. Will fold into a single patch, and will merge.

Thanks a lot :-) I will do my best to ensure it stays in good shape
during the -rc series.

> avr32-mtd-static-memory-controller-driver-try-2.patch
> avr32-mtd-at49bv6416-platform-device-for-atstk1000.patch
> avr32-mtd-unlock-flash-if-necessary.patch
>
> MTD changes for avr32. Will send to dwmw2.

It might actually make more sense to fold the first two into the avr32
architecture patch. Especially now that David has picked up the third
one (which ended up not being AVR32-specific at all.)

The static memory controller driver isn't really specific to MTD
anyway, it should be equally useful for all kinds of external
memory-mapped devices including CompactFlash.

Haavard

2006-09-22 14:42:55

by Christoph Hellwig

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

> ecryptfs-fs-makefile-and-fs-kconfig.patch
> ecryptfs-fs-makefile-and-fs-kconfig-kconfig-help-update.patch
> ecryptfs-documentation.patch
> ecryptfs-makefile.patch
> ecryptfs-main-module-functions.patch
> ecryptfs-header-declarations.patch
> ecryptfs-superblock-operations.patch
> #ecryptfs-superblock-operations-ecryptfs-build-fix.patch
> ecryptfs-dentry-operations.patch
> ecryptfs-file-operations.patch
> #ecryptfs-vs-streamline-generic_file_-interfaces-and-filemap.patch
> #ecryptfs-vs-streamline-generic_file_-interfaces-and-filemap-fix.patch
> ecryptfs-inode-operations.patch
> ecryptfs-mmap-operations.patch
> ecryptfs-mmap-operations-fix.patch
> ecryptfs-keystore.patch
> ecryptfs-crypto-functions.patch
> ecryptfs-crypto-functions-mutex-fixes.patch
> fs-ecryptfs-possible-cleanups.patch
> ecryptfs-debug-functions.patch
> ecryptfs-alpha-build-fix.patch
> ecryptfs-convert-assert-to-bug_on.patch
> ecryptfs-remove-pointless-bug_ons.patch
> ecryptfs-remove-unnecessary-null-checks.patch
> ecryptfs-rewrite-ecryptfs_fsync.patch
> ecryptfs-overhaul-file-locking.patch
> ecryptfs-remove-lock-propagation.patch
> ecryptfs-dont-muck-with-the-existing-nameidata-structures.patch
> ecryptfs-asm-scatterlisth-linux-scatterlisth.patch
> ecryptfs-support-for-larger-maximum-key-size.patch
> ecryptfs-add-codes-for-additional-ciphers.patch
> ecryptfs-unencrypted-key-size-based-on-encrypted-key-size.patch
> ecryptfs-packet-and-key-management-update-for-variable-key-size.patch
> ecryptfs-add-ecryptfs_-prefix-to-mount-options-key-size-parameter.patch
> ecryptfs-set-the-key-size-from-the-default-for-the-mount.patch
> ecryptfs-check-for-weak-keys.patch
> ecryptfs-add-define-values-for-cipher-codes-from-rfc2440-openpgp.patch
> ecryptfs-convert-bits-to-bytes.patch
> ecryptfs-more-elegant-aes-key-size-manipulation.patch
> ecryptfs-more-intelligent-use-of-tfm-objects.patch
> ecryptfs-remove-debugging-cruft.patch
> ecryptfs-get_sb_dev-fix.patch
> ecryptfs-validate-minimum-header-extent-size.patch
> ecryptfs-validate-body-size.patch
> ecryptfs-validate-packet-length-prior-to-parsing-add-comments.patch
> ecryptfs-use-the-passed-in-max-value-as-the-upper-bound.patch
> ecryptfs-change-the-maximum-size-check-when-writing-header.patch
> ecryptfs-print-the-actual-option-that-is-problematic.patch
> ecryptfs-add-a-maintainers-entry.patch
> ecryptfs-partial-signed-integer-to-size_t-conversion-updated-ii.patch
> ecryptfs-graceful-handling-of-mount-error.patch
> inode-diet-move-i_pipe-into-a-union-ecryptfs.patch
> inode-diet-eliminate-i_blksize-and-use-a-per-superblock-default-ecryptfs.patch
> streamline-generic_file_-interfaces-and-filemap-ecryptfs.patch
> ecryptfs-fix-printk-format-warnings.patch
> ecryptfs-associate-vfsmount-with-dentry-rather-than-superblock.patch
> ecryptfs-mntput-lower-mount-on-umount_begin.patch
> vfs-make-filldir_t-and-struct-kstat-deal-in-64-bit-inode-numbers-ecryptfs.patch
> make-kmem_cache_destroy-return-void-ecryptfs.patch
> ecryptfs-inode-numbering-fixes.patch
> ecryptfs-versioning-fixes.patch
> ecryptfs-versioning-fixes-tidy.patch
>
> Will fold into a single patch and will then merge.

Please add a

Not-Acked-By: Christoph Hellwig <[email protected]>

here as you don't seem to listen to any of the comments I had against
the current state and I want this clearly documented.

2006-09-22 15:48:59

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, Sep 22, 2006 at 09:35:42AM +0100, Russell King wrote:
> On Thu, Sep 21, 2006 at 06:05:39PM -0400, Dave Jones wrote:
> > We already have some subsystems that do once-per-release merges,
> > and then let fixes build up in their out-of-tree SCM for months
> > until the next window. It won't necessarily get worse, but unless
> > everyone is participating in the odd/even rules, we won't get
> > the benefits that it would offer.
>
> I'm heading in that direction (once-per-release merges) actually.
>
> On one hand, I'm credited with the ARM architecture being one of the
> best maintained embedded architectures in the kernel tree. On the
> other hand, that appears to be winding Linus up due to the regular
> merge requests, which were happening maybe once or twice a week.

Hmm. Some trees do seem to get pulled more often than others.
Linus, is there a upper limit on the number of times you want
to see pull requests? It strikes me as odd, so I'm wondering
if there are some crossed wires here.

> As far as -mm getting these, I have asked Andrew to pull this tree in
> the past, but whenever I rebase the trees (eg, when 2.6.18 comes out)
> and fix up the rejects, Andrew seems to have a hard time coping. I
> guess Andrew finds it too difficult to handle my devel branches.

Has Andrew commented on why this is proving to be more of a problem?
I've done regular rebases of cpufreq/agpgart (admittedly, they don't
reject hardly ever unless Len has ACPI bits touching cpufreq) without
causing too much headache.

> So, what I'm going to be doing this cycle is essentially sitting on
> stuff for quite some time and not really caring about where in the
> release cycle mainline actually is.

That doesn't sound like the right way to fix the 'caring for my Linus' problem :)

> As far as my future, I will be handing MMC off to Pierre Ossman during
> this cycle (there are other reasons for doing this which Pierre has been
> aware of for some time.)
>
> I'll also be dropping my serial tree entirely - I have no idea who could
> stand in for serial, so there's going to be no real "hand over" for that.
> I do have some outstanding in-progress changes which aren't really ready,
> but those will probably end up in /dev/null (in much the same way that my
> in-progress changes for PCMCIA ended up in a similar place when I handed
> that tree over.)

That's unfortunate. If you want someone to scoop bits up and feed Linus,
I'm happy to volunteer for the task, as long as you're still willing
to eyeball serial diffs until I get up to speed.

Dave

2006-09-22 15:55:35

[permalink] [raw]

Subject: Re: Autofs4 breakage (was 2.6.19 -mm merge plans)

On Thu, 21 Sep 2006 19:25:19 -0400
Trond Myklebust <[email protected]> wrote:

> On Wed, 2006-09-20 at 20:39 -0700, Andrew Morton wrote:
> > On Wed, 20 Sep 2006 17:55:33 -0400
> > Trond Myklebust <[email protected]> wrote:
> >
> > > On Wed, 2006-09-20 at 13:54 -0700, Andrew Morton wrote:
> > >
> > > > add-newline-to-nfs-dprintk.patch
> > > > fs-nfs-make-code-static.patch
> > > >
> > > > NFS queue -> Trond.
> > > >
> > > > The NFS git tree breaks autofs4 submounts. Still.
> > >
> > > I still suspect that is due to a misconfigured selinux setup on your
> > > machine. If autofs4 expects to be able to do mkdir() on your NFS
> > > partition (something which in itself is wrong), then selinux should be
> > > configured to allow it to do so.
> > >
> > > Anyhow, does reverting the patch
> > >
> > > http://kernel.org/git/gitweb.cgi?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=a634904a7de0d3a0bc606f608007a34e8c05bfee;hp=ddeff520f02b92128132c282c350fa72afffb84a
> > >
> > > 'fix' the issue for you?
> > >
> >
> > yes.
>
> Hmm... but you aren't seeing it with a clean 2.6.18 kernel?
>

Yes, I am. Mainline broke.

2006-09-22 16:01:11

by Alan

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Ar Gwe, 2006-09-22 am 11:48 -0400, ysgrifennodd Dave Jones:
> That's unfortunate. If you want someone to scoop bits up and feed Linus,
> I'm happy to volunteer for the task, as long as you're still willing
> to eyeball serial diffs until I get up to speed.

I'll give you a hand with that, I've got to do some work in that area
anyway for the arbitary speed support.

2006-09-22 16:03:46

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans (ecryptfs)

On Fri, 22 Sep 2006 15:42:33 +0100
Christoph Hellwig <[email protected]> wrote:

> > ecryptfs-versioning-fixes-tidy.patch
> >
> > Will fold into a single patch and will then merge.
>
> Please add a
>
> Not-Acked-By: Christoph Hellwig <[email protected]>
>
> here as you don't seem to listen to any of the comments I had against
> the current state and I want this clearly documented.

I thought everything got addressed, apart from

- please split all the generic stackable filesystem passthorugh routines
into a separated stackfs layer, in a few files in fs/stackfs/ that
you depend on. They'll get _GPL exported to all possible stackable
filesystem. They'll need their own store underlying object helpers,
but that can be made to work by embedding the generic stackfs data
as first thing in the ecryptfs object.

which seemed rather a lot to ask.

2006-09-22 16:09:24

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, Sep 22, 2006 at 05:24:01PM +0100, Alan Cox wrote:
> Ar Gwe, 2006-09-22 am 11:48 -0400, ysgrifennodd Dave Jones:
> > That's unfortunate. If you want someone to scoop bits up and feed Linus,
> > I'm happy to volunteer for the task, as long as you're still willing
> > to eyeball serial diffs until I get up to speed.
>
> I'll give you a hand with that, I've got to do some work in that area
> anyway for the arbitary speed support.

Cool, sounds good to me. Russell?

Dave

2006-09-22 16:24:56

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, 22 Sep 2006, Dave Jones wrote:
>
> Hmm. Some trees do seem to get pulled more often than others.
> Linus, is there a upper limit on the number of times you want
> to see pull requests? It strikes me as odd, so I'm wondering
> if there are some crossed wires here.

I personally prefer to not see _too_ many pull requests, since that to me
indicates that people don't take advantage of the distributed nature of
git, and don't let things "simmer" in their own tree for a while.

[ Side note, just to explain how I personally work: getting too many
requests about the same tree confuses and sometimes irritates me, since
I tend to "batch up" my work. For example, for the last couple of days,
I've been mostly in "discussion mode", and have been talking about
licenses and workflow issues etc.

And then at some point (probably later today) I decide to go into "merge
mode" and go back to old mails I ignored and start applying them and
pulling from other peoples git trees. And so if my "mode switching" has
a longer latency than the "please pull" frequency, I end up seeing two
requests for the same tree during the same "merge mode" thing, which
just means that when I look at the older one, it no longer matches what
is in the tree I'm pulling from.

I've long done this "batching" thing - it's something I eventually
worked out with my patch-flow with Alan, and that I think we've
perfected with Andrew (probably largely _because_ we worked it out with
Alan after a certain amount of friction ;).

I personally at least _feel_ like I'm more efficient when I can just
completely switch gears, rather than having a constant trickle of
different things happening.

Hopefully that explains the other side of why I prefer to not get two
pull requests for the same tree within days of each other - I may simply
not even have gotten _around_ to the first one yet, and then the second
one just irritates me. ]

For example, I think that project maintainers should to some degree just
talk about their _own_ trees, rather than try to get their changes into my
tree, and then point to that. One of the big ideas in distribution (at
least to me) is that I'm _not_ supposed to be the "one and only", and I
think we should aim for a situation where people who develop in certain
specific areas can interact directly with the people who are testing the
results, so that by the time I get a "please pull" request, most of the
bulk of the work should hopefully already have gone through a cycle.

And all this is not even really git-centric. It's obviously what Andrew
does with the -mm tree too - havign a certain amount of "latency" is good.

That said, the "release early, release often" thing still holds, and
letting things wait _too_ long just means that the _only_ people who test
it is some very specific group, and you may not see the problems that a
bigger environment would see.

So it's a balance between "by the time you send it on, it should hopefully
have had a day or two of testing" _and_ a "by the time you send it on you
shouldn't have forgotten the issues and think it's old and all done".

I would _personally_ judge that if you need to push me the same tree more
than once a week (not counting mistakes and brown-paper bugs that
obviously happen - I'm saying "in general" here), there's likely something
strange going on.

But at the same time, please do keep in mind thatr it's partly just a
matter of taste, and it's also very much a matter of work habits (and
about how active the tree is). Some people tend to work in certain ways.

I think rmk keeps his git trees in a private location (and I think it's
because the kernel.org maintainers asked us to not mirror things out
publicly if we didn't need to), so I think part of the reason the ARM
trees get pushed out more actively is simply because Russell has used my
tree as a "distribution point".

I don't think that's necessarily great, and there's been some friction
over it ("people are waiting for this"), but it's not been a _huge_
problem either, so I just lump it in the "different people, different ways
to work" pile..

Linus

2006-09-22 16:29:36

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

NOTE: Your mailer generates bogus Mail-Followup-To headers, and you
snipped rmk from the To/CC.

Dave Jones wrote:
> Hmm. Some trees do seem to get pulled more often than others.
> Linus, is there a upper limit on the number of times you want
> to see pull requests? It strikes me as odd, so I'm wondering
> if there are some crossed wires here.

Not speaking for Linus, but in general it seems like the more pull
requests you send (within reason), the more pulls that occur. Russell
and DaveM certainly seem to send frequent, successful pull requests.

> Has Andrew commented on why this is proving to be more of a problem?
> I've done regular rebases of cpufreq/agpgart (admittedly, they don't
> reject hardly ever unless Len has ACPI bits touching cpufreq) without
> causing too much headache.

Rebasing _inevitably_ causes more headaches than a simple tree update,
for any downstream consumer of your tree(s). It is best to avoid wanton
rebasing.

Think about it: if someone is pulling and merging your tree, all of a
sudden, without warning, the entire hash history is rewritten. So
rather than a Nice and Friendly minor update, the next time they pull
your stuff, the downstream user is forced to suffer through either (a) a
painful merge, or (b) back out your last tree (ugh!) and redo things
from scratch.

Rebasing might make a pretty history, but it is _not_ fun for random
consumers of your trees. It basically punishes people for following
your tree -- not something you want to do.

Jeff

2006-09-22 16:53:34

by Michael Halcrow

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans (ecryptfs)

On Fri, Sep 22, 2006 at 09:03:37AM -0700, Andrew Morton wrote:
> On Fri, 22 Sep 2006 15:42:33 +0100
> Christoph Hellwig <[email protected]> wrote:
>
> > > ecryptfs-versioning-fixes-tidy.patch
> > >
> > > Will fold into a single patch and will then merge.
> >
> > Please add a
> >
> > Not-Acked-By: Christoph Hellwig <[email protected]>
> >
> > here as you don't seem to listen to any of the comments I had
> > against the current state and I want this clearly documented.
>
> I thought everything got addressed, apart from
>
> - please split all the generic stackable filesystem passthorugh routines
> into a separated stackfs layer, in a few files in fs/stackfs/ that
> you depend on. They'll get _GPL exported to all possible stackable
> filesystem. They'll need their own store underlying object helpers,
> but that can be made to work by embedding the generic stackfs data
> as first thing in the ecryptfs object.
>
> which seemed rather a lot to ask.

A common framework that would be useful for all potential stackable
filesystems is indeed a major project in and of itself, and such a
thing right now would be premature. We need to see how code for other
stackable filesystems will look once it is actually in the
kernel. There is core behavior that differs between eCryptfs and
Unionfs -- for instance, how inodes are referenced. eCryptfs, in its
current form, can do just fine with referencing the internal inode
struct pointer, whereas Unionfs implements persistent inodes. Then
there are issues with readdir/filldir semantics, wherein Unionfs needs
a mechanism to implement whiteout operations. eCryptfs just has a
simple one-to-one VFS layer mapping, while Unionfs has to merge
several VFS mounts under it. The core *_info data structure diverge
quite a bit between the two. Right now, eCryptfs and Unionfs can get
by with a page-to-page mapping between the upper and lower files, but
that would not necessarily be the case for a compressing stacked
layer; the common framework would have to take that into
consideration.

Stacked filesystem authors are certainly interested in addressing
these sorts of issues, but an iterative strategy for solving them is
the most reasonable approach. The first step is to get at least two
stacked filesystems into the kernel (eCryptfs and Unionfs) and then
follow up with a series of patches to extract the common functionality
once we see somewhat finalized code bases for both.

Mike

2006-09-22 17:10:11

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, Sep 22, 2006 at 12:29:21PM -0400, Jeff Garzik wrote:
>
> NOTE: Your mailer generates bogus Mail-Followup-To headers, and you
> snipped rmk from the To/CC.

Hmm, curious. I'll file a bug.

> > Has Andrew commented on why this is proving to be more of a problem?
> > I've done regular rebases of cpufreq/agpgart (admittedly, they don't
> > reject hardly ever unless Len has ACPI bits touching cpufreq) without
> > causing too much headache.
>
> Rebasing _inevitably_ causes more headaches than a simple tree update,
> for any downstream consumer of your tree(s). It is best to avoid wanton
> rebasing.

Sorry, terminology confusion. I rarely throw away a tree and start afresh,
I meant of course just updating to linus' latest. This may explain the
my lack of understanding over Russell's issues.

Dave

2006-09-22 17:11:58

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, Sep 22, 2006 at 12:29:21PM -0400, Jeff Garzik wrote:
>
> NOTE: Your mailer generates bogus Mail-Followup-To headers, and you
> snipped rmk from the To/CC.

Actually, my mailer just respected the Mail-Followup-To: that
rmk set in his mail that I replied to. Nothing to see here.

Dave

2006-09-22 17:21:44

[permalink] [raw]

Subject: Re: ZONE_DMA

On Thursday, September 21, 2006 7:59 pm, Christoph Lameter wrote:
> > So if you just put all of memory in ZONE_DMA for your particular
> > machine, and bumped the DMA limit up to infinity, we wouldn't need
> > any of these patches, right? Which would also match what the other
> > arches do for this (eg PPC64).

This is what Altix did for a long time (and it looks like it still sets
MAX_DMA_ADDRESS to the top of addressable memory).

> That would mean abusing ZONE_DMA for a purpose it was not intended for.
> ZOME_DMA is used to partition memory for a DMA not for covering all of
> memory. That works yes but it shows a misunderstanding of the purpose
> for which ZONE_DMA was created.

AFAIK ZONE_DMA was created for crappy ISA devices that could only DMA to
low addresses. As various architectures were added, they either
misunderstood its purpose, abused it for their own purposes, or ignored it
in some way as it didn't really apply.

> ZONE_NORMAL is DMAable. GFP_DMA has never meant this is for DMA but it
> has always meant this is for a special restricted DMA zone. That is also
> why you have GFP_DMA32. Both GFP_DMA and GFP_DMA32 select special
> restricted memory areas for handicapped DMA devices that are not able to
> reach all of memory. Neither should cover all of memory.

If you have a decent enough IOMMU both or either could cover all of memory.
In the case of an Altix, the IOMMU is 32 bit capable, so it would make
sense for ZONE_DMA32 to contain all of memory...

But anyway, I agree with your broader point that we really need a different
allocator for this stuff. It has to be arch specific in some way though,
so we can take into account the advantages IOMMUs provide. I think jejb
said he'd come up with a sample implementation a couple of years ago... :)

>From a portability and definition perspective, I'd contend that ZONE_DMA
and ZONE_DMA32 are both broken. Only ZONE_NORMAL and ZONE_HIGHMEM have
sane definitions it seems.

Jesse

2006-09-22 17:30:24

by Randy Dunlap

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, 22 Sep 2006 11:10:01 +0100 David Woodhouse wrote:

> On Fri, 2006-09-22 at 10:24 +0100, David Woodhouse wrote:
> > On Wed, 2006-09-20 at 13:54 -0700, Andrew Morton wrote:
> > >
> > > mtd-maps-ixp4xx-partition-parsing.patch
> > > fix-the-unlock-addr-lookup-bug-in-mtd-jedec-probe.patch
> > > mtd-printk-format-warning.patch
> > > fs-jffs2-jffs2_fs_ih-removal-of-old-code.patch
> > > drivers-mtd-nand-au1550ndc-removal-of-old-code.patch
> > >
> > > MTD queue -> dwmw2
> >
> > Merged, with the exception of the unlock addr one which I'm still not
> > sure about -- about to investigate harder.
>
> I just reverted Randy's printk format 'fix', since rq->flags _is_ an
> unsigned long, so changing from %ld to %d actually _introduces_ a
> warning.
>
> Randy, that's the second time I recall recently that I've ended up
> reverting a printk format fix from you -- what are you doing? How did
> you manage to get the warning you reported:
>
> drivers/mtd/mtd_blkdevs.c:72: warning: long int format, different type arg (arg 2)

That was originally posted as a patch to -mm, where that field
has been changed to (unsigned?) int. Maybe it should be part
of Jens's block patches, since that is what changes the field
size.

---
~~Randy

2006-09-22 17:35:38

[permalink] [raw]

Subject: Re: ZONE_DMA

Christoph Lameter wrote:
> On Thu, 21 Sep 2006, Martin Bligh wrote:
>
>
>>Just ignoring GFP_DMA in the allocator seems like a horrible violation
>>to me. GFP_DMA means "give me memory from ZONE_DMA". We're both well
>>aware that the whole concept of ZONE_DMA doesn't make much sense, but
>>still, that's what it does.
>
>
> We agreed that the definition of ZONE_DMA is not consistent across
> architectures.
>
> The concept of ZONE_DMA makes only sense in terms of an architectures
> definition if a memory boundary (MAX_DMA_ADDRESS) exists for special DMA
> devices not able to reach all of memory. If we do not have ZONE_DMA then the
> architecture has to remove the definition of CONFIG_ZONE_DMA
> and with that action told us that it is allowable to ignore GFP_DMA since
> all devices can do DMA to all of memory (and all of memory is memory
> without a border which is of course in ZONE_NORMAL).
>
> GFP_DMA like GFP_HIGHMEM and GFP_DMA32 means give me memory from the zone
> if its there. If not (the arch has no such memory) we fall back to ZONE_NORMAL.
>
> This is fully consistent with established uses.

Given that the definition of ZONE_DMA is, I think we agree, nonsensical
in generic code anyway, whatever we do is going to be a pain.

I disagree that what you're doing is consistent with established
usage, see PPC64 for example, which I assumed you'd changed to be
consistent with what you're doing ... but it seems you haven't.

Your definition is one way of interpreting the current setup. It even
makes some sort of sense, I'll admit. However, there are other
definitions that make perfect sense too, which just goes to show that
the concept of ZONE_DMA is just a frigging inconsistent mess to start
with.

>>So if you just put all of memory in ZONE_DMA for your particular
>>machine, and bumped the DMA limit up to infinity, we wouldn't need
>>any of these patches, right? Which would also match what the other
>>arches do for this (eg PPC64).
>
> That would mean abusing ZONE_DMA for a purpose it was not intended for.
> ZOME_DMA is used to partition memory for a DMA not for covering all of
> memory. That works yes but it shows a misunderstanding of the purpose for
> which ZONE_DMA was created.

That is one possible way of defining it, but seeing as there is no
agreed to, documented definition, it's hard to tell whether this trumps
such other defined constants such as "GFP_DMA means gives me memory from
ZONE_DMA", which you're now violating.

> Also if you would do that then ZONE_NORMAL would be empty and you would
> not be able to reach the goal of a system with a single zone. The slab
> allocator gets thoroughly confused and waste pages allocating
> memory in different slabs for ZONE_NORMAL and ZONE_DMA but they end up in
> the same ZONE_DMA. Various other bits and pieces of the VM behave in
> strange way but it works mostly. Seems that you got lucky but this should
> be fixed.

It seems odd that PPC64 has worked fine this way for a long time then?

> ZONE_NORMAL is DMAable. GFP_DMA has never meant this is for DMA but it has
> always meant this is for a special restricted DMA zone. That is also why
> you have GFP_DMA32. Both GFP_DMA and GFP_DMA32 select special restricted
> memory areas for handicapped DMA devices that are not able to reach all of
> memory. Neither should cover all of memory.

The last sentence in this is your opinion, not an agreed-to definition.

Look, whatever we do is not going to be wholly clean, as the definitons
and requirements we start from are loose, inconsistent and somewhat
contradictary on occasion. So what we are left with is picking something
that is:

1. As consistent as possible across architectures.
2. As simple as possible.

If you can agree with the other arch maintainers (eg PPC64) that
stuffing it all in ZONE_NORMAL is somehow better than ZONE_DMA, then
maybe we can meet (1).

However, whatever you do, meeting (2) is rather hard - it's a damned
sight simpler to stuff it all in ZONE_DMA because that's the end of
the fallback list.

M.

2006-09-22 17:37:59

[permalink] [raw]

Subject: Re: ZONE_DMA

On Fri, 22 Sep 2006, Martin Bligh wrote:

> However, whatever you do, meeting (2) is rather hard - it's a damned
> sight simpler to stuff it all in ZONE_DMA because that's the end of
> the fallback list.

That used to be the case. If you switch ZONE_DMA off (like possible in mm)
then ZONE_NORMAL is the end of the fallback. I think we basically want
the same thing and get to the same result.

2006-09-22 17:43:58

[permalink] [raw]

Subject: Re: ZONE_DMA

On Friday, September 22, 2006 10:21 am, Jesse Barnes wrote:
> If you have a decent enough IOMMU both or either could cover all of
> memory. In the case of an Altix, the IOMMU is 32 bit capable, so it
> would make sense for ZONE_DMA32 to contain all of memory...
>
> But anyway, I agree with your broader point that we really need a
> different allocator for this stuff. It has to be arch specific in some
> way though, so we can take into account the advantages IOMMUs provide.
> I think jejb said he'd come up with a sample implementation a couple of
> years ago... :)
>
> From a portability and definition perspective, I'd contend that ZONE_DMA
> and ZONE_DMA32 are both broken. Only ZONE_NORMAL and ZONE_HIGHMEM have
> sane definitions it seems.

Ok and right after I sent this my brain returned from vacation and I
remembered jejb's DMA allocation API. It's powerful enough to cover most
driver use cases I think (users of GFP_DMA should probably be converted),
but for example block layer bounce buffering might need a different
interface as I see you've proposed in another mail.

Hm... s390 driver seem to like GFP_DMA a lot... and there are a few
remaining uses in drivers/scsi... and then of course there are the OSS
drivers...

Jesse

2006-09-22 18:09:00

[permalink] [raw]

Subject: Re: ZONE_DMA

On Fri, 22 Sep 2006, Jesse Barnes wrote:

> Ok and right after I sent this my brain returned from vacation and I
> remembered jejb's DMA allocation API. It's powerful enough to cover most
> driver use cases I think (users of GFP_DMA should probably be converted),
> but for example block layer bounce buffering might need a different
> interface as I see you've proposed in another mail.

Could you dig that out and give us some refs or even better port that
thing to a current mm tree?

2006-09-22 18:25:54

[permalink] [raw]

Subject: Re: ZONE_DMA

On Friday, September 22, 2006 11:08 am, Christoph Lameter wrote:
> On Fri, 22 Sep 2006, Jesse Barnes wrote:
> > Ok and right after I sent this my brain returned from vacation and I
> > remembered jejb's DMA allocation API. It's powerful enough to cover
> > most driver use cases I think (users of GFP_DMA should probably be
> > converted), but for example block layer bounce buffering might need a
> > different interface as I see you've proposed in another mail.
>
> Could you dig that out and give us some refs or even better port that
> thing to a current mm tree?

Oh, it's already there in the tree, but obviously some drivers still need
to be converted. See Documentation/DMA-API.txt. It's not PCI specific
like the old PCI DMA interface (Documentation/DMA-mapping.txt) and
provides a way for drivers to specify their addressing limitations
(dma_supported and dma_set_mask), which allows the underlying architecture
code to report a failure if necessary.

I think many of the examples I cited can be converted to use the DMA API,
but block layer bounce buffering might need special treatment or perhaps a
way to get at the underlying struct device for the associated request...

Jesse

2006-09-22 18:27:09

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Dave Jones wrote:
> On Fri, Sep 22, 2006 at 12:29:21PM -0400, Jeff Garzik wrote:
> >
> > NOTE: Your mailer generates bogus Mail-Followup-To headers, and you
> > snipped rmk from the To/CC.
>
> Actually, my mailer just respected the Mail-Followup-To: that
> rmk set in his mail that I replied to. Nothing to see here.

Ah, I stand corrected. Hello, RMK. :)

Jeff

2006-09-22 18:27:51

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, 22 Sep 2006 11:48:16 -0400
Dave Jones <[email protected]> wrote:

> > As far as -mm getting these, I have asked Andrew to pull this tree in
> > the past, but whenever I rebase the trees (eg, when 2.6.18 comes out)
> > and fix up the rejects, Andrew seems to have a hard time coping. I
> > guess Andrew finds it too difficult to handle my devel branches.
>
> Has Andrew commented on why this is proving to be more of a problem?

I don't recall any particular ARM problems, so I'm curious too. I'm
presently pulling git+ssh://master.kernel.org/home/rmk/linux-2.6-arm.git,
which is coming up empty.

OTOH, I doubt if anyone runtime tests -mm on arm.

OTOH^2, people do cross-compile.

OTOH^3, my attempts to build an arm cross-compiler haven't been very
successful. I do have a .config which compiles, but allmodconfig used to
die due to the compiler not understanding weird `-m' options. <tries it>.
OK, that got fixed, so without CONFIG_AUDIT, arm allmodconfig compiles. Am
happy.

<I maintain that it is in the interests of obscure-arch maintainers to help
others build cross-compilers for their arch..>

<how'd I fall off the cc?>

<looks at viro>

include/asm-generic/audit_dir_write.h:9: error: `__NR_mkdirat' undeclared here (not in a function)
include/asm-generic/audit_dir_write.h:9: error: initializer element is not constant
include/asm-generic/audit_dir_write.h:9: error: (near initialization for `dir_class[8]')
include/asm-generic/audit_dir_write.h:10: error: `__NR_mknodat' undeclared here (not in a function)
include/asm-generic/audit_dir_write.h:10: error: initializer element is not constant
include/asm-generic/audit_dir_write.h:10: error: (near initialization for `dir_class[9]')
include/asm-generic/audit_dir_write.h:11: error: `__NR_unlinkat' undeclared here (not in a function)

2006-09-22 18:32:50

[permalink] [raw]

Subject: Re: ZONE_DMA

On Fri, 22 Sep 2006, Jesse Barnes wrote:

> Oh, it's already there in the tree, but obviously some drivers still need
> to be converted. See Documentation/DMA-API.txt. It's not PCI specific
> like the old PCI DMA interface (Documentation/DMA-mapping.txt) and
> provides a way for drivers to specify their addressing limitations
> (dma_supported and dma_set_mask), which allows the underlying architecture
> code to report a failure if necessary.

AFAICT this is dealing with special dma issues and not with the problem of
allocating memory for a certain supported address range from the page
allocator. From the first glance at the docs it looks as if it is relying
on __GFP_DMAxx to get the allocations right. I think the code could be
changed though to call a new page allocator function to get the right
memory and that would work for all devices using that API.

2006-09-22 18:39:26

[permalink] [raw]

Subject: Re: ZONE_DMA

On Friday, September 22, 2006 11:32 am, Christoph Lameter wrote:
> On Fri, 22 Sep 2006, Jesse Barnes wrote:
> > Oh, it's already there in the tree, but obviously some drivers still
> > need to be converted. See Documentation/DMA-API.txt. It's not PCI
> > specific like the old PCI DMA interface
> > (Documentation/DMA-mapping.txt) and provides a way for drivers to
> > specify their addressing limitations (dma_supported and dma_set_mask),
> > which allows the underlying architecture code to report a failure if
> > necessary.
>
> AFAICT this is dealing with special dma issues and not with the problem
> of allocating memory for a certain supported address range from the page
> allocator.

That's right. But OTOH device drivers shouldn't be using the page
allocator to get DMAable memory, they should be using one of the DMA APIs
since only they can portably map memory for DMA.

> From the first glance at the docs it looks as if it is
> relying on __GFP_DMAxx to get the allocations right. I think the code
> could be changed though to call a new page allocator function to get the
> right memory and that would work for all devices using that API.

Right, the internals are arch specific and don't necessarily have to rely
on a zone, depending on their DMA constraints.

Jesse

2006-09-22 18:41:20

[permalink] [raw]

Subject: Re: ZONE_DMA

On Fri, 22 Sep 2006, Jesse Barnes wrote:

> Right, the internals are arch specific and don't necessarily have to rely
> on a zone, depending on their DMA constraints.

>From what I can see the arch specific do some tricks and then pick GFP_DMA
or something to get memory that is appropriately limited. Having the
ability to retrieve pages from a certain range from the page allocator
would fix this issue and improve the ability of devices to allocate
memory.

2006-09-22 18:53:07

by Pavel Machek

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Hi!

> > ntp-move-all-the-ntp-related-code-to-ntpc.patch
> > ntp-move-all-the-ntp-related-code-to-ntpc-fix.patch
> > ntp-add-ntp_update_frequency.patch
> > ntp-add-ntp_update_frequency-fix.patch
> > ntp-add-time_adj-to-tick-length.patch
> > ntp-add-time_freq-to-tick-length.patch
> > ntp-prescale-time_offset.patch
> > ntp-add-time_adjust-to-tick-length.patch
> > ntp-remove-time_tolerance.patch
> > ntp-convert-time_freq-to-nsec-value.patch
> > ntp-convert-to-the-ntp4-reference-model.patch
> > ntp-cleanup-defines-and-comments.patch
> > kernel-time-ntpc-possible-cleanups.patch
> > kill-wall_jiffies.patch
> >
> > Will merge.
>
> would be nice to merge the -hrt queue that goes right ontop this queue.
> Even if HIGH_RES_TIMERS is "default n" in the beginning. That gives us
> high-res timers and dynticks which are both very important features to
> certain classes of users/devices.

dynticks give benefit of 0.3W, or 20minutes (IIRC) from 8hours on thinkpad
x60... and they were around for way too long. (When baseline is
hz=250, it is 0.5W from hz=1000 baseline). It would be cool to
finally merge them.

Pavel
--
Thanks for all the (sleeping) penguins.

2006-09-22 18:53:43

by Pavel Machek

[permalink] [raw]

Subject: release cycle (Re: 2.6.19 -mm merge plans)

Hi!

> > > If you think that shortening the release cycle will cause people to be more
> > > disciplined in their changes, to spend less time going berzerk and to spend
> > > more time working with our users and testers on known bugs then I'm all
> > > ears.
> >
> > Honestly, I do think it would be positive. It would shorten the
> > feedback loop, and get more changes out to testers.
> >
> > It would also decrease the pressure of the 60+ trees trying to get
> > everything in, because they know the next release is 3-4 months away.
> > It would be _much_ easier to say "break the generic device stuff in
> > 2.6.20 not 2.6.19, please" if we knew 2.6.20 wasn't going to be a 2007
> > release.
>
> Well, it might be worth trying. But there's a practical problem: how do we
> get there when there's so much work pending? If we skip some people's
> trees then they'll get sore, and it's not obvious that it'll help much, as
> the various trees are fairly unrelated (ie: parallelisable).

Well, slightly evil way would be 'if we find vfs changes in your ocfs
tree, well, you wait for one more release' :-).

Pavel

--
Thanks for all the (sleeping) penguins.

2006-09-22 19:06:20

[permalink] [raw]

Subject: Re: ZONE_DMA

On Friday, September 22, 2006 11:40 am, Christoph Lameter wrote:
> On Fri, 22 Sep 2006, Jesse Barnes wrote:
> > Right, the internals are arch specific and don't necessarily have to
> > rely on a zone, depending on their DMA constraints.
>
> From what I can see the arch specific do some tricks and then pick
> GFP_DMA or something to get memory that is appropriately limited. Having
> the ability to retrieve pages from a certain range from the page
> allocator would fix this issue and improve the ability of devices to
> allocate memory.

Right, being able to allocate from specific ranges would obviate the need
for GFP_DMA and the various zones. It would come with a cost though since
the VM would have to become aware of pressure at various ranges rather
than just on zones like we have now. I think that's where things get
tricky.

Jesse

2006-09-22 19:08:10

[permalink] [raw]

Subject: Re: ZONE_DMA

On Fri, 22 Sep 2006, Jesse Barnes wrote:

> Right, being able to allocate from specific ranges would obviate the need
> for GFP_DMA and the various zones. It would come with a cost though since
> the VM would have to become aware of pressure at various ranges rather
> than just on zones like we have now. I think that's where things get
> tricky.

Here is a hyperthetical use scenario (no code yet to do this in the page
allocator, fake patch for discussion only):

Index: linux-2.6.18-rc7-mm1/arch/i386/kernel/pci-dma.c
===================================================================
--- linux-2.6.18-rc7-mm1.orig/arch/i386/kernel/pci-dma.c 2006-09-12 20:41:36.000000000 -0500
+++ linux-2.6.18-rc7-mm1/arch/i386/kernel/pci-dma.c 2006-09-22 13:59:29.017611573 -0500
@@ -26,6 +26,8 @@ void *dma_alloc_coherent(struct device *
dma_addr_t *dma_handle, gfp_t gfp)
{
void *ret;
+ unsigned long low = 0L;
+ unsigned long high = 0xffffffff;
struct dma_coherent_mem *mem = dev ? dev->dma_mem : NULL;
int order = get_order(size);
/* ignore region specifiers */
@@ -44,10 +46,14 @@ void *dma_alloc_coherent(struct device *
return NULL;
}

- if (dev == NULL || (dev->coherent_dma_mask < 0xffffffff))
- gfp |= GFP_DMA;
+ if (dev == NULL)
+ /* Apply safe ISA LIMITS */
+ high = 16*1024*1024L;
+ else
+ if (dev->coherent_dma_mask < 0xffffffff)
+ high = dev->coherent_dma_mask;

- ret = (void *)__get_free_pages(gfp, order);
+ ret = page_address(alloc_pages_range(low, high, gfp, order));

if (ret != NULL) {
memset(ret, 0, size);
Index: linux-2.6.18-rc7-mm1/include/linux/gfp.h
===================================================================
--- linux-2.6.18-rc7-mm1.orig/include/linux/gfp.h 2006-09-19 09:26:58.000000000 -0500
+++ linux-2.6.18-rc7-mm1/include/linux/gfp.h 2006-09-22 14:02:34.298613635 -0500
@@ -136,6 +136,9 @@ static inline struct page *alloc_pages_n
NODE_DATA(nid)->node_zonelists + gfp_zone(gfp_mask));
}

+extern struct page *alloc_pages_range(unsigned long low, unsigned long high,
+ gfp_t gfp_mask, unsigned int order);
+
#ifdef CONFIG_NUMA
extern struct page *alloc_pages_current(gfp_t gfp_mask, unsigned order);

2006-09-22 19:09:43

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

* Pavel Machek <[email protected]> wrote:

> > would be nice to merge the -hrt queue that goes right ontop this
> > queue. Even if HIGH_RES_TIMERS is "default n" in the beginning. That
> > gives us high-res timers and dynticks which are both very important
> > features to certain classes of users/devices.
>
> dynticks give benefit of 0.3W, or 20minutes (IIRC) from 8hours on
> thinkpad x60... and they were around for way too long. (When baseline
> is hz=250, it is 0.5W from hz=1000 baseline). It would be cool to
> finally merge them.

note that this is a new implementation of dynticks though, not Con's
older stuff which you probably used, right? But it's fairly low-impact
(just a few lines ontop of hrtimers, here and there), ontop of the
long-existing -hrt queue.

Ingo

2006-09-22 20:08:42

by Roland Dreier

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Linus> And then at some point (probably later today) I decide to
Linus> go into "merge mode" and go back to old mails I ignored and
Linus> start applying them and pulling from other peoples git
Linus> trees. And so if my "mode switching" has a longer latency
Linus> than the "please pull" frequency, I end up seeing two
Linus> requests for the same tree during the same "merge mode"
Linus> thing, which just means that when I look at the older one,
Linus> it no longer matches what is in the tree I'm pulling from.

My way of handling this has been to wait until you've acted on my
first merge request before sending another one. I also don't touch my
published "for-linus" branch in git until you've pulled it. I just
batch up pending changes in my "for-2.6.19" branch until my next merge
(and I also encourage people interested in Infiniband to run my
for-2.6.19 branch)

If I do see you merging other trees (in "merge mode") and skipping my
merge requests, I'll send a gentle reminder of my first merge request.

- R.

2006-09-22 20:13:38

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, Sep 22, 2006 at 01:06:48PM +0000, Pavel Machek wrote:

> > would be nice to merge the -hrt queue that goes right ontop this queue.
> > Even if HIGH_RES_TIMERS is "default n" in the beginning. That gives us
> > high-res timers and dynticks which are both very important features to
> > certain classes of users/devices.
>
> dynticks give benefit of 0.3W, or 20minutes (IIRC) from 8hours on thinkpad
> x60... and they were around for way too long. (When baseline is
> hz=250, it is 0.5W from hz=1000 baseline). It would be cool to
> finally merge them.

I actually saw much bigger wins when I tested with an Athlon XP based compaq laptop
a year or so back. dynticks moved idle from being stuck at 21W to a sinusoidal
cycle in single watt increments between 22W->18W. It would never stay in the
lower ranges for long because of timers firing all the time.

See http://kernelslacker.livejournal.com/33637.html for details.

There is much interest right now in fixing up various bits of userspace
to not do braindead things with timers/polling. The gnome people
have recently come up with a timertop-esque hack (that goes a bit further)
for eg. See http://blogs.gnome.org/ryanl for details.
Arjan also recently did battle with a huge amount of really dumb userspace,
and dwmw2 has been tracking a bunch of these for OLPC:
https://bugzilla.redhat.com/bugzilla/showdependencytree.cgi?id=204948

Damn all these busy people making me feel inadequate :)

Dave

2006-09-22 20:22:55

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Roland Dreier wrote:
> My way of handling this has been to wait until you've acted on my
> first merge request before sending another one. I also don't touch my
> published "for-linus" branch in git until you've pulled it. I just
> batch up pending changes in my "for-2.6.19" branch until my next merge
> (and I also encourage people interested in Infiniband to run my
> for-2.6.19 branch)

That's pretty much what I do. I run a
git branch upstream-linus upstream

and then submit a pull request for the upstream-linus branch. That way,
I can keep working and committing stuff, and don't have to wait for
Linus to pull.

Then, after the pull, I delete the branch
git branch -D upstream-linus

locally, and repeat the process next time a bunch of changes are queued up.

Jeff

2006-09-22 20:29:51

by Bill Rugolsky Jr.

[permalink] [raw]

Subject: hires timer patchset [was Re: 2.6.19 -mm merge plans]

On Fri, Sep 22, 2006 at 09:01:06PM +0200, Ingo Molnar wrote:
>
> * Pavel Machek <[email protected]> wrote:
>
> > > would be nice to merge the -hrt queue that goes right ontop this
> > > queue. Even if HIGH_RES_TIMERS is "default n" in the beginning. That
> > > gives us high-res timers and dynticks which are both very important
> > > features to certain classes of users/devices.
> >
> > dynticks give benefit of 0.3W, or 20minutes (IIRC) from 8hours on
> > thinkpad x60... and they were around for way too long. (When baseline
> > is hz=250, it is 0.5W from hz=1000 baseline). It would be cool to
> > finally merge them.
>
> note that this is a new implementation of dynticks though, not Con's
> older stuff which you probably used, right? But it's fairly low-impact
> (just a few lines ontop of hrtimers, here and there), ontop of the
> long-existing -hrt queue.

Just a data point, FWIW; I've made no systematic effort to find regressions.

We've been running Thomas's (pre-dynticks) 2.6.16-hrt patchset (and
HZ=1000) without issue as part of my standard kernel build on x86 and
x86_64 UP/SMP production hosts -- workstations, PostgreSQL (and other)
servers, and routers --- since I experienced latency/ntp problems with
an unpatched kernel using sata_nv on Tyan 2895 Nvidia CK804-chipset mobo
back in March 2006. I've also been running the (originally x86-only)
2.6.17-dynticks patch on a Tyan Athlon SMP workstation mobo, and occasionally
on my laptop, so far without issue.

I originally chose Thomas's -hrt variant of John Stultz's timekeeping
patchset because it had known-working x86_64 support ...

We're not doing anything exotic (such as audio, packet capture,
or serving thousands of connections) that might stress the timer or
clock system much, but the x86 routers have GigE traffic and the
x86_64 PostgreSQL servers are often heavily loaded.

Regards,

Bill Rugolsky

2006-09-23 02:18:07

[permalink] [raw]

Subject: Re: hires timer patchset [was Re: 2.6.19 -mm merge plans]

Sorry about dropping CC-s. I'm reading archived lkml and can't see them.
Adding the obvious candidates instead (Andrew, you're not obvious, but if
you'll pull in Thomas's stuff, you'll be...)

On 2006-09-22 20:29:40 Bill Rugolsky Jr. wrote:
> On Fri, Sep 22, 2006 at 09:01:06PM +0200, Ingo Molnar wrote:
> >
> > * Pavel Machek wrote:
> >
> > > > would be nice to merge the -hrt queue that goes right ontop
[...]
> > note that this is a new implementation of dynticks though, not Con's
> > older stuff which you probably used, right? But it's fairly
> > low-impact (just a few lines ontop of hrtimers, here and there),
> > ontop of the long-existing -hrt queue.
>
> Just a data point, FWIW; I've made no systematic effort to find
> regressions.
>
> We've been running Thomas's (pre-dynticks) 2.6.16-hrt patchset (and
> HZ=1000) without issue as part of my standard kernel build on x86 and
> x86_64 UP/SMP production hosts -- workstations, PostgreSQL (and other)
> servers, and routers --- since I experienced latency/ntp problems with
> an unpatched kernel using sata_nv on Tyan 2895 Nvidia CK804-chipset
> mobo back in March 2006. I've also been running the (originally
> x86-only) 2.6.17-dynticks patch on a Tyan Athlon SMP workstation mobo,
> and occasionally on my laptop, so far without issue.

Here's another data point: I tried 2.6.18-rt3 today on a x86_64
notebook. I'm on an eternal quest for extended battery time, so
NO_HZ would be perfect. Long story shortened, HIGH_RES_TIMERS
(prerequisite for NO_HZ) caused the CPU to never step down from max
speed (ondemand, powernow_k8) at 2200 MHz.

In addition, something invisible used very frequent bursts of ~30% SYS
CPU. Turning from PREEMPT_RT to PREEMPT_DESKTOP introduced occasional
bursts of ~50% USER CPU (mixed with the SYS). Toggling RCU model
made no difference.

Going from the default HIGH_RES_RESOLUTION=1000 to 10000 and even
100000 had no impact on the level or frequency of the bursts.

This resulted in the CPU temp never going lower than 53C, and thusly
the fan kept spinning at 'level 2' - it needs 49C to fall back to
normal speed. That sound tend to drive me insane...

Turning off HIGH_RES_TIMER allowed the CPU to enter the lowest 800 MHz
immediately, and fan to slow down after the normal time, post-boot.

I'd be glad to help investigate this issue, but since -rt is completely
new territory I'd need exact instructions about which knobs to turn.
The CPU usage cause _was_ invisible, at least to "top". And I had all
USB stuff unplugged, and no ehci_hcd loaded (that's another story,
written in the bugzilla). Completely plain console, no X, no net, no
nothing.

Here's a snippet from dmesg pertaining to the timers:

ACPI: PM-Timer IO Port: 0x4008
[...]
Event source pit configured with caps set: 07
time.c: Detected 2201.306 MHz processor.
[...]
Calibrating delay using timer specific routine.. 4404.80 BogoMIPS
(lpj=2202402)
[...]
Using local APIC timer interrupts.
result 12507441
Detected 12.507 MHz APIC timer.
lapic max_delta_ns: 670689262
Event source pit new caps set: 03
Event source lapic configured with caps set: 04
[...]
Real Time Clock Driver v1.12ac
[...]
Time: tsc clocksource has been installed.
Event source pit disabled
Event source lapic configured with caps set: 08
hrtimers: Switched to high resolution mode CPU 0

Oh, and here's a compiler warning fix (when PREEMPT_RT is off):

diff -Nurp linux-2.6.18-rt3/kernel/hrtimer.c linux-2.6.18-rt3-db/kernel/hrtimer.c
--- linux-2.6.18-rt3/kernel/hrtimer.c 2006-09-23 01:12:41.000000000 +0200
+++ linux-2.6.18-rt3-db/kernel/hrtimer.c 2006-09-23 02:16:29.000000000 +0200
@@ -786,7 +786,7 @@ static inline void hrtimer_raise_softirq
raise_softirq_irqoff(HRTIMER_SOFTIRQ);
}

-static inline int hrtimer_adjust_softirq_prio(struct hrtimer_cpu_base *base)
+static inline void hrtimer_adjust_softirq_prio(struct hrtimer_cpu_base *base)
{
}

Mvh
Mats Johannesson

2006-09-23 08:11:35

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, 21 Sep 2006 00:22:26 -0400
Jeff Garzik <[email protected]> wrote:

> > There are quite a lot of patches here which belong in subsystem trees.
> > I'll patchbomb the relevant maintainers soon. Could I pleeeeeze ask that
> > they either merge the patches or solidly nack them (with reasons)? Don't
> > just ignore it all and leave me hanging onto this stuff for ever. Thanks.
>
> I know this is probably heresy, but what would happen if we didn't merge
> all that stuff at once, and then committed to having a real 4-week cycle?
>
> The cycles seem to be stretching out again, and I don't really think
> it's worth it to hold up the entire kernel for every single piddly
> little regression to get fixed. We'll _never_ be perfect, even if we
> weren't slackers.

It's remarkable how one can wait for months for the next kernel release,
carefully lining up, testing and reviewing the patches for the next
release, then whoop! As soon as Linus cuts a new release all sorts of
hitherto unimagined stuff comes out of the woodwork and gets instaslammed
into mainline.

WARNING: "register_mtd_blktrans" [drivers/mtd/rfd_ftl.ko] undefined!
WARNING: "deregister_mtd_blktrans" [drivers/mtd/rfd_ftl.ko] undefined!
WARNING: "add_mtd_blktrans_dev" [drivers/mtd/rfd_ftl.ko] undefined!
WARNING: "del_mtd_blktrans_dev" [drivers/mtd/rfd_ftl.ko] undefined!
WARNING: "register_mtd_blktrans" [drivers/mtd/nftl.ko] undefined!
WARNING: "add_mtd_blktrans_dev" [drivers/mtd/nftl.ko] undefined!
WARNING: "deregister_mtd_blktrans" [drivers/mtd/nftl.ko] undefined!
WARNING: "del_mtd_blktrans_dev" [drivers/mtd/nftl.ko] undefined!
WARNING: "register_mtd_blktrans" [drivers/mtd/mtdblock_ro.ko] undefined!
WARNING: "add_mtd_blktrans_dev" [drivers/mtd/mtdblock_ro.ko] undefined!
WARNING: "del_mtd_blktrans_dev" [drivers/mtd/mtdblock_ro.ko] undefined!
WARNING: "deregister_mtd_blktrans" [drivers/mtd/mtdblock_ro.ko] undefined!
WARNING: "register_mtd_blktrans" [drivers/mtd/mtdblock.ko] undefined!
WARNING: "add_mtd_blktrans_dev" [drivers/mtd/mtdblock.ko] undefined!
WARNING: "del_mtd_blktrans_dev" [drivers/mtd/mtdblock.ko] undefined!
WARNING: "deregister_mtd_blktrans" [drivers/mtd/mtdblock.ko] undefined!
WARNING: "register_mtd_blktrans" [drivers/mtd/inftl.ko] undefined!
WARNING: "add_mtd_blktrans_dev" [drivers/mtd/inftl.ko] undefined!
WARNING: "deregister_mtd_blktrans" [drivers/mtd/inftl.ko] undefined!
WARNING: "del_mtd_blktrans_dev" [drivers/mtd/inftl.ko] undefined!
WARNING: "register_mtd_blktrans" [drivers/mtd/ftl.ko] undefined!
WARNING: "deregister_mtd_blktrans" [drivers/mtd/ftl.ko] undefined!
WARNING: "add_mtd_blktrans_dev" [drivers/mtd/ftl.ko] undefined!
WARNING: "del_mtd_blktrans_dev" [drivers/mtd/ftl.ko] undefined!

Please don't do this.

Quick review:

- search for "( " and " )", fix.

- Might want to do something about the 512-byte automatic variable in
get_valid_cis_sector()

- It's full of uint8_t drain bamage.

2006-09-23 08:29:57

by Ryan Anderson

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, Sep 22, 2006 at 04:22:43PM -0400, Jeff Garzik wrote:
> Roland Dreier wrote:
> >My way of handling this has been to wait until you've acted on my
> >first merge request before sending another one. I also don't touch my
> >published "for-linus" branch in git until you've pulled it. I just
> >batch up pending changes in my "for-2.6.19" branch until my next merge
> >(and I also encourage people interested in Infiniband to run my
> >for-2.6.19 branch)
>
>
> That's pretty much what I do. I run a
> git branch upstream-linus upstream
>
> and then submit a pull request for the upstream-linus branch. That way,
> I can keep working and committing stuff, and don't have to wait for
> Linus to pull.
>
> Then, after the pull, I delete the branch
> git branch -D upstream-linus
>
> locally, and repeat the process next time a bunch of changes are queued up.

Just in case anyone else is reading along and absorbing Git commands,
please use:
git branch -d upstream-linus
which will at least tell you if the commits in that tree aren't in your
current tree before blowing away work in hard-to-recover ways.

(Not necessarily aimed at Jeff.)

--

Ryan Anderson
sometimes Pug Majere

2006-09-23 09:53:08

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Sat, 2006-09-23 at 01:06 -0700, Andrew Morton wrote:
> WARNING: "register_mtd_blktrans" [drivers/mtd/rfd_ftl.ko] undefined!
> WARNING: "deregister_mtd_blktrans" [drivers/mtd/rfd_ftl.ko] undefined!
> WARNING: "add_mtd_blktrans_dev" [drivers/mtd/rfd_ftl.ko] undefined!
> WARNING: "del_mtd_blktrans_dev" [drivers/mtd/rfd_ftl.ko] undefined!

Ah, I hadn't realised that 'bool FOO depends on tristate BAR' would
allow FOO=y, BAR=m. And since we don't recurse into drivers/mtd at all
when building vmlinux and CONFIG_MTD=m, setting CONFIG_SSFDC=y would
mean that it _thought_ it was building mtd_blkdevs.c into the kernel but
in fact it wasn't. I hadn't tested that combination, since it makes no
sense -- yet it's what allmodconfig does. Sorry 'bout that.

I've fixed this by making SSFDC a tristate, and I've fixed it
differently for CMDLINEPARTS which has had the same problem for ages.

> Quick review:
>
> - search for "( " and " )", fix.

Not sure which you're referring to -- there's a NULL-terminated array of
strings which seems just fine as it is, and there's a size vs. C-H-S
table. The latter could _possibly_ use C99 initialisers, but for
something so simple I think it would just get in the way. It's an
entirely local structure anyway -- it's defined just above the table.
We don't have to worry about the initialiser getting out of sync with
the structure definition, so I didn't see the point in mandating C99
initialisers for it, since Claudio chose to do otherwise. It's his
choice, as far as I'm concerned.

I did notice that he misspelled 'MiB' and fix that though :)

> - Might want to do something about the 512-byte automatic variable in
> get_valid_cis_sector()

Done. Thanks for spotting that.

--
dwmw2

2006-09-23 10:44:36

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, 2006-09-22 at 10:24 +0100, David Woodhouse wrote:
> Merged, with the exception of the unlock addr one which I'm still not
> sure about -- about to investigate harder.

Please drop it (fix-the-unlock-addr-lookup-bug-in-mtd-jedec-probe.patch).

Returning the first value in the array is intentional -- it should be
automatically shifted to make it work if it's a x16 chip, and the right
thing should happen. If _that_ isn't working, give details and we'll
work out a proper fix.

--
dwmw2

2006-09-23 11:08:01

by Pavel Machek

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Hi!

> > > would be nice to merge the -hrt queue that goes right ontop this
> > > queue. Even if HIGH_RES_TIMERS is "default n" in the beginning. That
> > > gives us high-res timers and dynticks which are both very important
> > > features to certain classes of users/devices.
> >
> > dynticks give benefit of 0.3W, or 20minutes (IIRC) from 8hours on
> > thinkpad x60... and they were around for way too long. (When baseline
> > is hz=250, it is 0.5W from hz=1000 baseline). It would be cool to
> > finally merge them.
>
> note that this is a new implementation of dynticks though, not Con's
> older stuff which you probably used, right? But it's fairly low-impact

(Well, I cheated, and just measured difference between HZ=100 and
HZ=250/1000. Dynticks should be even slightly better than that.)
Pavel

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

2006-09-23 13:22:35

by Wu Fengguang

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thu, Sep 21, 2006 at 04:59:18PM -0700, Andrew Morton wrote:
> > On Wed, Sep 20, 2006 at 01:54:38PM -0700, Andrew Morton wrote:
> > >
> > > The readahead code is complex, I'm unconvinced that it has a lot of benefit
> > > and Wu has gone quiet. Will drop.
> >
> > Sorry, I've been putting efforts to meet the deadline of the google
> > SoC project "Rapid linux desktop startup through pre-caching", which
> > still can not be called success for now. And there's my pending paper
> > work...
>
> Oh, OK.
>
> That's a neat thing to be working on.

Thank you.

FYI: I attached a little presentation work, one for the boot time
stuff, another for the readahead patch.

> > I should be able to come back and concentrate on the readahead patch
> > after one month, whether it be dropped for now. I guess it can be
> > further improved in complexity and performance. It also needs a good
> > document for the overall design and benefits. And sure the
> > performance numbers :-)
>
> I'll hang onto the patches then - they're causing little maintenance
> trouble for me.

Sorry, I can imagine it, which upsets me.
I'll try to settle it in the next 3-month cycle.

Thanks,
Fengguang Wu

Attachments:

(No filename) (1.16 kB)
boot_linux_faster.pdf (376.98 kB)
adaptive_readahead.pdf (133.25 kB)
Download all attachments

2006-09-23 14:48:15

by Gene Heskett

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Thursday 21 September 2006 20:32, Fengguang Wu wrote:
>On Thu, Sep 21, 2006 at 04:59:18PM -0700, Andrew Morton wrote:
>> > On Wed, Sep 20, 2006 at 01:54:38PM -0700, Andrew Morton wrote:
>> > > The readahead code is complex, I'm unconvinced that it has a lot
>> > > of benefit and Wu has gone quiet. Will drop.
>> >
>> > Sorry, I've been putting efforts to meet the deadline of the google
>> > SoC project "Rapid linux desktop startup through pre-caching", which
>> > still can not be called success for now. And there's my pending paper
>> > work...
>>
>> Oh, OK.
>>
>> That's a neat thing to be working on.
>
>Thank you.
>
>FYI: I attached a little presentation work, one for the boot time
>stuff, another for the readahead patch.
>
>> > I should be able to come back and concentrate on the readahead patch
>> > after one month, whether it be dropped for now. I guess it can be
>> > further improved in complexity and performance. It also needs a good
>> > document for the overall design and benefits. And sure the
>> > performance numbers :-)
>>
>> I'll hang onto the patches then - they're causing little maintenance
>> trouble for me.
>
>Sorry, I can imagine it, which upsets me.
>I'll try to settle it in the next 3-month cycle.
>
>Thanks,
>Fengguang Wu

Both of your attached pdf's were missing the required fonts to display the
content on this locale=en machine, as the latest acroread for linux (7.05)
advised me about both, and the second one, while it did display the cover
page in english at a scale far larger than my 1600x1200 screen, it
actually locked up X and I had to reboot. I thought I had enough fonts
here to cover everything too. :(

--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Yahoo.com and AOL/TW attorneys please note, additions to the above
message by Gene Heskett are:
Copyright 2006 by Maurice Eugene Heskett, all rights reserved.

2006-09-23 15:25:59

[permalink] [raw]

Subject: Re: hires timer patchset [was Re: 2.6.19 -mm merge plans]

Andi,

See http://marc.theaimsgroup.com/?l=linux-kernel&m=115897798118500&w=2
for my original report.

I've now built without NO_HZ (but still HIGH_RES_TIMERS) and that fixes
the issue. You've got most of my machine specs since previous, even an
output from dmidecode, but feel free to ask for more if this bug
interests you.

I'm not particularly eager to get it solved... the nvidia driver still
needs one more hack for the glue to build against -rt3.

Mvh
Mats Johannesson

2006-09-23 16:34:46

by Andi Kleen

[permalink] [raw]

Subject: Re: hires timer patchset [was Re: 2.6.19 -mm merge plans]

On Sat, Sep 23, 2006 at 05:25:17PM +0200, Voluspa wrote:
> Andi,
>
> See http://marc.theaimsgroup.com/?l=linux-kernel&m=115897798118500&w=2
> for my original report.
>
> I've now built without NO_HZ (but still HIGH_RES_TIMERS) and that fixes
> the issue. You've got most of my machine specs since previous, even an
> output from dmidecode, but feel free to ask for more if this bug
> interests you.
>
> I'm not particularly eager to get it solved... the nvidia driver still
> needs one more hack for the glue to build against -rt3.

-rt* bugs are handled by Ingo and Thomas.

-Andi

2006-09-23 18:09:33

by Daniel Walker

[permalink] [raw]

Subject: Re: hires timer patchset [was Re: 2.6.19 -mm merge plans]

On Sat, 2006-09-23 at 04:17 +0200, Voluspa wrote:

> Here's another data point: I tried 2.6.18-rt3 today on a x86_64
> notebook. I'm on an eternal quest for extended battery time, so
> NO_HZ would be perfect. Long story shortened, HIGH_RES_TIMERS
> (prerequisite for NO_HZ) caused the CPU to never step down from max
> speed (ondemand, powernow_k8) at 2200 MHz.
>
> In addition, something invisible used very frequent bursts of ~30% SYS
> CPU. Turning from PREEMPT_RT to PREEMPT_DESKTOP introduced occasional
> bursts of ~50% USER CPU (mixed with the SYS). Toggling RCU model
> made no difference.
>

It seems like you don't need all of 2.6.18-rt3 , you just want dynamic
tick .. You can obtain just the HRT/dynamic tick patch from here,

http://www.tglx.de/projects/hrtimers/2.6.18/

Daniel

2006-09-23 19:22:48

[permalink] [raw]

Subject: Re: hires timer patchset [was Re: 2.6.19 -mm merge plans]

On Sat, 23 Sep 2006 11:09:26 -0700 Daniel Walker wrote:
> On Sat, 2006-09-23 at 04:17 +0200, Voluspa wrote:
>
> > Here's another data point: I tried 2.6.18-rt3 today on a x86_64
> > notebook. I'm on an eternal quest for extended battery time, so
> > NO_HZ would be perfect. Long story shortened, HIGH_RES_TIMERS
> > (prerequisite for NO_HZ) caused the CPU to never step down from max
> > speed (ondemand, powernow_k8) at 2200 MHz.
> >
> > In addition, something invisible used very frequent bursts of ~30%
> > SYS CPU. Turning from PREEMPT_RT to PREEMPT_DESKTOP introduced
> > occasional bursts of ~50% USER CPU (mixed with the SYS). Toggling
> > RCU model made no difference.
> >
>
> It seems like you don't need all of 2.6.18-rt3 , you just want dynamic
> tick .. You can obtain just the HRT/dynamic tick patch from here,
>
> http://www.tglx.de/projects/hrtimers/2.6.18/

Thank you pointing that out. I didn't realise that the features were so
sharply divided. Perhaps I'll even be able to debug it locally, then -
wearing ear-mufflers. I'm not kidding about the fan noise.

At the very least the experiment will tell if it's a -rt combo problem
or NO_HZ on its own.

Mvh
Mats Johannesson

2006-09-23 19:58:49

[permalink] [raw]

Subject: Re: hires timer patchset [was Re: 2.6.19 -mm merge plans]

On Sat, 23 Sep 2006 11:09:26 -0700 Daniel Walker wrote:
> On Sat, 2006-09-23 at 04:17 +0200, Voluspa wrote:
[...]
> It seems like you don't need all of 2.6.18-rt3 , you just want dynamic
> tick .. You can obtain just the HRT/dynamic tick patch from here,
>
> http://www.tglx.de/projects/hrtimers/2.6.18/

I'm not good enough at C yet to immediately see what needs fixing. And
the hour is too late anyways:

BUILD arch/x86_64/boot/bzImage
Root device is (3, 2)
Boot sector 512 bytes.
Setup is 4709 bytes.
System is 1461 kB
Kernel: arch/x86_64/boot/bzImage is ready (#1)
Building modules, stage 2.
MODPOST
[usual ACPI section mismatch warning deleted]
WARNING: "monotonic_clock" [drivers/char/hangcheck-timer.ko] undefined!
WARNING: "hrtimer_stop_sched_tick" [drivers/acpi/processor.ko] undefined!
WARNING: "hrtimer_restart_sched_tick" [drivers/acpi/processor.ko] undefined!
CC arch/x86_64/kernel/cpufreq/powernow-k8.mod.o
LD [M] arch/x86_64/kernel/cpufreq/powernow-k8.ko

Mvh
Mats Johannesson

2006-09-23 20:28:17

[permalink] [raw]

Subject: Re: hires timer patchset [was Re: 2.6.19 -mm merge plans]

* Voluspa <[email protected]> wrote:

> WARNING: "monotonic_clock" [drivers/char/hangcheck-timer.ko] undefined!

turn off the CONFIG_HANGCHECK_TIMER option.

> WARNING: "hrtimer_stop_sched_tick" [drivers/acpi/processor.ko] undefined!
> WARNING: "hrtimer_restart_sched_tick" [drivers/acpi/processor.ko] undefined!

add these two lins to the end of kernel/hrtimer.c:

EXPORT_SYMBOL_GPL(hrtimer_stop_sched_tick);
EXPORT_SYMBOL_GPL(hrtimer_restart_sched_tick);

Ingo

2006-09-24 02:36:55

[permalink] [raw]

Subject: Re: hires timer patchset [was Re: 2.6.19 -mm merge plans]

On Sat, 23 Sep 2006 22:20:27 +0200 Ingo Molnar wrote:
>
> * Voluspa wrote:
>
> > WARNING: "monotonic_clock" [drivers/char/hangcheck-timer.ko]
> > undefined!
>
> turn off the CONFIG_HANGCHECK_TIMER option.
>
> > WARNING: "hrtimer_stop_sched_tick" [drivers/acpi/processor.ko]
> > undefined! WARNING:
> > "hrtimer_restart_sched_tick" [drivers/acpi/processor.ko] undefined!
>
> add these two lins to the end of kernel/hrtimer.c:
>
> EXPORT_SYMBOL_GPL(hrtimer_stop_sched_tick);
> EXPORT_SYMBOL_GPL(hrtimer_restart_sched_tick);

My mind was clouded close to bedtime. I now remember the fix that
was published already at 2.6.17 time, from Steven Rostedt:

http://marc.theaimsgroup.com/?l=linux-kernel&m=115124086410874&w=2

Well, result is that NO_HZ indeed is the culprit for this CPU
issue. /proc/interrupts showed the timer to be stuck on an initial 3044
triggers after boot, while NMI: counted up almost as fast as LOC: (if
that tells any tale). Observing "top -d 1" for awhile revealed SYS
bursting (almost regularly alternating) between 50% and 100% CPU each 2
to 3 seconds. In between it was 0. USER also had the same pattern, but
with much longer duration. Perhaps 10 seconds from one show to the next.

I've gotten the broken out hrt-dyntick1 patches so will be able to
experiment on my own - slowly, on spare time.

I am of course available for any thoughts or trials you can come up
with in the meantime.

Mvh
Mats Johannesson

2006-09-24 02:42:27

by Wu Fengguang

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Sat, Sep 23, 2006 at 10:48:13AM -0400, Gene Heskett wrote:
> Both of your attached pdf's were missing the required fonts to display the
> content on this locale=en machine, as the latest acroread for linux (7.05)
> advised me about both, and the second one, while it did display the cover

Thanks for the feedback. I guess the main cause can be from a default
Chinese font, which is actually not used for the visible characters.
It should not cause serious readability issues.

> page in english at a scale far larger than my 1600x1200 screen, it
> actually locked up X and I had to reboot. I thought I had enough fonts
> here to cover everything too. :(

Sorry for that. I'll send a copy of .odp for you.

Regards,
Wu

2006-09-24 06:33:10

by Lennert Buytenhek

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, Sep 22, 2006 at 11:26:49AM -0700, Andrew Morton wrote:

> <I maintain that it is in the interests of obscure-arch maintainers
> to help others build cross-compilers for their arch..>

As I regularly test with different gcc versions and encourage others
to do the same, I've had a set available for a while at:

http://www.wantstofly.org/~buytenh/kernel/arm-cross/

(Generated with crosstool 0.42, see http://kegel.com/crosstool) Any
suggestions for making them easier to find for folks?

cheers,
Lennert

2006-09-24 07:48:40

by Lennert Buytenhek

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Fri, Sep 22, 2006 at 09:21:32AM -0700, Linus Torvalds wrote:

> > Hmm. Some trees do seem to get pulled more often than others.
> > Linus, is there a upper limit on the number of times you want
> > to see pull requests? It strikes me as odd, so I'm wondering
> > if there are some crossed wires here.
>
> I personally prefer to not see _too_ many pull requests, since that
> to me indicates that people don't take advantage of the distributed
> nature of git, and don't let things "simmer" in their own tree for
> a while.

ARM has many sub-architectures, and patches for each of the sub-archs
typically (not always) come from the same person in ARM land, which
typically means that the total set of ARM changes doesn't really need
much integration testing as a whole.

Changes that span sub-architectures (such as genirq) are usually
discussed on the mailing list first, and tested before they get
committed to any tree. I.e. on the few occasions that we do need
to test ARM-wide changes, we just email patches around rather than
asking folks "whether 2.6.30-rmk42 works."

Due to this, by the time a patch gets sent to rmk it's Obviously
Perfect, and the remaining testing work is 99% integration testing
with concurrent changes to other subsystems -- mtd, netdev, usb, etc.

In fact, I'd argue that there are more patches that span arch/arm plus
some other part of the tree (inter-related mtd bits, netdev bits, usb
bits, framebuffer bits), than there are patches that span different ARM
sub-architectures.

For example, a driver for the ethernet MAC in the ARM cpu that I have
here was recently applied by jgarzik, but I cannot submit the arch/arm
hooks to make use of this driver until jgarzik's tree is merged upstream
_and_ rmk rebases the half a megabyte of pending ARM changes on this
tree. If I submit the hooks to rmk when the driver goes upstream, rmk's
private tree (probably still based on 2.6.18 when that happens, so won't
have the relevant netdev changes) will break.

To make making these kinds of changes easier without sending stuff
upstream so regularly, rmk would have to either pull from upstream
regularly or rebase his tree on upstream regularly, both of which
don't seem like desirable options.

Also, I do want to track upstream git, as every now and then changes
are made upstream that break ARM, and we want to be able to detect that
quickly.

Due to the nature of ARM work, rmk maintaining a long-lived tree for
stuff to 'simmer in' would probably reduce the burden on Linus but
would not have much of an advantage otherwise, IMHO.

cheers,
Lennert

2006-09-24 09:20:23

by Russell King

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Sun, Sep 24, 2006 at 09:48:37AM +0200, Lennert Buytenhek wrote:
> On Fri, Sep 22, 2006 at 09:21:32AM -0700, Linus Torvalds wrote:
> > I personally prefer to not see _too_ many pull requests, since that
> > to me indicates that people don't take advantage of the distributed
> > nature of git, and don't let things "simmer" in their own tree for
> > a while.
>
> Changes that span sub-architectures (such as genirq) are usually
> discussed on the mailing list first, and tested before they get
> committed to any tree. I.e. on the few occasions that we do need
> to test ARM-wide changes, we just email patches around rather than
> asking folks "whether 2.6.30-rmk42 works."

Let's look at how two scenarios work for the case of the genirq patches.

1. genirq remains as patches until it's ready (iow, what actually happened)
- patches as a series were posted to the mailing list with a request
to test.
- feedback came, the patches were improved, and this repeated several
times
- eventually, people are happy
- in parallel, because these patches affect other architectures, they
were also submitted to -mm.
- the generic parts of genirq were submitted to Linus.
- once the generic parts of genirq are in mainline, the ARM specific
parts can then be applied to my git tree, and be sent to Linus.

2. genirq patches in ARM git tree.
- the initial entire patches get applied to the ARM tree on a
separate branch.
- the patches get mailed to the ARM mailing list as one large patch.
- feedback comes to me/mailing list rather than CC:'d to the genirq
people.
- maybe the genirq people send updates to me, which then need to be
applied to the ARM tree, and the result needs to be re-published.
- we go around the loop several times, maybe merging in later mainline
versions.
- in parallel with this, the non-ARM parts of genirq are submitted to
-mm.
- when the patches are ready, the generic parts are submitted to
Linus.
- rmk then faces a _big_ problem in resolving lots of conflicts in
his tree, because the generic parts submitted to Linus are different
from the initial generic parts committed to the ARM git tree.
- the ARM git tree for genirq probably has to be rebuilt from scratch.

Sure, someone else _could_ maintain genirq in their own git tree, so
replace "rmk" above with "tglx". The overall problem remains though -
at the end of it you can't push that git tree with all that history,
and the whole thing can't go via just one person. This in turn means
that there's a chunk of additional work to rework the changes (which
may introduce bugs) and then an additional round of re-testing to make
sure nothing broke.

The point I'm making is that for some things, keeping the changes as
patches until they're ready is far easier, more worthwhile and flexible
than having them simmering in some git tree somewhere.

> For example, a driver for the ethernet MAC in the ARM cpu that I have
> here was recently applied by jgarzik, but I cannot submit the arch/arm
> hooks to make use of this driver until jgarzik's tree is merged upstream
> _and_ rmk rebases the half a megabyte of pending ARM changes on this
> tree. If I submit the hooks to rmk when the driver goes upstream, rmk's
> private tree (probably still based on 2.6.18 when that happens, so won't
> have the relevant netdev changes) will break.
>
> To make making these kinds of changes easier without sending stuff
> upstream so regularly, rmk would have to either pull from upstream
> regularly or rebase his tree on upstream regularly, both of which
> don't seem like desirable options.

I believe that Linus has complained to tree maintainers when they've
done this because it complicates the history far too much.

Consider the effect on the history if I were to pull Linus' tree daily
into my trees, and I was committing about one change per day. You'd
have:

linus---othercommits--------othercommits---------othercommits-------merge
\ \ \ \ /
commit --- merge --- commit --- merge --- commit --- merge --- commit

The other solution is that I somehow start reviewing drivers for any
part of the kernel and accepting them. However, as we've seen already
on lkml, if anyone who isn't jgarzik accepts a change to a network
device driver and passes it to Linus, they get a roasting from Jeff
(see Dominic's incident when adding some additional PCMCIA IDs to PCMCIA
network device drivers.)

>From the point of view of keeping an ARM tree, I've been in that situation
before - from the beginnings of ARM support on Linux until the advent of
bitkeeper. This was when I had my own tree which was published regularly.
The problem that posed is that the tree was seen as "the ARM tree" where
_everything_ to do with ARM (be it device drivers) was _dumped_. Once it
was in there, the original authors decided that their job was done and
walked away.

The result was that I accumulated almost 1MB of compressed patch, and
we _never_ got to the situation where the merged ARM mainline was in
a buildable state despite _years_ of work to try to get it there. The
closest we got was one of the 2.4-ac trees, just before Alan quit doing
them (which might have been a contributary reason.)

To date, none of the 2.4 trees is buildable for ARM.

The biggest problem with it is synchronising changes with other trees,
and persuading other people to submit things to the relevant people.
Either such a tree accepts the ARM specific bits along side the device
drivers (and then ends up with a review and bypassing other maintainers
problem, resulting in the tree owner getting regularly roasted) or
they get moaned at for not accepting device driver patches which are
dependent on other changes in that tree.

Since then, I've vowed to _never_ maintain such a "community" tree ever
again. I credit 2.6 being buildable for ARM entirely on the choice to
get rid of this tree.

--
Russell King
2.6 ARM Linux - http://www.arm.linux.org.uk/

2006-09-24 14:26:00

by Diego Calleja

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

El Fri, 22 Sep 2006 08:32:23 +0800,
Fengguang Wu <[email protected]> escribi?:

> FYI: I attached a little presentation work, one for the boot time
> stuff, another for the readahead patch.

A nice and clean way to collect I/O traces that it's not mentioned in
the boot_linux_faster.pdf paper would be to use kprobes + systemtap.

2006-09-24 17:25:56

by Mike Galbraith

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Sun, 2006-09-24 at 10:20 +0100, Russell King wrote:

> The point I'm making is that for some things, keeping the changes as
> patches until they're ready is far easier, more worthwhile and flexible
> than having them simmering in some git tree somewhere.

I <heart> patch and </heart> scm (works for me:), but in theory, isn't
that the same?

2006-09-24 18:23:56

by Sean

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Sun, 24 Sep 2006 10:20:10 +0100
Russell King <[email protected]> wrote:

> The point I'm making is that for some things, keeping the changes as
> patches until they're ready is far easier, more worthwhile and flexible
> than having them simmering in some git tree somewhere.

It's not really easier, just different. Git allows you to make a
"topic branch" to keep separate items that need to bake before going
upstream without being mixed in with all your other worked. This
allows you to continue to pull in upstream changes to make sure that
the patch series still applies cleanly and doesn't need updates etc.

Of course you may be more comfortable with other tools, but Git _does_
make it easy and practical to do the same thing.

Sean

2006-09-24 18:58:45

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

* Linus Torvalds <[email protected]> wrote:

> I think the "big merges in the first two weeks, and a -rc1 after, and
> no new code after that" rule has been working because it brought
> everybody in on the same page.

yeah. I dont really support the even/odd release thing because even the
old 1.2/1.3/2.0/2.1/2.2/2.3/2.4 scheme _always_ confused non-insiders.
Sometimes i saw it confuse people who already understood the GPL ;-)
Furthermore it would just dillute our version numbers to encode some
information that "-rc1" indicates just as well. Insiders know perfectly
well that when -rc1 is released the merge window is closed. And what
causes -rc elongation is usually not the lack of communication towards
users or lack of testing but the lack of fixing power ...

Ingo

2006-09-24 22:38:06

by Stefan Richter

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Sean wrote:
> On Sun, 24 Sep 2006 10:20:10 +0100
> Russell King <[email protected]> wrote:
>
>> The point I'm making is that for some things, keeping the changes as
>> patches until they're ready is far easier, more worthwhile and flexible
>> than having them simmering in some git tree somewhere.
>
> It's not really easier, just different. Git allows you to make a
> "topic branch" to keep separate items that need to bake before going
> upstream without being mixed in with all your other worked.
...
> Git _does_ make it easy and practical to do the same thing.

I'm not convinced. Certain workflows are more focused on how changes
change (sic) rather than on how the end product i.e. the sources change.
I am referring to reworking of patches during tests and reviews as well
as rewriting descriptions, collecting Acks and Sign-offs etc. while
maintaining a certain identity of the patch or series of patches.

But maybe I'm just not aware of how git may support this effectively.
Perhaps thusly: Let the young and wild times of life of a patch actually
result into many commits to a topic branch; collapse a lot of these
commits into one or few diffs for each review round; move to a new topic
branch for bigger reworks of the changeset; and finally collapse it into
one or few commits to a staging branch for submission? Sounds still more
like a job for patch-centered tools like quilt.
--
Stefan Richter
-=====-=-==- =--= ==---
http://arcgraph.de/sr/

2006-09-24 23:08:04

by Sean

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Mon, 25 Sep 2006 00:34:45 +0200
Stefan Richter <[email protected]> wrote:

> I'm not convinced. Certain workflows are more focused on how changes
> change (sic) rather than on how the end product i.e. the sources change.
> I am referring to reworking of patches during tests and reviews as well
> as rewriting descriptions, collecting Acks and Sign-offs etc. while
> maintaining a certain identity of the patch or series of patches.
>
> But maybe I'm just not aware of how git may support this effectively.
> Perhaps thusly: Let the young and wild times of life of a patch actually
> result into many commits to a topic branch; collapse a lot of these
> commits into one or few diffs for each review round; move to a new topic
> branch for bigger reworks of the changeset; and finally collapse it into
> one or few commits to a staging branch for submission? Sounds still more
> like a job for patch-centered tools like quilt.

Well, you're absolutely right about native Git being more focused on
tracking the final product. However, there are tools growing up around
Git that attempt to give similar (although completely integrated with
Git) functionality of Quilt. One such tool is Stacked Git (StGit)
http://www.procode.org/stgit/. Since i'm not actually a user myself,
I can't vouch for it, but it does have a responsive community around it
and seems to be providing what it set out to accomplish.

Sean

2006-09-24 23:09:58

by Russell King

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Sun, Sep 24, 2006 at 02:23:53PM -0400, Sean wrote:
> On Sun, 24 Sep 2006 10:20:10 +0100
> Russell King <[email protected]> wrote:
>
> > The point I'm making is that for some things, keeping the changes as
> > patches until they're ready is far easier, more worthwhile and flexible
> > than having them simmering in some git tree somewhere.
>
> It's not really easier, just different. Git allows you to make a
> "topic branch" to keep separate items that need to bake before going
> upstream without being mixed in with all your other worked. This
> allows you to continue to pull in upstream changes to make sure that
> the patch series still applies cleanly and doesn't need updates etc.
>
> Of course you may be more comfortable with other tools, but Git _does_
> make it easy and practical to do the same thing.

Before people disagree with my statement (and I'm referring to all the
replies so far), maybe they should talk to Thomas who pioneered the
genirq changes to find out why he decided to work with patches rather
than a git repository?

--
Russell King
Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/
maintainer of: 2.6 Serial core

2006-09-24 23:23:12

by Sean

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Mon, 25 Sep 2006 00:09:48 +0100
Russell King <[email protected]> wrote:

> Before people disagree with my statement (and I'm referring to all the
> replies so far), maybe they should talk to Thomas who pioneered the
> genirq changes to find out why he decided to work with patches rather
> than a git repository?

When Thomas makes a sweeping statement about the applicability of one
tool over another people will respond to him. But if _you_ make such
a statement yourself (even if it's based on his conclusions) then
you better accept that people who disagree will respond to your statement.

Sean

2006-09-25 00:49:03

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

On Sun, 24 Sep 2006, Sean wrote:
>
> When Thomas makes a sweeping statement about the applicability of one
> tool over another people will respond to him. But if _you_ make such
> a statement yourself (even if it's based on his conclusions) then
> you better accept that people who disagree will respond to your statement.

I think it's unquestionable that sometimes it's better to work with
patches. The thing is, git by its very design is about tracking things
where history is firmly "set in stone", and that's not always what you
want.

We've done a number of things to help people relax that a bit (notably
"git rebase" and "git cherry-pick"), and there are projects like stgit
that are more of a patch-management system on top of git, but even during
the early design phases I said that it's likely that git will be used in
_conjunction_ with tools like quilt etc, that are less strict than git is.

So I don't think we need to attack the notion of using non-git tools at
all. Especially if you don't know where you're going, git's very strict
immobile history can be a bit daunting.

(Of course, once you get really used to git, you use git _anyway_, and
then you use cherry-pick and other tools to re-write a cleaned-up version
of the thing that you originally screwed up because you didn't know what
you were doing. So you _can_ do this too with git, but that doesn't mean
that git would necessarily be the best way to do it).

That said, maybe we could help the "fixup" phases evenmore using git. For
example, right now you can do "git cherry-pick" to transfer individual
patches, but if you want to combine two commits while cherry-picking, it
immediately gets more involved (still quite doable: use cherry-pick
multiple times with the "-n" flag, but it's not as obvious any more).

Linus

2006-09-25 11:19:10

by Arjan van de Ven

[permalink] [raw]

Subject: Re: hires timer patchset [was Re: 2.6.19 -mm merge plans]

> I'm not particularly eager to get it solved... the nvidia driver still
> needs one more hack for the glue to build against -rt3.

that's one for nvidia not this list

2006-09-25 11:55:01

by Nick Piggin

[permalink] [raw]

Subject: Re: 2.6.19 -mm merge plans

Ingo Molnar wrote:

>* Linus Torvalds <[email protected]> wrote:
>
>
>>I think the "big merges in the first two weeks, and a -rc1 after, and
>>no new code after that" rule has been working because it brought
>>everybody in on the same page.
>>
>
>yeah. I dont really support the even/odd release thing because even the
>old 1.2/1.3/2.0/2.1/2.2/2.3/2.4 scheme _always_ confused non-insiders.
>Sometimes i saw it confuse people who already understood the GPL ;-)
>Furthermore it would just dillute our version numbers to encode some
>information that "-rc1" indicates just as well. Insiders know perfectly
>well that when -rc1 is released the merge window is closed. And what
>causes -rc elongation is usually not the lack of communication towards
>users or lack of testing but the lack of fixing power ...
>

OTOH, if we were worried about confusing people, we wouldn't be
using the acronym 'rc' for our 'Ridiculous Count', and have our rc1
denote the result of 2 weeks of stuffing the tree with new features
and intrusive changes, where people might mistake that for the much
more common RC-as-in-'Release Candidate'. :)

Our -rc is what everyone else knows as -pre, and our dot zeros
basically correspond to what people think of as a release candidate.
As a developer it doesn't hurt me and I do like the current system,
but in principle I just dislike things that are more confusing than
they could be.

--

Send instant messages to your online friends http://au.messenger.yahoo.com

2006-09-25 12:02:10