adding David Miller and the wireless developers who had this idea as
well...
On Mon, Oct 12, 2009 at 04:54:53PM +0200, Ingo Molnar wrote:
> I think your interpretation is arbitrary - where did you get that ABI
> rule from? I'm sure it cannot be from any of the drivers/staging/
> discussions on lkml, i've followed them quite closely. If 'has a messy
> ABI' was the only requirement for drivers/staging/ then we could move
> 90% of drivers/staging/ into drivers/ straight away - and that would be
> counter-productive IMHO.
I agree with this, and the other points you raised that I snipped out.
> Sidenote, in fact i think we should expand on that: drivers/staging/
> should be used in the _other_ direction as well - to un-upstream stale
> drivers that are abandoned and unused, in a gradual fashion. 'git mv' is
> cheap.
Ok, this is about the 3rd or 4th time I've heard this, from totally
different people lately. It seems that I'm the only one that has the
ability to drop drivers out of the kernel tree, which is a funny
situation :)
In thinking about this a lot more, I don't really mind it. If people
want to push stuff out of "real" places in the kernel, into
drivers/staging/ and give the original authors and maintainers notice
about what is going on, _and_ provide a TODO file for what needs to
happen to get the code back into the main portion of the kernel tree,
then I'll be happy to help out with this and manage it.
I think a 6-9 month window (basically 3 kernel releases) should be
sufficient time to have a driver that has been in drivers/staging/ be
cleaned up enough to move back into the main kernel tree. If not, it
could be easily dropped.
Any objections to this?
> Basically, drivers/staging/ gives us an excellent opportunity to
> _increase_ the quality of drivers by applying stronger upstream
> inclusion filters, without having to hurt users/developers in the
> process. We just have to start using it that way as well.
I totally agree. And so far, it does seem to be working well for this.
A number of companies have used it to successfully get their code into
the main kernel, as well as driving them to clean up their code better
to keep it from having to go into the staging tree.
thanks,
greg k-h
On Tue, Oct 13, 2009 at 11:08:15AM -0700, Luis R. Rodriguez wrote:
> On Mon, Oct 12, 2009 at 4:24 PM, Greg KH <[email protected]> wrote:
> > On Mon, Oct 12, 2009 at 05:42:44PM +0200, Ingo Molnar wrote:
> >> Hm, i think i even gave drivers/staging/ its name?
> >
> > Yes you did, and I appreciate it :)
> >
> >> > [...] It seems that I'm the only one that has the ability to drop
> >> > drivers out of the kernel tree, which is a funny situation :)
> >>
> >> You are the only one who has the ability to send a warning shot towards
> >> drivers _without hurting users_, and by moving it into the focus of a
> >> team of cleanup oriented developers.
> >>
> >> I think that's an important distinction ;-)
> >
> > Good point.
> >
> >> > In thinking about this a lot more, I don't really mind it. ??If people
> >> > want to push stuff out of "real" places in the kernel, into
> >> > drivers/staging/ and give the original authors and maintainers notice
> >> > about what is going on, _and_ provide a TODO file for what needs to
> >> > happen to get the code back into the main portion of the kernel tree,
> >> > then I'll be happy to help out with this and manage it.
> >> >
> >> > I think a 6-9 month window (basically 3 kernel releases) should be
> >> > sufficient time to have a driver that has been in drivers/staging/ be
> >> > cleaned up enough to move back into the main kernel tree. ??If not, it
> >> > could be easily dropped.
> >> >
> >> > Any objections to this?
> >>
> >> Sounds excellent to me!
> >
> > Great, I'll await the patches to move stuff to drivers/staging/ now.
> >
> > Wireless developers, warm up your editors :)
>
> OK -- prism54 seems like a good candidate, instead of removing it
> completely as I originally outlined on the feature removal schedule.
> Do we have a file to give notices to move drivers to staging because
> they are old as with the feature removal schedule? The more visible
> these things become the better it is for users.
We've found that the feature removal file is also ignored :)
How about when it was scheduled to be removed, we put it in staging and
I'll add it to my announcements about the staging tree every release?
Unless you can think of a better way?
thanks,
greg k-h
Ingo Molnar wrote:
> Yes, that's a real worry. Some time ago i suggested:
>
> drivers/staging/good/
> drivers/staging/bad/
> drivers/staging/ugly/
>
> good: drivers that are to go upstream in the next cycle
> bad: outgoing drivers being obsoleted or abandoned
> ugly: incoming messy drivers with active developers
>
> The messaging of this looks nice and the names are short and obvious.
>
> An added benefit is that this kind of separation makes it easy for
> people interested in drivers/staging to follow the 'status' of drivers.
> Once stuff goes into 'good' a different kind of review is needed than if
> a driver goes into 'ugly'.
>
> The main disadvantage would be the PR angle: putting new drivers into a
> path named 'ugly'. Not something you want to put into a quarterly status
> report, right? If we put drivers/staging/ugly/ drivers into
> drivers/staging/ itself, we'd solve that problem. I.e. we'd keep the
> current scheme, but we'd also add drivers/staging/good/ and
> drivers/staging/bad/ as two extra stages for incoming and outgoing
> drivers.
Change "ugly" to "wip" (work in progress). Should remove the negative
connotation and keeps things short. Does miss the spaghetti western theme
though :)
-- james s
On Mon, Oct 12, 2009 at 05:42:44PM +0200, Ingo Molnar wrote:
> Hm, i think i even gave drivers/staging/ its name?
Yes you did, and I appreciate it :)
> > [...] It seems that I'm the only one that has the ability to drop
> > drivers out of the kernel tree, which is a funny situation :)
>
> You are the only one who has the ability to send a warning shot towards
> drivers _without hurting users_, and by moving it into the focus of a
> team of cleanup oriented developers.
>
> I think that's an important distinction ;-)
Good point.
> > In thinking about this a lot more, I don't really mind it. If people
> > want to push stuff out of "real" places in the kernel, into
> > drivers/staging/ and give the original authors and maintainers notice
> > about what is going on, _and_ provide a TODO file for what needs to
> > happen to get the code back into the main portion of the kernel tree,
> > then I'll be happy to help out with this and manage it.
> >
> > I think a 6-9 month window (basically 3 kernel releases) should be
> > sufficient time to have a driver that has been in drivers/staging/ be
> > cleaned up enough to move back into the main kernel tree. If not, it
> > could be easily dropped.
> >
> > Any objections to this?
>
> Sounds excellent to me!
Great, I'll await the patches to move stuff to drivers/staging/ now.
Wireless developers, warm up your editors :)
thanks,
greg k-h
On Mon, Oct 12, 2009 at 10:43:06AM -0500, James Bottomley wrote:
> If you want to make this a mandatory path for old drivers, then, I think
> it's far too rigid, yes. There's a huge amount of danger to changing
> working drivers simply on grounds of code cleanup and that danger
> increases exponentially as they get older and the hardware gets rarer.
> Look at what happened to the initio driver in 2008 for instance. That
> was cleaned up by Alan Cox, no mean expert in the field, with the
> assistance of a tester with the actual card, so basically a textbook
> operation. However, a bug crept in during this process that wasn't
> spotted by the tester. When it was spotted (bug report ~6 months later)
> the original tester wasn't available and code inspection across the
> cleanup was very hard. Fortunately, the reporter was motivated to track
> down and patch the driver, so it worked out all right in the end, but a
> lot of bug reporters aren't so capable (or so motivated). Plus most
> clean up patches for old hardware tend only to be compile tested, so the
> potential for bugs is far greater.
I understand the potential for bugs, and am not saying to do this for
all drivers, so it is not mandatory at all.
I have just received a bunch of people asking me if we can use
drivers/staging/ to get stuff that is known broken, or has other
problems (style issues[1]), out into an area where people know it needs
to be fixed up otherwise it will be dropped.
thanks,
greg k-h
[1] No, floppy.c doesn't count, no matter how much people might want it
to :)
On Wed, Oct 14, 2009 at 08:33:08AM +0200, Ingo Molnar wrote:
>
> * Joe Perches <[email protected]> wrote:
>
> > On Tue, 2009-10-13 at 21:45 -0700, Greg KH wrote:
> > > How about when it was scheduled to be removed, we put it in staging and
> > > I'll add it to my announcements about the staging tree every release?
> > > Unless you can think of a better way?
> >
> > staging/to_be_removed_unless_fixed_by/v.x.y ?
>
> Yes, that's a real worry. Some time ago i suggested:
>
> drivers/staging/good/
> drivers/staging/bad/
> drivers/staging/ugly/
>
> good: drivers that are to go upstream in the next cycle
> bad: outgoing drivers being obsoleted or abandoned
> ugly: incoming messy drivers with active developers
>
> The messaging of this looks nice and the names are short and obvious.
Yeah, but they make my life much harder than it needs to be.
I'd prefer to stick with the directory naming scheme we have today, it
seems to work well so far.
thanks,
greg k-h
Ingo Molnar wrote:
> * Joe Perches <[email protected]> wrote:
>
>> On Tue, 2009-10-13 at 21:45 -0700, Greg KH wrote:
>> > How about when it was scheduled to be removed, we put it in staging and
>> > I'll add it to my announcements about the staging tree every release?
>> > Unless you can think of a better way?
>>
>> staging/to_be_removed_unless_fixed_by/v.x.y ?
>
> Yes, that's a real worry. Some time ago i suggested:
>
> drivers/staging/good/
> drivers/staging/bad/
> drivers/staging/ugly/
How well do "git am", "quilt import" and friends cope with ever changing
directories?
How about using drivers/staging/this_driver/TODO and (or) its Kconfig
help text to leave a note about the plans for this driver?
The worry that these will be ignored like
Documentation/feature-removal-schedule.txt is being ignored may apply to
the path name based solution too, I'm afraid.
(Besides, Greg's release announcements are a good channel for this.
These announcements are actually picked up by the specialized press.)
--
Stefan Richter
-=====-==--= =-=- -===-
http://arcgraph.de/sr/
* Greg KH <[email protected]> wrote:
> > Sidenote, in fact i think we should expand on that: drivers/staging/
> > should be used in the _other_ direction as well - to un-upstream
> > stale drivers that are abandoned and unused, in a gradual fashion.
> > 'git mv' is cheap.
>
> Ok, this is about the 3rd or 4th time I've heard this, from totally
> different people lately. [...]
Heh. I guess i had a good crystal ball on that then - i raised that
issue in my very first comments on drivers/staging/, 2.5 years ago - see
the "Linux Kernel developer statement about closed source code" thread,
where i wrote:
| - We must accept open-source drivers (which are license-compatible
| with the kernel) for new hardware the moment they are offered. No
| ifs and when. No whining about quality, style, security, re-use,
| non-reuse, obsolete APIs, overlapping functionality, the already
| busy merge schedule of a maintainer or whatever other thing can come
| up on lkml.
|
| We can create arbitrary quarantine mechanisms we wish to use:
| CONFIG_REALLY_BROKEN. We could even create drivers/staging/ as a
| temporary staging area for drivers.
|
| What we cannot do is to _deny_ the distribution channel and exclude
| users. The moment we do that (and we still do that in way too many
| areas of the kernel) we have lost the availability race.
[...]
| If it's not maintained actively(sc) (i.e. it's a fire-and-forget
| driver) then it can get marked BROKEN with time, and can get removed
| as well.
Hm, i think i even gave drivers/staging/ its name?
> [...] It seems that I'm the only one that has the ability to drop
> drivers out of the kernel tree, which is a funny situation :)
You are the only one who has the ability to send a warning shot towards
drivers _without hurting users_, and by moving it into the focus of a
team of cleanup oriented developers.
I think that's an important distinction ;-)
> In thinking about this a lot more, I don't really mind it. If people
> want to push stuff out of "real" places in the kernel, into
> drivers/staging/ and give the original authors and maintainers notice
> about what is going on, _and_ provide a TODO file for what needs to
> happen to get the code back into the main portion of the kernel tree,
> then I'll be happy to help out with this and manage it.
>
> I think a 6-9 month window (basically 3 kernel releases) should be
> sufficient time to have a driver that has been in drivers/staging/ be
> cleaned up enough to move back into the main kernel tree. If not, it
> could be easily dropped.
>
> Any objections to this?
Sounds excellent to me!
> > Basically, drivers/staging/ gives us an excellent opportunity to
> > _increase_ the quality of drivers by applying stronger upstream
> > inclusion filters, without having to hurt users/developers in the
> > process. We just have to start using it that way as well.
>
> I totally agree. And so far, it does seem to be working well for
> this. A number of companies have used it to successfully get their
> code into the main kernel, as well as driving them to clean up their
> code better to keep it from having to go into the staging tree.
Yeah. I think we were hurting from an under-estimated trust problem:
driver authors couldnt trust _us maintainers_ to follow through with
upstreaming. drivers/staging/ improved that IMHO.
I.e. the old process of upstreaming drivers was too arbitrary, incurred
big latencies and was dependent on the whims of maintainers. I.e. new
developers got exposed to some of the worst social aspects of a
distributed development process.
Now there's basically a single (and friendly! ;-) upstreaming channel
that people (yet-)unfamilar with Linux practices can standardize on.
This reduces the pressure on maintainers and also creates a reference
point for upstreaming honesty - which is almost unconditionally good i
think.
Ingo
* Joe Perches <[email protected]> wrote:
> On Tue, 2009-10-13 at 21:45 -0700, Greg KH wrote:
> > How about when it was scheduled to be removed, we put it in staging and
> > I'll add it to my announcements about the staging tree every release?
> > Unless you can think of a better way?
>
> staging/to_be_removed_unless_fixed_by/v.x.y ?
Yes, that's a real worry. Some time ago i suggested:
drivers/staging/good/
drivers/staging/bad/
drivers/staging/ugly/
good: drivers that are to go upstream in the next cycle
bad: outgoing drivers being obsoleted or abandoned
ugly: incoming messy drivers with active developers
The messaging of this looks nice and the names are short and obvious.
An added benefit is that this kind of separation makes it easy for
people interested in drivers/staging to follow the 'status' of drivers.
Once stuff goes into 'good' a different kind of review is needed than if
a driver goes into 'ugly'.
The main disadvantage would be the PR angle: putting new drivers into a
path named 'ugly'. Not something you want to put into a quarterly status
report, right? If we put drivers/staging/ugly/ drivers into
drivers/staging/ itself, we'd solve that problem. I.e. we'd keep the
current scheme, but we'd also add drivers/staging/good/ and
drivers/staging/bad/ as two extra stages for incoming and outgoing
drivers.
A third version would be a more neutral name:
drivers/staging/incoming/
drivers/staging/outgoing/
I think it has many advantages, but (of course!) it all depends on
whether Greg wants to have any separation like this.
Ingo
* Stefan Richter <[email protected]> wrote:
> > Well, the answer is obvious i think. Tell me, at a glance, if you
> > see a patch on lkml, which one is for a staging driver to be
> > obsoleted, and which one is the one going upstream real soon? The
> > patches say:
> >
> > +++ a/drivers/staging/foo/x.c
> >
> > +++ a/drivers/staging/bar/y.c
> >
> > Then tell me the same at a glance if you see patches for:
> >
> > +++ a/drivers/staging/wip/x.c
> >
> > +++ a/drivers/staging/bad/y.c
>
> Does this information matter much?
Yes. You might not appreciate it as you are active in a relatively
narrow field (so all patches in your world have an 'obvious' place) -
but i for example take most of the context of a change from the email
itself and the more self-descriptive it is, the better. I would be more
likely to review work-in-progress patches while not bother about
obsolete drivers on the way out. YMMV.
> What's more interesting is whether development activity will _lead_ to
> a driver being moved from bad or ugly to good.
... a prerequisite of which is for more developers to be accutely aware
of in what state a driver is.
Anyway ... it's all up to Greg and he indicated that he wants the
simplest structure, which is fair enough.
Ingo
* Stefan Richter <[email protected]> wrote:
> Ingo Molnar wrote:
> > * Joe Perches <[email protected]> wrote:
> >
> >> On Tue, 2009-10-13 at 21:45 -0700, Greg KH wrote:
> >> > How about when it was scheduled to be removed, we put it in staging and
> >> > I'll add it to my announcements about the staging tree every release?
> >> > Unless you can think of a better way?
> >>
> >> staging/to_be_removed_unless_fixed_by/v.x.y ?
> >
> > Yes, that's a real worry. Some time ago i suggested:
> >
> > drivers/staging/good/
> > drivers/staging/bad/
> > drivers/staging/ugly/
>
> How well do "git am", "quilt import" and friends cope with ever
> changing directories?
Once a driver is in a tree it's in Git and git mv is easy. People
working with Linux better familiarize themselves with Git workflow - the
sooner the better.
If it's not in tree then it will adopt to whatever layout there is once
it gets into Greg's tree. I dont see the problem.
> How about using drivers/staging/this_driver/TODO and (or) its Kconfig
> help text to leave a note about the plans for this driver?
Well, the answer is obvious i think. Tell me, at a glance, if you see a
patch on lkml, which one is for a staging driver to be obsoleted, and
which one is the one going upstream real soon? The patches say:
+++ a/drivers/staging/foo/x.c
+++ a/drivers/staging/bar/y.c
Then tell me the same at a glance if you see patches for:
+++ a/drivers/staging/wip/x.c
+++ a/drivers/staging/bad/y.c
> The worry that these will be ignored like
> Documentation/feature-removal-schedule.txt is being ignored may apply
> to the path name based solution too, I'm afraid.
It wont be 'ignored', as it's in every patch, it's in every commit, it's
in every substantial communication about that driver.
The problem with feature-removal-schedule.txt is that it's too much out
of sight and not part of the regular patch workflow. Same goes for any
TODO file. Experience has shown that the actual _path_ were drivers end
up does matter quite a bit, to general visibility and to mindset.
That's one of the reasons why we have _half a thousand_ directories in
drivers/ to begin with. The directory namespace is very powerful, and we
use it to convey all sorts of information about the logical category a
driver is in.
Using it in drivers/staging/ instead of the current flat hierarchy would
thus be pretty natural.
Ingo
On Tue, 2009-10-13 at 21:45 -0700, Greg KH wrote:
> How about when it was scheduled to be removed, we put it in staging and
> I'll add it to my announcements about the staging tree every release?
> Unless you can think of a better way?
staging/to_be_removed_unless_fixed_by/v.x.y ?
On Mon, 2009-10-12 at 08:09 -0700, Greg KH wrote:
> adding David Miller and the wireless developers who had this idea as
> well...
>
> On Mon, Oct 12, 2009 at 04:54:53PM +0200, Ingo Molnar wrote:
> > I think your interpretation is arbitrary - where did you get that ABI
> > rule from? I'm sure it cannot be from any of the drivers/staging/
> > discussions on lkml, i've followed them quite closely. If 'has a messy
> > ABI' was the only requirement for drivers/staging/ then we could move
> > 90% of drivers/staging/ into drivers/ straight away - and that would be
> > counter-productive IMHO.
>
> I agree with this, and the other points you raised that I snipped out.
>
> > Sidenote, in fact i think we should expand on that: drivers/staging/
> > should be used in the _other_ direction as well - to un-upstream stale
> > drivers that are abandoned and unused, in a gradual fashion. 'git mv' is
> > cheap.
>
> Ok, this is about the 3rd or 4th time I've heard this, from totally
> different people lately. It seems that I'm the only one that has the
> ability to drop drivers out of the kernel tree, which is a funny
> situation :)
>
> In thinking about this a lot more, I don't really mind it. If people
> want to push stuff out of "real" places in the kernel, into
> drivers/staging/ and give the original authors and maintainers notice
> about what is going on, _and_ provide a TODO file for what needs to
> happen to get the code back into the main portion of the kernel tree,
> then I'll be happy to help out with this and manage it.
>
> I think a 6-9 month window (basically 3 kernel releases) should be
> sufficient time to have a driver that has been in drivers/staging/ be
> cleaned up enough to move back into the main kernel tree. If not, it
> could be easily dropped.
>
> Any objections to this?
Not as an optional tool to use when necessary.
If you want to make this a mandatory path for old drivers, then, I think
it's far too rigid, yes. There's a huge amount of danger to changing
working drivers simply on grounds of code cleanup and that danger
increases exponentially as they get older and the hardware gets rarer.
Look at what happened to the initio driver in 2008 for instance. That
was cleaned up by Alan Cox, no mean expert in the field, with the
assistance of a tester with the actual card, so basically a textbook
operation. However, a bug crept in during this process that wasn't
spotted by the tester. When it was spotted (bug report ~6 months later)
the original tester wasn't available and code inspection across the
cleanup was very hard. Fortunately, the reporter was motivated to track
down and patch the driver, so it worked out all right in the end, but a
lot of bug reporters aren't so capable (or so motivated). Plus most
clean up patches for old hardware tend only to be compile tested, so the
potential for bugs is far greater.
James
> > Basically, drivers/staging/ gives us an excellent opportunity to
> > _increase_ the quality of drivers by applying stronger upstream
> > inclusion filters, without having to hurt users/developers in the
> > process. We just have to start using it that way as well.
>
> I totally agree. And so far, it does seem to be working well for this.
> A number of companies have used it to successfully get their code into
> the main kernel, as well as driving them to clean up their code better
> to keep it from having to go into the staging tree.
>
> thanks,
>
> greg k-h
On Mon, Oct 12, 2009 at 4:24 PM, Greg KH <[email protected]> wrote:
> On Mon, Oct 12, 2009 at 05:42:44PM +0200, Ingo Molnar wrote:
>> Hm, i think i even gave drivers/staging/ its name?
>
> Yes you did, and I appreciate it :)
>
>> > [...] It seems that I'm the only one that has the ability to drop
>> > drivers out of the kernel tree, which is a funny situation :)
>>
>> You are the only one who has the ability to send a warning shot towards
>> drivers _without hurting users_, and by moving it into the focus of a
>> team of cleanup oriented developers.
>>
>> I think that's an important distinction ;-)
>
> Good point.
>
>> > In thinking about this a lot more, I don't really mind it. If people
>> > want to push stuff out of "real" places in the kernel, into
>> > drivers/staging/ and give the original authors and maintainers notice
>> > about what is going on, _and_ provide a TODO file for what needs to
>> > happen to get the code back into the main portion of the kernel tree,
>> > then I'll be happy to help out with this and manage it.
>> >
>> > I think a 6-9 month window (basically 3 kernel releases) should be
>> > sufficient time to have a driver that has been in drivers/staging/ be
>> > cleaned up enough to move back into the main kernel tree. If not, it
>> > could be easily dropped.
>> >
>> > Any objections to this?
>>
>> Sounds excellent to me!
>
> Great, I'll await the patches to move stuff to drivers/staging/ now.
>
> Wireless developers, warm up your editors :)
OK -- prism54 seems like a good candidate, instead of removing it
completely as I originally outlined on the feature removal schedule.
Do we have a file to give notices to move drivers to staging because
they are old as with the feature removal schedule? The more visible
these things become the better it is for users.
Luis
On 14 Oct, Ingo Molnar wrote:
> * Stefan Richter <[email protected]> wrote:
>> How well do "git am", "quilt import" and friends cope with ever
>> changing directories?
[Perhaps the -p option does the trick. And git-am could be done on a
temporary branch from before the move of the files.]
> Once a driver is in a tree it's in Git and git mv is easy. People
> working with Linux better familiarize themselves with Git workflow - the
> sooner the better.
Even if author and committer both work with git, they often use e-mail
to transfer patches.
> If it's not in tree then it will adopt to whatever layout there is once
> it gets into Greg's tree.
Many "good" drivers will start as "ugly" or "bad" ones.
>> How about using drivers/staging/this_driver/TODO and (or) its Kconfig
>> help text to leave a note about the plans for this driver?
>
> Well, the answer is obvious i think. Tell me, at a glance, if you see a
> patch on lkml, which one is for a staging driver to be obsoleted, and
> which one is the one going upstream real soon? The patches say:
>
> +++ a/drivers/staging/foo/x.c
>
> +++ a/drivers/staging/bar/y.c
>
> Then tell me the same at a glance if you see patches for:
>
> +++ a/drivers/staging/wip/x.c
>
> +++ a/drivers/staging/bad/y.c
Does this information matter much?
What's more interesting is whether development activity will _lead_ to a
driver being moved from bad or ugly to good.
--
Stefan Richter
-=====-==--= =-=- -===-
http://arcgraph.de/sr/