There have been a large number of emails on lkml (too many for me to
dredge up and quote) plus a fair few private ones on the subject of when
to merge drivers. The opinions seem to vary from "immediately they show
up on the mailing list" to " after we're sure they're demonstrated to be
working and maintainable". The fact that the arguments have been going
round and round on the various mailing lists without generating any
consensus would seem to indicate the topic would benefit from some face
to face discussion time.
I think the Kernel Summit would be a good place to have a discussion of
what the criteria are for merging a driver (even if, in the end, it's at
the discretion of the subsystem maintainers).
I think everyone agrees that we can't put just anything that appears on
the mailing list in the tree, since they all have to be at least code
inspected and be checked for the usual errors, omissions and root holes.
However, disagreements seem to set in after this.
For the record, my own view is that when a new driver does appear we
have a limited time to get the author to make any necessary changes, so
I try to get it reviewed and most of the major issues elucidated as soon
as possible. However, since the only leverage I have is inclusion, I
tend to hold it out of tree until the problems are sorted out.
Experience has shown that for most SCSI drivers, the authors tend to be
the people producing the hardware and without documentation, no-one else
can fix up anything other than obvious coding errors, so we can't put it
in the tree and hope someone else will fix it if they have a problem.
One possible way of doing this is certainly to put the drivers in the
staging tree:
http://marc.info/?l=linux-kernel&m=121312896923196
The only slight wrinkle (at least for me) is that often the process of
cleaning up a driver is fairly intensive for a maintainer and turn
around is a lot faster if you're doing it in a tree you control. (All
the scsi drivers we've done like this have lived in temporary branches
while they were being worked on). So perhaps in addition we should be
encouraging maintainers to run staging branches under similar rules in
the staging tree, but allowing inclusion into linux-next?
James
> Experience has shown that for most SCSI drivers, the authors tend to be
> the people producing the hardware and without documentation, no-one else
> can fix up anything other than obvious coding errors, so we can't put it
> in the tree and hope someone else will fix it if they have a problem.
Is there an opportunity for a carrot here ... a driver submitted
*with documentation* gets a faster path into the tree?
-Tony
On Jun. 18, 2008, 18:27 +0300, James Bottomley <[email protected]> wrote:
> There have been a large number of emails on lkml (too many for me to
> dredge up and quote) plus a fair few private ones on the subject of when
> to merge drivers. The opinions seem to vary from "immediately they show
> up on the mailing list" to " after we're sure they're demonstrated to be
> working and maintainable". The fact that the arguments have been going
> round and round on the various mailing lists without generating any
> consensus would seem to indicate the topic would benefit from some face
> to face discussion time.
>
> I think the Kernel Summit would be a good place to have a discussion of
> what the criteria are for merging a driver (even if, in the end, it's at
> the discretion of the subsystem maintainers).
>
> I think everyone agrees that we can't put just anything that appears on
> the mailing list in the tree, since they all have to be at least code
> inspected and be checked for the usual errors, omissions and root holes.
> However, disagreements seem to set in after this.
>
> For the record, my own view is that when a new driver does appear we
> have a limited time to get the author to make any necessary changes, so
> I try to get it reviewed and most of the major issues elucidated as soon
> as possible. However, since the only leverage I have is inclusion, I
> tend to hold it out of tree until the problems are sorted out.
> Experience has shown that for most SCSI drivers, the authors tend to be
> the people producing the hardware and without documentation, no-one else
> can fix up anything other than obvious coding errors, so we can't put it
> in the tree and hope someone else will fix it if they have a problem.
>
> One possible way of doing this is certainly to put the drivers in the
> staging tree:
>
> http://marc.info/?l=linux-kernel&m=121312896923196
>
> The only slight wrinkle (at least for me) is that often the process of
> cleaning up a driver is fairly intensive for a maintainer and turn
> around is a lot faster if you're doing it in a tree you control. (All
> the scsi drivers we've done like this have lived in temporary branches
> while they were being worked on). So perhaps in addition we should be
> encouraging maintainers to run staging branches under similar rules in
> the staging tree, but allowing inclusion into linux-next?
This is a very good idea.
Exposing the not-yet-ready-to-be-released code to linux-next will
expose conflicts earlier, and hopefully in smaller, more manageable
deltas.
However, some projects that need staging may change kernel APIs
to such extent that including them in linux-next will require
committing to the API changes in the -next time frame. If that's
a problem, staging them in the maintainer's tree may still be valuable.
Benny
>
> James
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
On Wed, 2008-06-18 at 10:15 -0700, Luck, Tony wrote:
> > Experience has shown that for most SCSI drivers, the authors tend to be
> > the people producing the hardware and without documentation, no-one else
> > can fix up anything other than obvious coding errors, so we can't put it
> > in the tree and hope someone else will fix it if they have a problem.
>
> Is there an opportunity for a carrot here ... a driver submitted
> *with documentation* gets a faster path into the tree?
You mean with actual how to program it information? er dunno ... never
seen one of those ...
But I suppose, theoretically, since anyone could access the
documentation to maintain it, it would be much more likely to be fixed
up in tree, yes.
James
On Wed, 18 Jun 2008 10:27:46 -0500
James Bottomley <[email protected]> wrote:
> I think the Kernel Summit would be a good place to have a discussion of
> what the criteria are for merging a driver (even if, in the end, it's at
> the discretion of the subsystem maintainers).
I agree.
> So perhaps in addition we should be
> encouraging maintainers to run staging branches under similar rules in
> the staging tree, but allowing inclusion into linux-next?
This seems a good idea for drivers.
One potencial issue that we may discuss is some sort of policy on how to deal
with drivers at the "generic" staging tree versus a subsystem maintainer
staging tree. I can foresee a few troubles if a maintainer have such tree, like:
- After the fixes at -staging, the patch may be kept for a while at
maintainer's staging tree. So, the workflow will be longer than having just one tree;
- Two different patches for the same device can be sent to the
"generic" staging and to the maintainer's staging tree. So, a conflict will
rise (hopefully detected at linux-next, but may require additional checks for
conflicts with PCI/USB ID's);
On the other hand, without the maintainer's staging tree, this would mean that
the maintainer would have to send their cooking drivers to the generic tree,
with doesn't seem to be very productive.
A possible solution would be if each maintainer with such staging tree would
keep the wiki at linux driver project updated, but this adds some additional
tasks to the maintainer's arms.
Cheers,
Mauro
On Wed, 18 Jun 2008, Benny Halevy wrote:
> Exposing the not-yet-ready-to-be-released code to linux-next will expose
> conflicts earlier, and hopefully in smaller, more manageable deltas.
I like the way linux-next works now, i.e. it should reflect what is going
into the next major kernel release. Nothing more, nothing less.
Also, when talking about entirely new drivers for hardware that hasn't
been supported by the kernel at all so far (this is the situation we are
talking about, right?), there shouldn't be a lot of conflicts to be dealt
with, right? Ideally, drivers should be pretty isolated standalone pieces
of code that don't have any business changing any code that has been
already there.
--
Jiri Kosina
SUSE Labs
> The only slight wrinkle (at least for me) is that often the process of
> cleaning up a driver is fairly intensive for a maintainer and turn
> around is a lot faster if you're doing it in a tree you control. (All
> the scsi drivers we've done like this have lived in temporary branches
> while they were being worked on). So perhaps in addition we should be
> encouraging maintainers to run staging branches under similar rules in
> the staging tree, but allowing inclusion into linux-next?
.../...
linux-next should, imho, exclusively be for things we are pretty much
commited to merge in the next release. ie, a staging place to fixup
things like build breakages, patch conflicts, etc...
Or else, it will just be another -mm ....
So while what you say makes sense, I think we should only put drivers in
once we have pretty much decided that the drivers in question would be
merged... in which case, why not straight upstream ?
Cheers,
Ben.
On Thu, 2008-06-19 at 09:31 +1000, Benjamin Herrenschmidt wrote:
> > The only slight wrinkle (at least for me) is that often the process of
> > cleaning up a driver is fairly intensive for a maintainer and turn
> > around is a lot faster if you're doing it in a tree you control. (All
> > the scsi drivers we've done like this have lived in temporary branches
> > while they were being worked on). So perhaps in addition we should be
> > encouraging maintainers to run staging branches under similar rules in
> > the staging tree, but allowing inclusion into linux-next?
>
> .../...
>
> linux-next should, imho, exclusively be for things we are pretty much
> commited to merge in the next release. ie, a staging place to fixup
> things like build breakages, patch conflicts, etc...
>
> Or else, it will just be another -mm ....
Well, as Greg said for his driver staging tree, I think we can elide
this requirement for new drivers. The good thing about drivers is that
there's nothing we're really doing to break anything: before the driver
the hardware was just plane unusable with Linux after the driver well,
we're hoping it might be ...
> So while what you say makes sense, I think we should only put drivers in
> once we have pretty much decided that the drivers in question would be
> merged... in which case, why not straight upstream ?
The reason for staging early in linux-next is twofold
1. We want to encourage the driver submitter that something is
being done
2. we want to make the driver available to a pool of testers who
might have the hardware but might not otherwise find it so they
can report errors that aren't picked up via the normal code
inspection and analysis methods.
Arguably, I can do this by putting it into my upstream tree (which feeds
into linux-next) the reason for not doing so is that Linus likes us to
preserve history in there when we can. If I need to pull the driver for
a reroll it really screws the git history (plus makes it hard for me).
If it's in its own branch, I can toy with it as much as I need to
without affecting my upstream tree. Plus the commitment is that
anything which goes into my upstream tree should be for the next merge
window. A driver with substantive code issues (or which shows problems
under test) might not be such a candidate until the issues are sorted
out.
James
On Wed, 18 Jun 2008, James Bottomley wrote:
> Arguably, I can do this by putting it into my upstream tree (which feeds
> into linux-next) the reason for not doing so is that Linus likes us to
> preserve history in there when we can.
This of course applies only for branches/trees for which there are
downstream git users.
If the only reason you merge the driver into your tree is to have it
propagated into linux-next, you could very well create a separate branch
for it, and pull this branch into the one linux-next pulls from you (I
guess you have a dedicated branch for linux-next which is safe to be
rebased by definition, right?).
> If I need to pull the driver for a reroll it really screws the git
> history
But not in any of the branches that have upstream git users.
--
Jiri Kosina
SUSE Labs
On Jun. 19, 2008, 2:20 +0300, Jiri Kosina <[email protected]> wrote:
> On Wed, 18 Jun 2008, Benny Halevy wrote:
>
>> Exposing the not-yet-ready-to-be-released code to linux-next will expose
>> conflicts earlier, and hopefully in smaller, more manageable deltas.
>
> I like the way linux-next works now, i.e. it should reflect what is going
> into the next major kernel release. Nothing more, nothing less.
>
> Also, when talking about entirely new drivers for hardware that hasn't
> been supported by the kernel at all so far (this is the situation we are
> talking about, right?), there shouldn't be a lot of conflicts to be dealt
> with, right? Ideally, drivers should be pretty isolated standalone pieces
> of code that don't have any business changing any code that has been
> already there.
>
The showcase example I have in mind is our OSD initiator for which
we pushed all dependencies already upstream, so currently it has
no conflicts with the existing code. However, it is not ready
for the next release since it should be reviewed, documented,
and accepted by the sub-system maintainer first.
I would like it to be included in linux-next so to be alerted of any
arising conflict so I can fix it in "real time" rather than waiting
for the next release to come out. Initially I thought it'd be a good
candidate for linux-staging, but Greg doesn't quite agree with that
(http://lkml.org/lkml/2008/6/11/225) since it's destined for linux-scsi.
Hence, a -staging branch/tree based off of James' tree makes sense.
Benny
James Bottomley wrote:
> On Thu, 2008-06-19 at 09:31 +1000, Benjamin Herrenschmidt wrote:
>> linux-next should, imho, exclusively be for things we are pretty much
>> commited to merge in the next release. ie, a staging place to fixup
>> things like build breakages, patch conflicts, etc...
>>
>> Or else, it will just be another -mm ....
>
> Well, as Greg said for his driver staging tree, I think we can elide
> this requirement for new drivers. The good thing about drivers is that
> there's nothing we're really doing to break anything: before the driver
> the hardware was just plane unusable with Linux after the driver well,
> we're hoping it might be ...
This requires that the staged driver does not contain build bugs on any
architecture.
Besides, the definition of what we published in -next so far was:
Everything in -next is *merge-ready* now (from a subsystem
maintainer's POV). The only reason that it is not merged yet
is that Linus doesn't have a merge window open right now.
And the goal of -next was to check for potential integration problems
that cannot be detected in a maintainer's tree.
So IMO the question is not whether to put stuff which is not ready to be
merged into -next. It shouldn't, IMO. The questions rather are:
- What are criteria for merge-readiness of drivers?
(When do we have to / are we able to address issues like CamelCase
names or use of obsolete APIs --- before or after merge?)
- How to publish drivers which are not yet merge-ready but should get
into the hands of testers?
IMO the answer to the latter question is still not -next, because the
testing that -next gets is (from what I understood) also with the
specific goal to detect potential integration problems. Remember, you
publish code in -next of which you can say: This is ready for mainline,
except that there is little experience yet WRT potential integration issues.
--
Stefan Richter
-=====-==--- -==- =--==
http://arcgraph.de/sr/