Since this discussion of getting driver authors to submit their driver
for inclusion I started writing a document to send them. I think it's
best included in the kernel tree.
Signed-off-by: Ben Collins <[email protected]>
--- /dev/null 2006-01-05 16:54:17.144000000 -0500
+++ b/Documentation/HOWTO-KernelMaintainer 2006-01-08 11:02:34.000000000 -0500
@@ -0,0 +1,150 @@
+ How to be a kernel driver maintainer
+ ------------------------------------
+
+
+This document explains what you must know before becoming the maintainer
+of a portion of the Linux kernel. Please read SubmittingPatches and
+SubmittingDrivers and CodingStyle, also in the Documentation/ directory.
+
+
+Why should I submit my driver?
+------------------------------
+
+This is often the the question a driver maintainer is faced with. Most
+driver authors really don't see the benefit of having their code in the
+main kernel.
+
+The primary benefit is availability. When people want to compile a kernel,
+they want to have everything there in the kernel tree. No one (not even
+kernel developers) likes having to search for, download, and build
+external drivers out-of-tree (outside the stock kernel source). It's often
+difficult to find the right driver (one known to work correctly), and is
+even harder to find one that works on the kernel version they are
+building.
+
+The benefit to users compiling their own kernel is immense. The benefit to
+distributions is even greater. Linux distributions already have a large
+amount of work to provide a kernel that works for most users. If a driver
+has to be provided for users that isn't in the primary kernel source, it
+adds additional work to maintaining (tracking the external driver,
+patching it into the build system, often times fixing build problems).
+With a driver in the kernel source, it's as simple as tracking the main
+kernel tree.
+
+This assumes that the distribution finds your driver worth the time of
+doing all this. If they don't, then the few users needing your driver will
+probably never get it (since most users are not capable of compiling their
+own modules).
+
+Having drivers in the main kernel tree benefits everyone.
+
+
+What should I do to prepare for code submission?
+------------------------------------------------
+
+First you need to inspect your code and make sure it meets criteria for
+inclusion. Read Documentation/CodingStyle for help on proper coding format
+(indentation, comment style, etc).
+
+Once you have properly formatted the code, you also need to check a few
+other areas. Most drivers include backward compatibility for older kernels
+(usually ifdef's with LINUX_VERSION_CODE). This backward compatibility
+needs to be removed. It's considered a waste of code for the driver to be
+backward compatible within the kernel source tree, since it is going to be
+compiled with a known version of the kernel.
+
+Proper location in the kernel source needs to be determined. Find drivers
+similar to yours, and use the same location. For example, USB network
+drivers go in drivers/usb/net/, and filesystems go in fs/.
+
+The driver should then be prepared for building from the main source tree
+in this location. A proper Makefile and Kconfig file in the Kbuild format
+should be provided. Most times it is enough to just add your entries to
+existing files. Here are some good rules to follow:
+
+ - If your driver is a single source file (or single .c with a single .h),
+ then it's typical to place the driver in an existing directory. Also,
+ modify existing Makefile/Kconfig for that directory.
+
+ - If your driver is made up of several source files, then it is typical
+ to create a subdirectory for it under the existing directory where it
+ applies. Separate Makefile should be included, with a reference in the
+ above Makefile to make sure to descend into the one you created.
+
+ + In this case, it is usually still correct to just add the Kconfig
+ entry to the existing one. Unless your driver has 2 or more config
+ options (debug options, extra features, etc), then creating a
+ standalone Kconfig may be best. Make sure to source this new Kconfig
+ from the parent directory.
+
+Lastly, you'll need to create an entry in the MAINTAINERS file. It should
+reference you or the team responsible for the code being submitted (this
+should be the same person/team submitting the code).
+
+
+Code review
+-----------
+
+Once your patches are ready, you can submit them to the linux-kernel
+mailing list. However, since most drivers fall under some subsystem (net,
+usb, etc), then it is often required that you also Cc the mailing list for
+this subsystem (see MAINTAINERS file for help finding the correct
+address).
+
+The code review process is there for two reasons. First, it ensures that
+only good code, that follows current API's and coding practices, gets into
+the kernel. The kernel developers know you have good intentions of
+maintaining your driver, but too often a driver is submitted to the
+kernel, and some time later becomes unmaintained. Then developers who are
+not familiar with the code or it's purpose are left with keeping it
+compiling and working. So the code needs to be readable, and easily
+modified.
+
+Secondly, the code review helps you to make your driver better. The folks
+looking at your code have been doing Linux kernel work for years, and are
+intimately familiar with all the nuances of the code. They can help with
+locking issues as well as big-endian/little-endian and 64-bit portability.
+
+Be prepared to take some heavy criticism. It's very rare than anyone comes
+out of this process without a scratch. Usually code review takes several
+tries. You'll need to follow the suggested changes, and make them to your
+code. Once you've made the changes required, resubmit. Try not to take it
+personally. The suggestions are meant to help you, your code, and your
+users (and is often times seen as a right of passage).
+
+
+What is expected of me after my driver is accepted?
+---------------------------------------------------
+
+The real work of maintainership begins after your code is in the tree.
+This is where some maintainers fail, and is the reason the kernel
+developers are so reluctant to allow new drivers into the main tree.
+
+There are two aspects of maintaining your driver in the kernel tree. The
+obvious first duty is to keep your code synced to the kernel source. This
+means submitting regular patch updates to the linux-kernel mailing list
+and to the particular tree maintainer (e.g. Linus or Andrew). Now that
+your code is included and properly styled and coded (with that shiny new
+driver smell), it should be fairly easy to keep it that way.
+
+The other side of the coin is keeping changes in the kernel synced to your
+code. Often times, it is necessary to change a kernel API (driver model,
+USB stack changes, networking subsystem change, etc). These sorts of
+changes usually affect a large number of drivers. It is not feasible for
+these changes to be individually submitted to the driver maintainers. So
+instead, the changes are made together in the kernel tree. If your driver
+is affected, you are expected to pick up these changes and merge them with
+your primary code (e.g. if you have a CVS repo for maintaining your code).
+Usually this job is made easier if you use the same source control system
+that the kernel maintainers use (currently, git), but this is not
+required.
+
+There are times where changes to your driver may happen that are not the
+API type of changes described above. A user of your driver may submit a
+patch directly to Linus to fix an obvious bug in the code. Sometimes these
+trivial and obvious patches will be accepted without feedback from the
+driver maintainer. Don't take this personally. We're all in this together.
+Just pick up the change and keep in sync with it. If you think the change
+was incorrect, try to find the mailing list thread or log comments
+regarding the change to see what was going on. Then email the patch author
+about the change to start discussion.
--
Ben Collins <[email protected]>
Developer
Ubuntu Linux
On Sun, 2006-01-08 at 11:07 -0500, Ben Collins wrote:
>
> +The other side of the coin is keeping changes in the kernel synced to
> your
> +code. Often times, it is necessary to change a kernel API (driver
> model,
> +USB stack changes, networking subsystem change, etc). These sorts of
> +changes usually affect a large number of drivers. It is not feasible
> for
> +these changes to be individually submitted to the driver maintainers.
> So
> +instead, the changes are made together in the kernel tree. If your
> driver
> +is affected, you are expected to pick up these changes and merge them
> with
> +your primary code (e.g. if you have a CVS repo for maintaining your
> code).
> +Usually this job is made easier if you use the same source control
> system
> +that the kernel maintainers use (currently, git), but this is not
> +required.
I don't quite agree with this part. This encourages cvs use, and "cvs
mentality". I *much* rather have something written as "the primary
location of your driver becomes the kernel.org git tree. This may feel
like you're giving away control, but it's not really. If you maintain
your driver there, people will still send patches via you for
approval/review. Of course you can keep a master copy in your own
version control repository, however be aware that most users will see
the kernel.org tree one as THE drivers. In addition, merging changes and
keeping uptodate is a lot harder that way. And worse, keeping the "main"
version outside the kernel.org tree tends to cause huge deviations and
backlogs between your main tree and the "real" kernel.org tree, with the
result that it becomes impossible to find regressions when you DO merge
the changes over.
On Sun, 2006-01-08 at 17:33 +0100, Arjan van de Ven wrote:
> On Sun, 2006-01-08 at 11:07 -0500, Ben Collins wrote:
> >
> > +The other side of the coin is keeping changes in the kernel synced to
> > your
> > +code. Often times, it is necessary to change a kernel API (driver
> > model,
> > +USB stack changes, networking subsystem change, etc). These sorts of
> > +changes usually affect a large number of drivers. It is not feasible
> > for
> > +these changes to be individually submitted to the driver maintainers.
> > So
> > +instead, the changes are made together in the kernel tree. If your
> > driver
> > +is affected, you are expected to pick up these changes and merge them
> > with
> > +your primary code (e.g. if you have a CVS repo for maintaining your
> > code).
> > +Usually this job is made easier if you use the same source control
> > system
> > +that the kernel maintainers use (currently, git), but this is not
> > +required.
>
> I don't quite agree with this part. This encourages cvs use, and "cvs
> mentality". I *much* rather have something written as "the primary
> location of your driver becomes the kernel.org git tree. This may feel
> like you're giving away control, but it's not really. If you maintain
> your driver there, people will still send patches via you for
> approval/review. Of course you can keep a master copy in your own
> version control repository, however be aware that most users will see
> the kernel.org tree one as THE drivers. In addition, merging changes and
> keeping uptodate is a lot harder that way. And worse, keeping the "main"
> version outside the kernel.org tree tends to cause huge deviations and
> backlogs between your main tree and the "real" kernel.org tree, with the
> result that it becomes impossible to find regressions when you DO merge
> the changes over.
But this isn't at al true. Almost all subsystems maintain the primary
tree outside of the kernel, with the kernel being the primary _stable_
tree. USB, Netdev, Alsa, etc. All changes go someplace else before being
pushed to the primary kernel tree. 99% of the time, patches are going
somewhere else before going into the main kernel. So the above
paragraphs is really misleading.
--
Ben Collins <[email protected]>
Developer
Ubuntu Linux
> But this isn't at al true. Almost all subsystems maintain the primary
> tree outside of the kernel, with the kernel being the primary _stable_
> tree. USB, Netdev,
patches yes. but usually only small stuff
> Alsa, etc. All changes go someplace else before being
> pushed to the primary kernel tree. 99% of the time, patches are going
> somewhere else before going into the main kernel.
that's different... that's a patch queue. That's not the same as being
the prime repository.
> So the above
> paragraphs is really misleading.
I guess neither is good then. I certainly would absolutely not want to
encourage developers to have a main "real driver" outside the kernel
source. Linus calls that "the cvs mentality" and time after time that
has proven to be really bad. You mention alsa, and to some degree alsa
is suffering from this ;(
(this is different from net/usb/scsi where changes are queued but merged
regularly and near immediately in case of bugfixes, unlike things like
alsa and firewire where basically the only choice is "all or nothing"
where "all" is bugfixes and new bugs)
On Sun, 2006-01-08 at 19:43 +0100, Arjan van de Ven wrote:
> > But this isn't at al true. Almost all subsystems maintain the primary
> > tree outside of the kernel, with the kernel being the primary _stable_
> > tree. USB, Netdev,
>
> patches yes. but usually only small stuff
>
> > Alsa, etc. All changes go someplace else before being
> > pushed to the primary kernel tree. 99% of the time, patches are going
> > somewhere else before going into the main kernel.
>
> that's different... that's a patch queue. That's not the same as being
> the prime repository.
this deserves expanding.
What net/usb/scsi queue is *deltas* to the kernel.org kernel. This is
fundamentally different from having the main driver be in its own
repository. Each delta is meant to do a certain change to the driver, eg
it's a CHANGE BASED thing. While "own repository" is "here is new
code" (even though you can disguise it as changes pretty well).
The linux development model is based on introducing changes, not on
introducing new code (of course the difference goes away if you
introduce a new driver, but that's a corner case)
The result is also highly different. In the net/usb/scsi case, there is
no "we need to move changes in mainline to our tree or they get lost".
The only thing needed would be resolving conflicts in the proposed
changes in the subsystem maintainers queue and the changes already in
mainline.
Note: this is independent of what kind of tool is used to store and
distribute such changes. quilt and git are used most, but git can also
be used in a CVS way if you want. But it's the "the main driver is in
the kernel, and we have proposed improvements to it" that counts (versus
"we have the main driver that we push to the kernel occasionally if we
feel like it").
Here's an updated document. I integrated the suggestions. For Arjan, I
added a new section at the end. Hopefully that addresses the concerns
for cvs-mentality.
--- /dev/null 2006-01-05 16:54:17.144000000 -0500
+++ b/Documentation/HOWTO-KernelMaintainer 2006-01-08 16:35:48.000000000 -0500
@@ -0,0 +1,206 @@
+ How to be a kernel driver maintainer
+ ------------------------------------
+
+
+This document explains what you must know before becoming the maintainer
+of a portion of the Linux kernel. Please read SubmittingPatches,
+SubmittingDrivers and CodingStyle, also in the Documentation/ directory.
+
+With the large amount of hardware available for Linux, it's becoming
+increasingly common for drivers for new or rare hardware to be maintained
+outside of the main kernel tree. Usually these drivers end up in the
+kernel tree once they are stable, but many times users and distribution
+maintainers are left with collecting these external drivers in order to
+support the required hardware.
+
+The purpose of this document is to provide information for the authors of
+these drivers to eventually have their code in the mainline kernel tree.
+
+
+Why should I submit my driver?
+------------------------------
+
+This is often the question a driver maintainer is faced with. Most driver
+authors really don't see the benefit of having their code in the main
+kernel. Some even see it as giving up control of their code. This is
+simply not the case, and then end result is always beneficial to users and
+developers alike.
+
+The primary benefit is availability. When people want to compile a kernel,
+they want to have everything there in the kernel tree. No one (not even
+kernel developers) likes having to search for, download, and build
+external drivers out-of-tree (outside the stock kernel source). It's often
+difficult to find the right driver (one known to work correctly), and is
+even harder to find one that works on the kernel version they are
+building.
+
+The benefit to users compiling their own kernel is immense. The benefit to
+distributions is even greater. Linux distributions already have a large
+amount of work to provide a kernel that works for most users. If a driver
+has to be provided for users that isn't in the primary kernel source, it
+adds additional work to maintaining (tracking the external driver,
+patching it into the build system, often times fixing build problems).
+With a driver in the kernel source, it's as simple as tracking the main
+kernel tree.
+
+This assumes that the distribution finds your driver worth the time of
+doing all this. If they don't, then the few users needing your driver will
+probably never get it (since most users are not capable of compiling their
+own modules).
+
+Another benefit of having the driver in the kernel tree is to promote the
+hardware that it supports. Many companies who have written drivers for
+their hardware to run under Linux have not yet taken the leap to placing
+the driver in the main kernel. The "Old Way" of providing downloadable
+drivers doesn't work as well for Linux, since it's almost impossible to
+provide pre-compiled versions for any given system. Having the driver in
+the kernel tree means it will always be available. It also means that
+users wishing to purchase hardware that "Just Works" with Linux will have
+more options. A well written and stable driver is a good reason for a user
+to choose that particular type of hardware.
+
+Having drivers in the main kernel tree benefits everyone.
+
+
+What should I do to prepare for code submission?
+------------------------------------------------
+
+First you need to inspect your code and make sure it meets criteria for
+inclusion. Read Documentation/CodingStyle for help on proper coding format
+(indentation, comment style, etc). It is strongly suggested that your
+driver builds cleanly when checked by the "sparse" tool. You will probably
+need to annotate the driver so sparse can tell that it is following the
+kernel's rules for address space accesses and endianness. Adding these
+annotations is a simple, but time-consuming, operation that often exposes
+real portability problems in drivers.
+
+Once you have properly formatted the code, you also need to check a few
+other areas. Most drivers include backward compatibility for older kernels
+(usually ifdef's with LINUX_VERSION_CODE). This backward compatibility
+needs to be removed. It's considered a waste of code for the driver to be
+backward compatible within the kernel source tree, since it is going to be
+compiled with a known version of the kernel.
+
+Proper location in the kernel source needs to be determined. Find drivers
+similar to yours, and use the same location. For example, USB network
+drivers go in drivers/usb/net/, and filesystems go in fs/.
+
+The driver should then be prepared for building from the main source tree
+in this location. A proper Makefile and Kconfig file in the Kbuild format
+should be provided. Most times it is enough to just add your entries to
+existing files. Here are some good rules to follow:
+
+ - If your driver is a single source file (or single .c with a single .h),
+ then it's typical to place the driver in an existing directory. Also,
+ modify existing Makefile/Kconfig for that directory.
+
+ - If your driver is made up of several source files, then it is typical
+ to create a subdirectory for it under the existing directory where it
+ applies. Separate Makefile should be included, with a reference in the
+ parent Makefile to make sure to descend into the one you created.
+
+ + In this case, it is usually still correct to just add the Kconfig
+ entry to the existing one. Unless your driver has 2 or more config
+ options (debug options, extra features, etc), then creating a
+ standalone Kconfig may be best. Make sure to source this new Kconfig
+ from the parent directory.
+
+ - When creating the Kconfig entries be sure to keep in mind the criteria
+ for the driver to be build. For example, a wireless network driver may
+ need to "depend on NET && IEEE80211". Also, if you driver is specific
+ to a certain architecture, be sure the Kconfig entry reflects this. DO
+ NOT force your driver to a specific architecture simply because the
+ driver is not written portably.
+
+Lastly, you'll need to create an entry in the MAINTAINERS file. It should
+reference you or the team responsible for the code being submitted (this
+should be the same person/team submitting the code).
+
+
+Code review
+-----------
+
+Once your patches are ready, you can submit them to the linux-kernel
+mailing list. However, since most drivers fall under some subsystem (net,
+usb, etc), then it is often required that you also Cc the mailing list for
+this subsystem (see MAINTAINERS file for help finding the correct
+address).
+
+The code review process is there for two reasons. First, it ensures that
+only good code, that follows current API's and coding practices, gets into
+the kernel. The kernel developers know you have good intentions of
+maintaining your driver, but too often a driver is submitted to the
+kernel, and some time later becomes unmaintained. Then developers who are
+not familiar with the code or it's purpose are left with keeping it
+compiling and working. So the code needs to be readable, and easily
+modified.
+
+Secondly, the code review helps you to make your driver better. The folks
+looking at your code have been doing Linux kernel work for years, and are
+intimately familiar with all the nuances of the code. They can help with
+locking issues as well as big-endian/little-endian and 64-bit portability.
+
+Be prepared to take some heavy criticism. It's very rare that anyone comes
+out of this process without a scratch. Usually code review takes several
+tries. You'll need to follow the suggested changes, and make these to your
+code, or have clear, acceptable reasons for *not* following the
+suggestions. Code reviewers are generally receptive to reasoned argument.
+If you do not follow a reviewer's initial suggestions, you should add
+descriptive comments to the appropriate parts of the driver, so that
+future contributors can understand why things are in a possibly unexpected
+state. Once you've made the changes required, resubmit. Try not to take it
+personally. The suggestions are meant to help you, your code, and your
+users (and is often times seen as a rite of passage).
+
+
+What is expected of me after my driver is accepted?
+---------------------------------------------------
+
+The real work of maintainership begins after your code is in the tree.
+This is where some maintainers fail, and is the reason the kernel
+developers are so reluctant to allow new drivers into the main tree.
+
+There are two aspects of maintaining your driver in the kernel tree. The
+obvious first duty is to keep your code synced to the kernel source. This
+means submitting regular patch updates to the linux-kernel mailing list
+and to the particular tree maintainer (e.g. Linus or Andrew). Now that
+your code is included and properly styled and coded (with that shiny new
+driver smell), it should be fairly easy to keep it that way.
+
+The other side of the coin is keeping changes in the kernel synced to your
+code. Often times, it is necessary to change a kernel API (driver model,
+USB stack changes, networking subsystem change, etc). These sorts of
+changes usually affect a large number of drivers. It is not feasible for
+these changes to be individually submitted to the driver maintainers. So
+instead, the changes are made together in the kernel tree. If your driver
+is affected, you are expected to pick up these changes and merge them with
+your primary code (e.g. if you have a CVS repo for maintaining your code).
+Usually this job is made easier if you use the same source control system
+that the kernel maintainers use (currently, git), but this is not
+required.
+
+There are times where changes to your driver may happen that are not the
+API type of changes described above. A user of your driver may submit a
+patch directly to Linus to fix an obvious bug in the code. Sometimes these
+trivial and obvious patches will be accepted without feedback from the
+driver maintainer. Don't take this personally. We're all in this together.
+Just pick up the change and keep in sync with it. If you think the change
+was incorrect, try to find the mailing list thread or log comments
+regarding the change to see what was going on. Then email the patch author
+about the change to start discussion.
+
+
+How should I maintain my code after it's in the kernel tree?
+------------------------------------------------------------
+
+The suggested, and certainly the easiest method, is to start a git tree
+cloned from the primary kernel tree. In this way, you are able to
+automatically track the kernel changes by pulling from Linus' tree. You
+can read more about maintaining a kernel git tree at
+http://linux.yyz.us/git-howto.html.
+
+Whatever you decide to use for keeping your kernel tree, just remember
+that the kernel tree source is the primary code. Your repository should
+mainly be used for queuing patches, and doing development. Users should
+not have to regularly go to your source in order to get a stable and
+usable driver.
--
Ben Collins <[email protected]>
Developer
Ubuntu Linux
FYI, I am keeping this document available here:
http://people.ubuntu.com/~bcollins/HOWTO-KernelMaintainer.txt
for anyone wishing to reference it via a stable URL.
--
Ben Collins <[email protected]>
Developer
Ubuntu Linux
On Sun, 2006-01-08 at 16:45 -0500, Ben Collins wrote:
> Here's an updated document. I integrated the suggestions. For Arjan, I
> added a new section at the end. Hopefully that addresses the concerns
> for cvs-mentality.
it doesn't enough ;(
you still do a major suggestion to keep the code in a repo outside the
kernel. For a single driver really that's at best "optional" and
shouldn't be the prime recommendation.
"If your driver is affected, you are expected to pick up these changes
and merge them with your primary code (e.g. if you have a CVS repo for
maintaining your code)."
that sentence is just really the one that I hate. It's bogus. It still
calls the private CVS copy "primary".
If you do the right thing (and store deltas against mainline and not
full code except for scratch stuff) then this is no question of "merging
back from mainline" at all.
On Sun, Jan 08, 2006 at 07:43:58PM +0100, Arjan van de Ven wrote:
> > Alsa, etc. All changes go someplace else before being
> > pushed to the primary kernel tree. 99% of the time, patches are going
> > somewhere else before going into the main kernel.
>
> that's different... that's a patch queue. That's not the same as being
> the prime repository.
As a data point, ocfs2 is dropping its subversion repository and
moving to exactly this model -- ocfs2 development is a set of patches
pending for mainline, with mainline as the prime repository. Really,
there's no other way to do it. Otherwise, you get way out of sync.
Joel
--
"The suffering man ought really to consume his own smoke; there is no
good in emitting smoke till you have made it into fire."
- thomas carlyle
Joel Becker
Principal Software Developer
Oracle
E-mail: [email protected]
Phone: (650) 506-8127
On Mon, 2006-01-09 at 08:46 +0100, Arjan van de Ven wrote:
> On Sun, 2006-01-08 at 16:45 -0500, Ben Collins wrote:
> > Here's an updated document. I integrated the suggestions. For Arjan, I
> > added a new section at the end. Hopefully that addresses the concerns
> > for cvs-mentality.
>
> it doesn't enough ;(
>
> you still do a major suggestion to keep the code in a repo outside the
> kernel. For a single driver really that's at best "optional" and
> shouldn't be the prime recommendation.
>
> "If your driver is affected, you are expected to pick up these changes
> and merge them with your primary code (e.g. if you have a CVS repo for
> maintaining your code)."
>
> that sentence is just really the one that I hate. It's bogus. It still
> calls the private CVS copy "primary".
> If you do the right thing (and store deltas against mainline and not
> full code except for scratch stuff) then this is no question of "merging
> back from mainline" at all.
But it says "your primary code". I'm not sure of another way to put it.
Obviously, they have to do their work, and their development on
something that isn't in Linus tree. If they are doing this work, they
need to make sure that when they diff for patches, that they merge
changes before diffing. The only way this is close to automatic is with
git. Any other method requires manually merging.
How else would you explain this without telling them that git is
required?
--
Ben Collins <[email protected]>
Developer
Ubuntu Linux
On Mon, 09 Jan 2006 08:34:09 -0500, Ben Collins <[email protected]> wrote:
>
>But it says "your primary code". I'm not sure of another way to put it.
>Obviously, they have to do their work, and their development on
>something that isn't in Linus tree. If they are doing this work, they
>need to make sure that when they diff for patches, that they merge
>changes before diffing. The only way this is close to automatic is with
>git. Any other method requires manually merging.
>
>How else would you explain this without telling them that git is
>required?
I have my name on one driver ;) Once the driver is in the kernel
I no longer need worry about the source nor backups of my driver,
okay? The primary code is in mainstream, this is how it works for
'leaf node' driver maintainers, at least from my limited perspective.
During development I had to rediff against -mm_latest, as that is
how drivers (usually?) start the path to mainstream. But the only
part need merging for a new driver is the Kconfig entry, a trivial
issue?
Cheers,
Grant.
On Mon, 2006-01-09 at 08:34 -0500, Ben Collins wrote:
> On Mon, 2006-01-09 at 08:46 +0100, Arjan van de Ven wrote:
> > On Sun, 2006-01-08 at 16:45 -0500, Ben Collins wrote:
> > > Here's an updated document. I integrated the suggestions. For Arjan, I
> > > added a new section at the end. Hopefully that addresses the concerns
> > > for cvs-mentality.
> >
> > it doesn't enough ;(
> >
> > you still do a major suggestion to keep the code in a repo outside the
> > kernel. For a single driver really that's at best "optional" and
> > shouldn't be the prime recommendation.
> >
> > "If your driver is affected, you are expected to pick up these changes
> > and merge them with your primary code (e.g. if you have a CVS repo for
> > maintaining your code)."
> >
> > that sentence is just really the one that I hate. It's bogus. It still
> > calls the private CVS copy "primary".
> > If you do the right thing (and store deltas against mainline and not
> > full code except for scratch stuff) then this is no question of "merging
> > back from mainline" at all.
>
> But it says "your primary code". I'm not sure of another way to put it.
"your temporary development copy"
> Obviously, they have to do their work, and their development on
> something that isn't in Linus tree. If they are doing this work, they
> need to make sure that when they diff for patches, that they merge
> changes before diffing. The only way this is close to automatic is with
> git. Any other method requires manually merging.
not correct. quilt is a very excellent counter example of that.
On Mon, 9 Jan 2006, Arjan van de Ven wrote:
>
> > Obviously, they have to do their work, and their development on
> > something that isn't in Linus tree. If they are doing this work, they
> > need to make sure that when they diff for patches, that they merge
> > changes before diffing. The only way this is close to automatic is with
> > git. Any other method requires manually merging.
>
> not correct. quilt is a very excellent counter example of that.
Yes. Anything that keeps a nice patch series that you can merge as a nice
patch series works fine.
For example, Al Viro used to use (still uses?) RCS to maintain his
work-in-progress. That worked fine, because he had a process where he
would just extract them as patches.
The reason CVS doesn't work well is partly because CVS just sucks at so
many levels, and because people start using it as more than a "series of
patches" repository. People might cherry-pick one or two changes from a
CVS, but it quickly becomes totally impossible to do anything sane at all,
or even to cherry-pick more than a few patches, because after a while
you've lost the ability to pick out individual changes.
Something like quilt works fine, because individual patches never get lost
in other patches (they might get merged with another patch on purpose, but
that's a separate issue). Anything that understands the notion of
changesets and can be taught to re-order them should be able to work the
same way.
So the important thing is to have _some_ proper linear changeset history,
preferably one where you can re-order them (so that if you cherry-pick a
set of changesets, you can mark them as having been merged, and keep the
_rest_ as a linear changeset history).
CVS just sucks. End of story. It works badly at so many levels that it's
just not even funny.
Linus
So is there a document I can point to that explains how to maintain your
code properly? It's somewhat out of scope, so I really don't want a
HOWTO within a HOWTO.
--
Ben Collins <[email protected]>
Developer
Ubuntu Linux
Hi!
> Since this discussion of getting driver authors to submit their driver
> for inclusion I started writing a document to send them. I think it's
> best included in the kernel tree.
>
> Signed-off-by: Ben Collins <[email protected]>
Well, nice document, but it does not really match the title. I
expected something like "how to be Jeff Garzik" and it is more "how to
get your driver into kernel and keep it there". It does not deal with
taking patches from other people, etc... I think it should go into
submitting driver howto or something like that.
Pavel
> --- /dev/null 2006-01-05 16:54:17.144000000 -0500
> +++ b/Documentation/HOWTO-KernelMaintainer 2006-01-08 11:02:34.000000000 -0500
> @@ -0,0 +1,150 @@
> + How to be a kernel driver maintainer
> + ------------------------------------
> +
> +
> +This document explains what you must know before becoming the maintainer
> +of a portion of the Linux kernel. Please read SubmittingPatches and
> +SubmittingDrivers and CodingStyle, also in the Documentation/ directory.
--
Thanks, Sharp!
On Mon, 2006-01-09 at 20:26 +0100, Pavel Machek wrote:
> Hi!
>
> > Since this discussion of getting driver authors to submit their driver
> > for inclusion I started writing a document to send them. I think it's
> > best included in the kernel tree.
> >
> > Signed-off-by: Ben Collins <[email protected]>
>
> Well, nice document, but it does not really match the title. I
> expected something like "how to be Jeff Garzik" and it is more "how to
> get your driver into kernel and keep it there". It does not deal with
> taking patches from other people, etc... I think it should go into
> submitting driver howto or something like that.
Guess it could be renamed "How to become a kernel driver maintainer"
--
Ben Collins <[email protected]>
Developer
Ubuntu Linux
Only small changes, title and a little rewording regarding "local
development copy". I will submit a patch if there are no more changes to
this document.
How to become a kernel driver maintainer
----------------------------------------
This document explains what you must know before becoming the maintainer
of a portion of the Linux kernel. Please read SubmittingPatches,
SubmittingDrivers and CodingStyle, also in the Documentation/ directory.
With the large amount of hardware available for Linux, it's becoming
increasingly common for drivers for new or rare hardware to be
maintained
outside of the main kernel tree. Usually these drivers end up in the
kernel tree once they are stable, but many times users and distribution
maintainers are left with collecting these external drivers in order to
support the required hardware.
The purpose of this document is to provide information for the authors
of
these drivers to eventually have their code in the mainline kernel tree.
Why should I submit my driver?
------------------------------
This is often the question a driver maintainer is faced with. Most
driver
authors really don't see the benefit of having their code in the main
kernel. Some even see it as giving up control of their code. This is
simply not the case, and then end result is always beneficial to users
and
developers alike.
The primary benefit is availability. When people want to compile a
kernel,
they want to have everything there in the kernel tree. No one (not even
kernel developers) likes having to search for, download, and build
external drivers out-of-tree (outside the stock kernel source). It's
often
difficult to find the right driver (one known to work correctly), and is
even harder to find one that works on the kernel version they are
building.
The benefit to users compiling their own kernel is immense. The benefit
to
distributions is even greater. Linux distributions already have a large
amount of work to provide a kernel that works for most users. If a
driver
has to be provided for users that isn't in the primary kernel source, it
adds additional work to maintaining (tracking the external driver,
patching it into the build system, often times fixing build problems).
With a driver in the kernel source, it's as simple as tracking the main
kernel tree.
This assumes that the distribution finds your driver worth the time of
doing all this. If they don't, then the few users needing your driver
will
probably never get it (since most users are not capable of compiling
their
own modules).
Another benefit of having the driver in the kernel tree is to promote
the
hardware that it supports. Many companies who have written drivers for
their hardware to run under Linux have not yet taken the leap to placing
the driver in the main kernel. The "Old Way" of providing downloadable
drivers doesn't work as well for Linux, since it's almost impossible to
provide pre-compiled versions for any given system. Having the driver in
the kernel tree means it will always be available. It also means that
users wishing to purchase hardware that "Just Works" with Linux will
have
more options. A well written and stable driver is a good reason for a
user
to choose that particular type of hardware.
Having drivers in the main kernel tree benefits everyone.
What should I do to prepare for code submission?
------------------------------------------------
First you need to inspect your code and make sure it meets criteria for
inclusion. Read Documentation/CodingStyle for help on proper coding
format
(indentation, comment style, etc). It is strongly suggested that your
driver builds cleanly when checked by the "sparse" tool. You will
probably
need to annotate the driver so sparse can tell that it is following the
kernel's rules for address space accesses and endianness. Adding these
annotations is a simple, but time-consuming, operation that often
exposes
real portability problems in drivers.
Once you have properly formatted the code, you also need to check a few
other areas. Most drivers include backward compatibility for older
kernels
(usually ifdef's with LINUX_VERSION_CODE). This backward compatibility
needs to be removed. It's considered a waste of code for the driver to
be
backward compatible within the kernel source tree, since it is going to
be
compiled with a known version of the kernel.
Proper location in the kernel source needs to be determined. Find
drivers
similar to yours, and use the same location. For example, USB network
drivers go in drivers/usb/net/, and filesystems go in fs/.
The driver should then be prepared for building from the main source
tree
in this location. A proper Makefile and Kconfig file in the Kbuild
format
should be provided. Most times it is enough to just add your entries to
existing files. Here are some good rules to follow:
- If your driver is a single source file (or single .c with a
single .h),
then it's typical to place the driver in an existing directory. Also,
modify existing Makefile/Kconfig for that directory.
- If your driver is made up of several source files, then it is typical
to create a subdirectory for it under the existing directory where it
applies. Separate Makefile should be included, with a reference in
the
parent Makefile to make sure to descend into the one you created.
+ In this case, it is usually still correct to just add the Kconfig
entry to the existing one. Unless your driver has 2 or more config
options (debug options, extra features, etc), then creating a
standalone Kconfig may be best. Make sure to source this new
Kconfig
from the parent directory.
- When creating the Kconfig entries be sure to keep in mind the
criteria
for the driver to be build. For example, a wireless network driver
may
need to "depend on NET && IEEE80211". Also, if you driver is
specific
to a certain architecture, be sure the Kconfig entry reflects this.
DO
NOT force your driver to a specific architecture simply because the
driver is not written portably.
Lastly, you'll need to create an entry in the MAINTAINERS file. It
should
reference you or the team responsible for the code being submitted (this
should be the same person/team submitting the code).
Code review
-----------
Once your patches are ready, you can submit them to the linux-kernel
mailing list. However, since most drivers fall under some subsystem
(net,
usb, etc), then it is often required that you also Cc the mailing list
for
this subsystem (see MAINTAINERS file for help finding the correct
address).
The code review process is there for two reasons. First, it ensures that
only good code, that follows current API's and coding practices, gets
into
the kernel. The kernel developers know you have good intentions of
maintaining your driver, but too often a driver is submitted to the
kernel, and some time later becomes unmaintained. Then developers who
are
not familiar with the code or it's purpose are left with keeping it
compiling and working. So the code needs to be readable, and easily
modified.
Secondly, the code review helps you to make your driver better. The
folks
looking at your code have been doing Linux kernel work for years, and
are
intimately familiar with all the nuances of the code. They can help with
locking issues as well as big-endian/little-endian and 64-bit
portability.
Be prepared to take some heavy criticism. It's very rare that anyone
comes
out of this process without a scratch. Usually code review takes several
tries. You'll need to follow the suggested changes, and make these to
your
code, or have clear, acceptable reasons for *not* following the
suggestions. Code reviewers are generally receptive to reasoned
argument.
If you do not follow a reviewer's initial suggestions, you should add
descriptive comments to the appropriate parts of the driver, so that
future contributors can understand why things are in a possibly
unexpected
state. Once you've made the changes required, resubmit. Try not to take
it
personally. The suggestions are meant to help you, your code, and your
users (and is often times seen as a rite of passage).
What is expected of me after my driver is accepted?
---------------------------------------------------
The real work of maintainership begins after your code is in the tree.
This is where some maintainers fail, and is the reason the kernel
developers are so reluctant to allow new drivers into the main tree.
There are two aspects of maintaining your driver in the kernel tree. The
obvious first duty is to keep your code synced to the kernel source.
This
means submitting regular patch updates to the linux-kernel mailing list
and to the particular tree maintainer (e.g. Linus or Andrew). Now that
your code is included and properly styled and coded (with that shiny new
driver smell), it should be fairly easy to keep it that way.
The other side of the coin is keeping changes in the kernel synced to
your
code. Often times, it is necessary to change a kernel API (driver model,
USB stack changes, networking subsystem change, etc). These sorts of
changes usually affect a large number of drivers. It is not feasible for
these changes to be individually submitted to the driver maintainers. So
instead, the changes are made together in the kernel tree. If your
driver
is affected, you are expected to pick up these changes and merge them
with
your temporary development copy. Usually this job is made easier if you
use the same source control system that the kernel maintainers use
(currently, git), but this is not required. Using git, however, allows
you
to merge more easily.
There are times where changes to your driver may happen that are not the
API type of changes described above. A user of your driver may submit a
patch directly to Linus to fix an obvious bug in the code. Sometimes
these
trivial and obvious patches will be accepted without feedback from the
driver maintainer. Don't take this personally. We're all in this
together.
Just pick up the change and keep in sync with it. If you think the
change
was incorrect, try to find the mailing list thread or log comments
regarding the change to see what was going on. Then email the patch
author
about the change to start discussion.
How should I maintain my code after it's in the kernel tree?
------------------------------------------------------------
The suggested, and certainly the easiest method, is to start a git tree
cloned from the primary kernel tree. In this way, you are able to
automatically track the kernel changes by pulling from Linus' tree. You
can read more about maintaining a kernel git tree at
http://linux.yyz.us/git-howto.html.
Whatever you decide to use for keeping your kernel tree, just remember
that the kernel tree source is the primary code. Your repository should
mainly be used for queuing patches, and doing development. Users should
not have to regularly go to your source in order to get a stable and
usable driver.
--
Ubuntu - http://www.ubuntu.com/
Debian - http://www.debian.org/
Linux 1394 - http://www.linux1394.org/
SwissDisk - http://www.swissdisk.com/
On 3/8/06, Ben Collins <[email protected]> wrote:
Nice work Ben. A few comments and suggestions below.
> authors really don't see the benefit of having their code in the main
> kernel. Some even see it as giving up control of their code. This is
> simply not the case, and then end result is always beneficial to users
s/and then end result/and the end result /
>
> What should I do to prepare for code submission?
> ------------------------------------------------
>
> First you need to inspect your code and make sure it meets criteria for
> inclusion. Read Documentation/CodingStyle for help on proper coding
> format
> (indentation, comment style, etc). It is strongly suggested that your
> driver builds cleanly when checked by the "sparse" tool. You will
suggested text:
driver builds cleanly without warnings from the compiler or the "sparse" tool.
>
> - When creating the Kconfig entries be sure to keep in mind the
> criteria
> for the driver to be build. For example, a wireless network driver
> may
> need to "depend on NET && IEEE80211". Also, if you driver is
> specific
> to a certain architecture, be sure the Kconfig entry reflects this.
> DO
> NOT force your driver to a specific architecture simply because the
> driver is not written portably.
>
Suggested addition:
Also make sure you provide useful help text for every entry you add
to Kconfig so that users of your driver will be able to read about
what it does, what hardware it supports and perhaps find a reference
to more extensive documentation.
> Lastly, you'll need to create an entry in the MAINTAINERS file. It
> should
> reference you or the team responsible for the code being submitted (this
> should be the same person/team submitting the code).
>
Suggested addition:
Optionally you may also want to add an entry to the CREDITS file.
> not familiar with the code or it's purpose are left with keeping it
> compiling and working. So the code needs to be readable, and easily
> modified.
>
s/modified/modifiable/
> Secondly, the code review helps you to make your driver better. The
> folks
s/folks/people/ ???
> There are times where changes to your driver may happen that are not the
> API type of changes described above. A user of your driver may submit a
> patch directly to Linus to fix an obvious bug in the code. Sometimes
> these
> trivial and obvious patches will be accepted without feedback from the
> driver maintainer. Don't take this personally. We're all in this
> together.
> Just pick up the change and keep in sync with it. If you think the
> change
> was incorrect, try to find the mailing list thread or log comments
> regarding the change to see what was going on. Then email the patch
> author
> about the change to start discussion.
>
Perhaps add a bit of text here about integrating patches send to you,
as maintainer of the driver, by random people.
--
Jesper Juhl <[email protected]>
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please http://www.expita.com/nomime.html
> > about the change to start discussion.
> >
> Perhaps add a bit of text here about integrating patches send to you,
> as maintainer of the driver, by random people.
it's not integrating.
It's reviewing and passing on for merge ;)
fundamental difference. You don't "own" the driver in the tree. You just
keep it nice and shiny and clean. Best done by blessing patches, and by
developing in "patches" not "new codebase". The entire "new codebase"
thinking is the problem with CVS-think. Think in patches (once merged),
not in code/tree.
On 3/8/06, Arjan van de Ven <[email protected]> wrote:
>
> > > about the change to start discussion.
> > >
> > Perhaps add a bit of text here about integrating patches send to you,
> > as maintainer of the driver, by random people.
>
> it's not integrating.
> It's reviewing and passing on for merge ;)
> fundamental difference. You don't "own" the driver in the tree. You just
> keep it nice and shiny and clean. Best done by blessing patches, and by
> developing in "patches" not "new codebase". The entire "new codebase"
> thinking is the problem with CVS-think. Think in patches (once merged),
> not in code/tree.
>
Yeah, right, that's basically what I meant even if it's not what I
actually said.
I just wanted to say that it should be described that you also have a
responsability as maintainer for working with other people, review
their patches, sign off on them, make sure they get picked up and
forwarded to the proper people etc etc.
Thank you for making it more clear.
--
Jesper Juhl <[email protected]>
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please http://www.expita.com/nomime.html
On Wed, 08 Mar 2006 13:03:33 -0500 Ben Collins wrote:
> Only small changes, title and a little rewording regarding "local
> development copy". I will submit a patch if there are no more changes to
> this document.
in addition to other comments:
> Another benefit of having the driver in the kernel tree is to promote
> the
> hardware that it supports. Many companies who have written drivers for
s/who/that/ (companies are not a person; "who" is for persons)
> their hardware to run under Linux have not yet taken the leap to placing
> the driver in the main kernel. The "Old Way" of providing downloadable
> drivers doesn't work as well for Linux, since it's almost impossible to
> provide pre-compiled versions for any given system. Having the driver in
> the kernel tree means it will always be available. It also means that
> users wishing to purchase hardware that "Just Works" with Linux will
> have
> more options. A well written and stable driver is a good reason for a
well-written
> user
> to choose that particular type of hardware.
>
> Having drivers in the main kernel tree benefits everyone.
>
>
> What should I do to prepare for code submission?
> ------------------------------------------------
>
> First you need to inspect your code and make sure it meets criteria for
> inclusion. Read Documentation/CodingStyle for help on proper coding
> format
> (indentation, comment style, etc). It is strongly suggested that your
> driver builds cleanly when checked by the "sparse" tool. You will
> probably
> need to annotate the driver so sparse can tell that it is following the
> kernel's rules for address space accesses and endianness. Adding these
> annotations is a simple, but time-consuming, operation that often
> exposes
> real portability problems in drivers.
>
> Once you have properly formatted the code, you also need to check a few
> other areas. Most drivers include backward compatibility for older
> kernels
> (usually ifdef's with LINUX_VERSION_CODE). This backward compatibility
> needs to be removed. It's considered a waste of code for the driver to
> be
> backward compatible within the kernel source tree, since it is going to
> be
> compiled with a known version of the kernel.
Also use "make checkstack", "make buildcheck", and "make namespacecheck"
to check for problems that they can detect.
> Proper location in the kernel source needs to be determined. Find
> drivers
> similar to yours, and use the same location. For example, USB network
> drivers go in drivers/usb/net/, and filesystems go in fs/.
>
> The driver should then be prepared for building from the main source
> tree
> in this location. A proper Makefile and Kconfig file in the Kbuild
> format
and Kbuild Makefile & Kconfig files are described in Documentation/kbuild/*
> should be provided. Most times it is enough to just add your entries to
> existing files. Here are some good rules to follow:
>
> - If your driver is a single source file (or single .c with a
> single .h),
> then it's typical to place the driver in an existing directory. Also,
> modify existing Makefile/Kconfig for that directory.
>
> - If your driver is made up of several source files, then it is typical
> to create a subdirectory for it under the existing directory where it
> applies. Separate Makefile should be included, with a reference in
> the
> parent Makefile to make sure to descend into the one you created.
>
> + In this case, it is usually still correct to just add the Kconfig
> entry to the existing one. Unless your driver has 2 or more config
s/Unless/If/
> options (debug options, extra features, etc), then creating a
> standalone Kconfig may be best. Make sure to source this new
> Kconfig
> from the parent directory.
>
> - When creating the Kconfig entries be sure to keep in mind the
> criteria
> for the driver to be build. For example, a wireless network driver
> may
s/build/built/
> need to "depend on NET && IEEE80211". Also, if you driver is
> specific
s/you/your/
> to a certain architecture, be sure the Kconfig entry reflects this.
> DO
> NOT force your driver to a specific architecture simply because the
> driver is not written portably.
>
> Lastly, you'll need to create an entry in the MAINTAINERS file. It
> should
> reference you or the team responsible for the code being submitted (this
> should be the same person/team submitting the code).
and the mailing list that should be used for correspondence.
> Code review
> -----------
>
> Once your patches are ready, you can submit them to the linux-kernel
> mailing list. However, since most drivers fall under some subsystem
> (net,
> usb, etc), then it is often required that you also Cc the mailing list
> for
> this subsystem (see MAINTAINERS file for help finding the correct
> address).
I disagree that lkml should always be used. It is becoming terribly
over-burdened recently so either people are spending more time reading
too much email or they are ignoring it. It would be better IMO to
send patches for focused mailing lists that are appropriate for the
driver.
> The code review process is there for two reasons. First, it ensures that
> only good code, that follows current API's and coding practices, gets
> into
> the kernel. The kernel developers know you have good intentions of
> maintaining your driver, but too often a driver is submitted to the
> kernel, and some time later becomes unmaintained. Then developers who
> are
> not familiar with the code or it's purpose are left with keeping it
s/it's/its/
> compiling and working. So the code needs to be readable, and easily
> modified.
> What is expected of me after my driver is accepted?
> ---------------------------------------------------
>
> The real work of maintainership begins after your code is in the tree.
> This is where some maintainers fail, and is the reason the kernel
> developers are so reluctant to allow new drivers into the main tree.
I disagree with that last part. I guess I have seen some reluctance
occasionally, but I don't think that it's the (main) reason for
drivers not being accepted into the kernel tree.
> How should I maintain my code after it's in the kernel tree?
> ------------------------------------------------------------
>
> Whatever you decide to use for keeping your kernel tree, just remember
> that the kernel tree source is the primary code. Your repository should
> mainly be used for queuing patches, and doing development. Users should
> not have to regularly go to your source in order to get a stable and
> usable driver.
amen.
---
~Randy