Subject: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

Describing arguments at top of a struct definition works fine
for small/medium size structs, but it definitely doesn't work well
for struct with a huge list of elements.

Keeping the arguments list inside the struct body makes it easier
to maintain the documentation.
ie:
/**
* struct my_struct - short description
* @a: first member
* @b: second member
*
* Longer description
*/
struct my_struct {
int a;
int b;
/**
* @c: This is longer description of C
*
* You can use paragraphs to describe arguments
* using this method.
*/
int c;
};

This patch allows the use of this kind of syntax. Only one argument
per comment and user can use how many paragraphs he needs. It should
start with /**, which is already being used by kernel-doc. If those
comment doesn't follow those rules, it will be ignored.

Signed-off-by: Danilo Cesar Lemes de Paula <[email protected]>
Cc: Randy Dunlap <[email protected]>
Cc: Daniel Vetter <[email protected]>
Cc: Laurent Pinchart <[email protected]>
Cc: Jonathan Corbet <[email protected]>
Cc: Herbert Xu <[email protected]>
Cc: Stephan Mueller <[email protected]>
Cc: Michal Marek <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: intel-gfx <[email protected]>
Cc: dri-devel <[email protected]>
---
scripts/kernel-doc | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 78 insertions(+), 2 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 9922e66..9bfa8b9 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -133,6 +133,30 @@ use strict;
#
# All descriptions can be multiline, except the short function description.
#
+# For really longs structs, you can also describe arguments inside the
+# body of the struct.
+# eg.
+# /**
+# * struct my_struct - short description
+# * @a: first member
+# * @b: second member
+# *
+# * Longer description
+# */
+# struct my_struct {
+# int a;
+# int b;
+# /**
+# * @c: This is longer description of C
+# *
+# * You can use paragraphs to describe arguments
+# * using this method.
+# */
+# int c;
+# };
+#
+# This should be use for arguments only.
+#
# You can also add additional sections. When documenting kernel functions you
# should document the "Context:" of the function, e.g. whether the functions
# can be called form interrupts. Unlike other sections you can end it with an
@@ -287,9 +311,19 @@ my $lineprefix="";
# 2 - scanning field start.
# 3 - scanning prototype.
# 4 - documentation block
+# 5 - gathering documentation outside main block
my $state;
my $in_doc_sect;

+# Split Doc State
+# 0 - Invalid (Before start or after finish)
+# 1 - Is started (the /** was found inside a struct)
+# 2 - The @parameter header was found, start accepting multi paragraph text.
+# 3 - Finished (the */ was found)
+# 4 - Error - Comment without header was found. Spit a warning as it's not
+# proper kernel-doc and ignore the rest.
+my $split_doc_state;
+
#declaration types: can be
# 'function', 'struct', 'union', 'enum', 'typedef'
my $decl_type;
@@ -304,6 +338,9 @@ my $doc_decl = $doc_com . '(\w+)';
my $doc_sect = $doc_com . '([' . $doc_special . ']?[\w\s]+):(.*)';
my $doc_content = $doc_com_body . '(.*)';
my $doc_block = $doc_com . 'DOC:\s*(.*)?';
+my $doc_split_start = '^\s*/\*\*\s*$';
+my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
+my $doc_split_end = '^\s*\*/\s*$';

my %constants;
my %parameterdescs;
@@ -2181,6 +2218,7 @@ sub reset_state {
$prototype = "";

$state = 0;
+ $split_doc_state = 0;
}

sub tracepoint_munge($) {
@@ -2453,7 +2491,6 @@ sub process_file($) {
}
$section = $newsection;
} elsif (/$doc_end/) {
-
if (($contents ne "") && ($contents ne "\n")) {
dump_section($file, $section, xml_escape($contents));
$section = $section_default;
@@ -2494,8 +2531,47 @@ sub process_file($) {
print STDERR "Warning(${file}:$.): bad line: $_";
++$warnings;
}
+ } elsif ($state == 5) { # scanning for split parameters
+
+ # First line (state 1) needs to be a @parameter
+ if ($split_doc_state == 1 && /$doc_split_sect/o) {
+ $section = $1;
+ $contents = $2;
+ if ($contents ne "") {
+ while ((substr($contents, 0, 1) eq " ") ||
+ substr($contents, 0, 1) eq "\t") {
+ $contents = substr($contents, 1);
+ }
+ $contents .= "\n";
+ }
+ $split_doc_state = 2;
+
+ # End commend */
+ } elsif (/$doc_split_end/) {
+ if (($contents ne "") && ($contents ne "\n")) {
+ dump_section($file, $section, xml_escape($contents));
+ $section = $section_default;
+ $contents = "";
+ }
+ $state = 3;
+ $split_doc_state = 0;
+
+ # Regular text
+ } elsif (/$doc_content/) {
+ if ($split_doc_state == 2) {
+ $contents .= $1 . "\n";
+ } elsif ($split_doc_state == 1) {
+ $split_doc_state = 4;
+ print STDERR "Warning(${file}:$.): ";
+ print STDERR "Incorrect use of kernel-doc format: $_";
+ ++$warnings;
+ }
+ }
} elsif ($state == 3) { # scanning for function '{' (end of prototype)
- if ($decl_type eq 'function') {
+ if (/$doc_split_start/) {
+ $state = 5;
+ $split_doc_state = 1;
+ } elsif ($decl_type eq 'function') {
process_state3_function($_, $file);
} else {
process_state3_type($_, $file);
--
2.4.6


2015-08-01 11:22:15

by Jonathan Corbet

[permalink] [raw]
Subject: Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

On Fri, 31 Jul 2015 18:06:45 -0300
Danilo Cesar Lemes de Paula <[email protected]> wrote:

> Describing arguments at top of a struct definition works fine
> for small/medium size structs, but it definitely doesn't work well
> for struct with a huge list of elements.
>
> Keeping the arguments list inside the struct body makes it easier
> to maintain the documentation.

Interesting approach. I think it could make sense, but I fear pushback
from a subset of maintainers refusing to accept this mode. I wonder what
it would take to get a consensus on allowing these in-struct comments?

I'm wondering if we need a kernel summit session on commenting
conventions, markdown-in-kerneldoc, etc? Maybe I'll stick a proposal out
there.

Thanks,

jon

2015-08-01 12:42:40

by Laurent Pinchart

[permalink] [raw]
Subject: Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

Hi Jon,

On Saturday 01 August 2015 13:22:10 Jonathan Corbet wrote:
> On Fri, 31 Jul 2015 18:06:45 -0300 Danilo Cesar Lemes de Paula wrote:
> > Describing arguments at top of a struct definition works fine
> > for small/medium size structs, but it definitely doesn't work well
> > for struct with a huge list of elements.
> >
> > Keeping the arguments list inside the struct body makes it easier
> > to maintain the documentation.
>
> Interesting approach. I think it could make sense, but I fear pushback
> from a subset of maintainers refusing to accept this mode. I wonder what
> it would take to get a consensus on allowing these in-struct comments?
>
> I'm wondering if we need a kernel summit session on commenting
> conventions, markdown-in-kerneldoc, etc? Maybe I'll stick a proposal out
> there.

The topic of documentation has been raised in the mail thread of the kernel
recruitment proposal for the kernel summit. I believe it's an important one,
and I will be very interested in contributing if it is discussed as a separate
topic.

--
Regards,

Laurent Pinchart

2015-08-03 08:23:26

by Daniel Vetter

[permalink] [raw]
Subject: Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

On Sat, Aug 01, 2015 at 01:22:10PM +0200, Jonathan Corbet wrote:
> On Fri, 31 Jul 2015 18:06:45 -0300
> Danilo Cesar Lemes de Paula <[email protected]> wrote:
>
> > Describing arguments at top of a struct definition works fine
> > for small/medium size structs, but it definitely doesn't work well
> > for struct with a huge list of elements.
> >
> > Keeping the arguments list inside the struct body makes it easier
> > to maintain the documentation.
>
> Interesting approach. I think it could make sense, but I fear pushback
> from a subset of maintainers refusing to accept this mode. I wonder what
> it would take to get a consensus on allowing these in-struct comments?

At least in drm we have a lot of such comments (as non-kerneldoc) right
above struct members to explain some details. Common examples are:
- locks, with a description of what they protect and maybe also how they
nest.
- vfunc ops structs, with a per-function description of what each hook
does.
- tricky stuff which can't be described in one sentence only.

So it's not just huge structs by number of members, but huge by number of
comment lines. Trying to stuff that all into the top kerneldoc comment
means that it's much harder to jump to the right comment, and it's also
easier to ignore the comments (since it e.g. won't show up in the diff
context).

The current approach at least in drm is to duplicate comments and that
just results in inconsistency.

> I'm wondering if we need a kernel summit session on commenting
> conventions, markdown-in-kerneldoc, etc? Maybe I'll stick a proposal out
> there.

Might be useful, but I'm not sure how many people really would actively
work on improving the tooling. The only comment I've seen is to maybe use
gtkdoc, but that would be a pain since it's slightly incompatible with
kerneldoc.

And it's the improved tooling I really need for my long-term plan to get
solid docs for drm & drm/i915. Next step is to start building a proper doc
writer team to make all the bits we already have into a consistent hole
(and nag developers to fill in the areas still undocumented). For that
I've already pulled Danilo's patches into the drm-intel integration tree
and I plan to use them for any further drm kerneldoc I write since we
really need them.

Thanks, Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

2015-08-03 14:37:48

by Jonathan Corbet

[permalink] [raw]
Subject: Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

On Mon, 3 Aug 2015 10:23:19 +0200
Daniel Vetter <[email protected]> wrote:

> > I'm wondering if we need a kernel summit session on commenting
> > conventions, markdown-in-kerneldoc, etc? Maybe I'll stick a proposal out
> > there.
>
> Might be useful, but I'm not sure how many people really would actively
> work on improving the tooling. The only comment I've seen is to maybe use
> gtkdoc, but that would be a pain since it's slightly incompatible with
> kerneldoc.

The idea was to get a sense for what sort of improvements would be
useful, to begin with. But my attempt to start a discussion on the
kernel summit list appears to have hit the ground pretty hard; I guess
that means I have free rein :)

I expect I'll apply the struct-args doc patch in the fairly near future.
Then we'll see if others complain when patches using it start to show up,
but the feature itself shouldn't break anything. I'm *really* hoping to
take a hard look at Danilo's stuff for a 4.3 merge as well. It should be
possible, but there's real-world obnoxiousness that is doing its best to
get in the way.

jon

2015-08-03 15:34:03

by Daniel Vetter

[permalink] [raw]
Subject: Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

On Mon, Aug 03, 2015 at 08:37:41AM -0600, Jonathan Corbet wrote:
> On Mon, 3 Aug 2015 10:23:19 +0200
> Daniel Vetter <[email protected]> wrote:
>
> > > I'm wondering if we need a kernel summit session on commenting
> > > conventions, markdown-in-kerneldoc, etc? Maybe I'll stick a proposal out
> > > there.
> >
> > Might be useful, but I'm not sure how many people really would actively
> > work on improving the tooling. The only comment I've seen is to maybe use
> > gtkdoc, but that would be a pain since it's slightly incompatible with
> > kerneldoc.
>
> The idea was to get a sense for what sort of improvements would be
> useful, to begin with. But my attempt to start a discussion on the
> kernel summit list appears to have hit the ground pretty hard; I guess
> that means I have free rein :)

Wrt feature wishlists the 3 things Danilo has worked on thus far
(hyperlinks, markdown and inline struct member kerneldoc) are really the
things I'd like to have. Of course there's room for some more
prettification, but I think that would better fit as improvements to
pandoc. One example is more flexible table handling with row/column
spanning - currently pandoc doesn't handle that in the docbook converter.

> I expect I'll apply the struct-args doc patch in the fairly near future.
> Then we'll see if others complain when patches using it start to show up,
> but the feature itself shouldn't break anything. I'm *really* hoping to
> take a hard look at Danilo's stuff for a 4.3 merge as well. It should be
> possible, but there's real-world obnoxiousness that is doing its best to
> get in the way.

Awesome. Missing 4.3 wouldn't be a big deal for i915 really since drm
feature freeze should happen around -rc5 anyway, so everything new I pull
in will be for 4.4 only. But getting it in early always helps, just in
case there's something unexpected.
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

2015-08-03 15:59:06

by Randy Dunlap

[permalink] [raw]
Subject: Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

On 07/31/15 14:06, Danilo Cesar Lemes de Paula wrote:
> Describing arguments at top of a struct definition works fine
> for small/medium size structs, but it definitely doesn't work well
> for struct with a huge list of elements.
>
> Keeping the arguments list inside the struct body makes it easier
> to maintain the documentation.
> ie:
> /**
> * struct my_struct - short description
> * @a: first member
> * @b: second member
> *
> * Longer description
> */
> struct my_struct {
> int a;
> int b;
> /**
> * @c: This is longer description of C
> *
> * You can use paragraphs to describe arguments
> * using this method.
> */
> int c;
> };
>
> This patch allows the use of this kind of syntax. Only one argument
> per comment and user can use how many paragraphs he needs. It should
> start with /**, which is already being used by kernel-doc. If those
> comment doesn't follow those rules, it will be ignored.
>
> Signed-off-by: Danilo Cesar Lemes de Paula <[email protected]>
> Cc: Randy Dunlap <[email protected]>
> Cc: Daniel Vetter <[email protected]>
> Cc: Laurent Pinchart <[email protected]>
> Cc: Jonathan Corbet <[email protected]>
> Cc: Herbert Xu <[email protected]>
> Cc: Stephan Mueller <[email protected]>
> Cc: Michal Marek <[email protected]>
> Cc: [email protected]
> Cc: [email protected]
> Cc: intel-gfx <[email protected]>
> Cc: dri-devel <[email protected]>
> ---
> scripts/kernel-doc | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
> 1 file changed, 78 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> index 9922e66..9bfa8b9 100755
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -133,6 +133,30 @@ use strict;
> #
> # All descriptions can be multiline, except the short function description.
> #
> +# For really longs structs, you can also describe arguments inside the
> +# body of the struct.
> +# eg.
> +# /**
> +# * struct my_struct - short description
> +# * @a: first member
> +# * @b: second member
> +# *
> +# * Longer description
> +# */
> +# struct my_struct {
> +# int a;
> +# int b;
> +# /**
> +# * @c: This is longer description of C
> +# *
> +# * You can use paragraphs to describe arguments
> +# * using this method.
> +# */
> +# int c;
> +# };
> +#
> +# This should be use for arguments only.

used

What are "arguments" in this context? do you mean struct fields or members?

> +#
> # You can also add additional sections. When documenting kernel functions you
> # should document the "Context:" of the function, e.g. whether the functions
> # can be called form interrupts. Unlike other sections you can end it with an
> @@ -287,9 +311,19 @@ my $lineprefix="";
> # 2 - scanning field start.
> # 3 - scanning prototype.
> # 4 - documentation block
> +# 5 - gathering documentation outside main block
> my $state;
> my $in_doc_sect;
>
> +# Split Doc State
> +# 0 - Invalid (Before start or after finish)
> +# 1 - Is started (the /** was found inside a struct)
> +# 2 - The @parameter header was found, start accepting multi paragraph text.
> +# 3 - Finished (the */ was found)
> +# 4 - Error - Comment without header was found. Spit a warning as it's not
> +# proper kernel-doc and ignore the rest.
> +my $split_doc_state;
> +
> #declaration types: can be
> # 'function', 'struct', 'union', 'enum', 'typedef'
> my $decl_type;
> @@ -304,6 +338,9 @@ my $doc_decl = $doc_com . '(\w+)';
> my $doc_sect = $doc_com . '([' . $doc_special . ']?[\w\s]+):(.*)';
> my $doc_content = $doc_com_body . '(.*)';
> my $doc_block = $doc_com . 'DOC:\s*(.*)?';
> +my $doc_split_start = '^\s*/\*\*\s*$';
> +my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
> +my $doc_split_end = '^\s*\*/\s*$';
>
> my %constants;
> my %parameterdescs;
> @@ -2181,6 +2218,7 @@ sub reset_state {
> $prototype = "";
>
> $state = 0;
> + $split_doc_state = 0;
> }
>
> sub tracepoint_munge($) {
> @@ -2453,7 +2491,6 @@ sub process_file($) {
> }
> $section = $newsection;
> } elsif (/$doc_end/) {
> -
> if (($contents ne "") && ($contents ne "\n")) {
> dump_section($file, $section, xml_escape($contents));
> $section = $section_default;
> @@ -2494,8 +2531,47 @@ sub process_file($) {
> print STDERR "Warning(${file}:$.): bad line: $_";
> ++$warnings;
> }
> + } elsif ($state == 5) { # scanning for split parameters
> +
> + # First line (state 1) needs to be a @parameter
> + if ($split_doc_state == 1 && /$doc_split_sect/o) {
> + $section = $1;
> + $contents = $2;
> + if ($contents ne "") {
> + while ((substr($contents, 0, 1) eq " ") ||
> + substr($contents, 0, 1) eq "\t") {
> + $contents = substr($contents, 1);
> + }
> + $contents .= "\n";
> + }
> + $split_doc_state = 2;
> +
> + # End commend */

comment

> + } elsif (/$doc_split_end/) {
> + if (($contents ne "") && ($contents ne "\n")) {
> + dump_section($file, $section, xml_escape($contents));
> + $section = $section_default;
> + $contents = "";
> + }
> + $state = 3;
> + $split_doc_state = 0;
> +

--
~Randy

Subject: Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body



On 08/03/2015 12:59 PM, Randy Dunlap wrote:
> On 07/31/15 14:06, Danilo Cesar Lemes de Paula wrote:
>> Describing arguments at top of a struct definition works fine
>> for small/medium size structs, but it definitely doesn't work well
>> for struct with a huge list of elements.
>>
>> Keeping the arguments list inside the struct body makes it easier
>> to maintain the documentation.
>> ie:
>> /**
>> * struct my_struct - short description
>> * @a: first member
>> * @b: second member
>> *
>> * Longer description
>> */
>> struct my_struct {
>> int a;
>> int b;
>> /**
>> * @c: This is longer description of C
>> *
>> * You can use paragraphs to describe arguments
>> * using this method.
>> */
>> int c;
>> };
>>
>> This patch allows the use of this kind of syntax. Only one argument
>> per comment and user can use how many paragraphs he needs. It should
>> start with /**, which is already being used by kernel-doc. If those
>> comment doesn't follow those rules, it will be ignored.
>>
>> Signed-off-by: Danilo Cesar Lemes de Paula <[email protected]>
>> Cc: Randy Dunlap <[email protected]>
>> Cc: Daniel Vetter <[email protected]>
>> Cc: Laurent Pinchart <[email protected]>
>> Cc: Jonathan Corbet <[email protected]>
>> Cc: Herbert Xu <[email protected]>
>> Cc: Stephan Mueller <[email protected]>
>> Cc: Michal Marek <[email protected]>
>> Cc: [email protected]
>> Cc: [email protected]
>> Cc: intel-gfx <[email protected]>
>> Cc: dri-devel <[email protected]>
>> ---
>> scripts/kernel-doc | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
>> 1 file changed, 78 insertions(+), 2 deletions(-)
>>
>> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
>> index 9922e66..9bfa8b9 100755
>> --- a/scripts/kernel-doc
>> +++ b/scripts/kernel-doc
>> @@ -133,6 +133,30 @@ use strict;
>> #
>> # All descriptions can be multiline, except the short function description.
>> #
>> +# For really longs structs, you can also describe arguments inside the
>> +# body of the struct.
>> +# eg.
>> +# /**
>> +# * struct my_struct - short description
>> +# * @a: first member
>> +# * @b: second member
>> +# *
>> +# * Longer description
>> +# */
>> +# struct my_struct {
>> +# int a;
>> +# int b;
>> +# /**
>> +# * @c: This is longer description of C
>> +# *
>> +# * You can use paragraphs to describe arguments
>> +# * using this method.
>> +# */
>> +# int c;
>> +# };
>> +#
>> +# This should be use for arguments only.
>
> used
>
> What are "arguments" in this context? do you mean struct fields or members?

Yes. I can change the text if you want to.

>
>> +#
>> # You can also add additional sections. When documenting kernel functions you
>> # should document the "Context:" of the function, e.g. whether the functions
>> # can be called form interrupts. Unlike other sections you can end it with an
>> @@ -287,9 +311,19 @@ my $lineprefix="";
>> # 2 - scanning field start.
>> # 3 - scanning prototype.
>> # 4 - documentation block
>> +# 5 - gathering documentation outside main block
>> my $state;
>> my $in_doc_sect;
>>
>> +# Split Doc State
>> +# 0 - Invalid (Before start or after finish)
>> +# 1 - Is started (the /** was found inside a struct)
>> +# 2 - The @parameter header was found, start accepting multi paragraph text.
>> +# 3 - Finished (the */ was found)
>> +# 4 - Error - Comment without header was found. Spit a warning as it's not
>> +# proper kernel-doc and ignore the rest.
>> +my $split_doc_state;
>> +
>> #declaration types: can be
>> # 'function', 'struct', 'union', 'enum', 'typedef'
>> my $decl_type;
>> @@ -304,6 +338,9 @@ my $doc_decl = $doc_com . '(\w+)';
>> my $doc_sect = $doc_com . '([' . $doc_special . ']?[\w\s]+):(.*)';
>> my $doc_content = $doc_com_body . '(.*)';
>> my $doc_block = $doc_com . 'DOC:\s*(.*)?';
>> +my $doc_split_start = '^\s*/\*\*\s*$';
>> +my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
>> +my $doc_split_end = '^\s*\*/\s*$';
>>
>> my %constants;
>> my %parameterdescs;
>> @@ -2181,6 +2218,7 @@ sub reset_state {
>> $prototype = "";
>>
>> $state = 0;
>> + $split_doc_state = 0;
>> }
>>
>> sub tracepoint_munge($) {
>> @@ -2453,7 +2491,6 @@ sub process_file($) {
>> }
>> $section = $newsection;
>> } elsif (/$doc_end/) {
>> -
>> if (($contents ne "") && ($contents ne "\n")) {
>> dump_section($file, $section, xml_escape($contents));
>> $section = $section_default;
>> @@ -2494,8 +2531,47 @@ sub process_file($) {
>> print STDERR "Warning(${file}:$.): bad line: $_";
>> ++$warnings;
>> }
>> + } elsif ($state == 5) { # scanning for split parameters
>> +
>> + # First line (state 1) needs to be a @parameter
>> + if ($split_doc_state == 1 && /$doc_split_sect/o) {
>> + $section = $1;
>> + $contents = $2;
>> + if ($contents ne "") {
>> + while ((substr($contents, 0, 1) eq " ") ||
>> + substr($contents, 0, 1) eq "\t") {
>> + $contents = substr($contents, 1);
>> + }
>> + $contents .= "\n";
>> + }
>> + $split_doc_state = 2;
>> +
>> + # End commend */
>
> comment

Fixed in my next v2.

Thanks

>
>> + } elsif (/$doc_split_end/) {
>> + if (($contents ne "") && ($contents ne "\n")) {
>> + dump_section($file, $section, xml_escape($contents));
>> + $section = $section_default;
>> + $contents = "";
>> + }
>> + $state = 3;
>> + $split_doc_state = 0;
>> +
>

2015-08-04 09:04:22

by Daniel Vetter

[permalink] [raw]
Subject: Re: [PATCH] scripts/kernel-doc Allow struct arguments documentation in struct body

On Fri, Jul 31, 2015 at 06:06:45PM -0300, Danilo Cesar Lemes de Paula wrote:
> Describing arguments at top of a struct definition works fine
> for small/medium size structs, but it definitely doesn't work well
> for struct with a huge list of elements.
>
> Keeping the arguments list inside the struct body makes it easier
> to maintain the documentation.
> ie:
> /**
> * struct my_struct - short description
> * @a: first member
> * @b: second member
> *
> * Longer description
> */
> struct my_struct {
> int a;
> int b;
> /**
> * @c: This is longer description of C
> *
> * You can use paragraphs to describe arguments
> * using this method.
> */
> int c;
> };
>
> This patch allows the use of this kind of syntax. Only one argument
> per comment and user can use how many paragraphs he needs. It should
> start with /**, which is already being used by kernel-doc. If those
> comment doesn't follow those rules, it will be ignored.
>
> Signed-off-by: Danilo Cesar Lemes de Paula <[email protected]>
> Cc: Randy Dunlap <[email protected]>
> Cc: Daniel Vetter <[email protected]>
> Cc: Laurent Pinchart <[email protected]>
> Cc: Jonathan Corbet <[email protected]>
> Cc: Herbert Xu <[email protected]>
> Cc: Stephan Mueller <[email protected]>
> Cc: Michal Marek <[email protected]>
> Cc: [email protected]
> Cc: [email protected]
> Cc: intel-gfx <[email protected]>
> Cc: dri-devel <[email protected]>
> ---
> scripts/kernel-doc | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
> 1 file changed, 78 insertions(+), 2 deletions(-)
>
> diff --git a/scripts/kernel-doc b/scripts/kernel-doc
> index 9922e66..9bfa8b9 100755
> --- a/scripts/kernel-doc
> +++ b/scripts/kernel-doc
> @@ -133,6 +133,30 @@ use strict;
> #
> # All descriptions can be multiline, except the short function description.
> #
> +# For really longs structs, you can also describe arguments inside the
> +# body of the struct.
> +# eg.
> +# /**
> +# * struct my_struct - short description
> +# * @a: first member
> +# * @b: second member
> +# *
> +# * Longer description
> +# */
> +# struct my_struct {
> +# int a;
> +# int b;
> +# /**
> +# * @c: This is longer description of C
> +# *
> +# * You can use paragraphs to describe arguments
> +# * using this method.
> +# */
> +# int c;
> +# };
> +#
> +# This should be use for arguments only.
> +#
> # You can also add additional sections. When documenting kernel functions you
> # should document the "Context:" of the function, e.g. whether the functions
> # can be called form interrupts. Unlike other sections you can end it with an
> @@ -287,9 +311,19 @@ my $lineprefix="";
> # 2 - scanning field start.
> # 3 - scanning prototype.
> # 4 - documentation block
> +# 5 - gathering documentation outside main block
> my $state;
> my $in_doc_sect;
>
> +# Split Doc State
> +# 0 - Invalid (Before start or after finish)
> +# 1 - Is started (the /** was found inside a struct)
> +# 2 - The @parameter header was found, start accepting multi paragraph text.
> +# 3 - Finished (the */ was found)
> +# 4 - Error - Comment without header was found. Spit a warning as it's not
> +# proper kernel-doc and ignore the rest.
> +my $split_doc_state;
> +
> #declaration types: can be
> # 'function', 'struct', 'union', 'enum', 'typedef'
> my $decl_type;
> @@ -304,6 +338,9 @@ my $doc_decl = $doc_com . '(\w+)';
> my $doc_sect = $doc_com . '([' . $doc_special . ']?[\w\s]+):(.*)';
> my $doc_content = $doc_com_body . '(.*)';
> my $doc_block = $doc_com . 'DOC:\s*(.*)?';
> +my $doc_split_start = '^\s*/\*\*\s*$';
> +my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
> +my $doc_split_end = '^\s*\*/\s*$';
>
> my %constants;
> my %parameterdescs;
> @@ -2181,6 +2218,7 @@ sub reset_state {
> $prototype = "";
>
> $state = 0;
> + $split_doc_state = 0;
> }
>
> sub tracepoint_munge($) {
> @@ -2453,7 +2491,6 @@ sub process_file($) {
> }
> $section = $newsection;
> } elsif (/$doc_end/) {
> -
> if (($contents ne "") && ($contents ne "\n")) {
> dump_section($file, $section, xml_escape($contents));
> $section = $section_default;
> @@ -2494,8 +2531,47 @@ sub process_file($) {
> print STDERR "Warning(${file}:$.): bad line: $_";
> ++$warnings;
> }
> + } elsif ($state == 5) { # scanning for split parameters
> +
> + # First line (state 1) needs to be a @parameter
> + if ($split_doc_state == 1 && /$doc_split_sect/o) {
> + $section = $1;
> + $contents = $2;

You're using a mix of tabs and spaces here to indent. Ofc we need 4 spaces
for odd indent levels, but for others it shouldn't be required.
-Daniel

> + if ($contents ne "") {
> + while ((substr($contents, 0, 1) eq " ") ||
> + substr($contents, 0, 1) eq "\t") {
> + $contents = substr($contents, 1);
> + }
> + $contents .= "\n";
> + }
> + $split_doc_state = 2;
> +
> + # End commend */
> + } elsif (/$doc_split_end/) {
> + if (($contents ne "") && ($contents ne "\n")) {
> + dump_section($file, $section, xml_escape($contents));
> + $section = $section_default;
> + $contents = "";
> + }
> + $state = 3;
> + $split_doc_state = 0;
> +
> + # Regular text
> + } elsif (/$doc_content/) {
> + if ($split_doc_state == 2) {
> + $contents .= $1 . "\n";
> + } elsif ($split_doc_state == 1) {
> + $split_doc_state = 4;
> + print STDERR "Warning(${file}:$.): ";
> + print STDERR "Incorrect use of kernel-doc format: $_";
> + ++$warnings;
> + }
> + }
> } elsif ($state == 3) { # scanning for function '{' (end of prototype)
> - if ($decl_type eq 'function') {
> + if (/$doc_split_start/) {
> + $state = 5;
> + $split_doc_state = 1;
> + } elsif ($decl_type eq 'function') {
> process_state3_function($_, $file);
> } else {
> process_state3_type($_, $file);
> --
> 2.4.6
>

--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

Subject: [PATCH v2] scripts/kernel-doc Allow struct arguments documentation in struct body

Describing arguments at top of a struct definition works fine
for small/medium size structs, but it definitely doesn't work well
for struct with a huge list of elements.

Keeping the arguments list inside the struct body makes it easier
to maintain the documentation.
ie:
/**
* struct my_struct - short description
* @a: first member
* @b: second member
*
* Longer description
*/
struct my_struct {
int a;
int b;
/**
* @c: This is longer description of C
*
* You can use paragraphs to describe arguments
* using this method.
*/
int c;
};

This patch allows the use of this kind of syntax. Only one argument
per comment and user can use how many paragraphs he needs. It should
start with /**, which is already being used by kernel-doc. If those
comment doesn't follow those rules, it will be ignored.

Signed-off-by: Danilo Cesar Lemes de Paula <[email protected]>
Cc: Randy Dunlap <[email protected]>
Cc: Daniel Vetter <[email protected]>
Cc: Laurent Pinchart <[email protected]>
Cc: Jonathan Corbet <[email protected]>
Cc: Herbert Xu <[email protected]>
Cc: Stephan Mueller <[email protected]>
Cc: Michal Marek <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: intel-gfx <[email protected]>
Cc: dri-devel <[email protected]>
---
Changelog:
v2:
- Fixed comment typo
- Fixed identantion by following the rest of the kernel-doc script pattern.
- Improved usage comment.

scripts/kernel-doc | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++++--
1 file changed, 75 insertions(+), 2 deletions(-)

diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index 9922e66..5adc7ee 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -133,6 +133,30 @@ use strict;
#
# All descriptions can be multiline, except the short function description.
#
+# For really longs structs, you can also describe arguments inside the
+# body of the struct.
+# eg.
+# /**
+# * struct my_struct - short description
+# * @a: first member
+# * @b: second member
+# *
+# * Longer description
+# */
+# struct my_struct {
+# int a;
+# int b;
+# /**
+# * @c: This is longer description of C
+# *
+# * You can use paragraphs to describe arguments
+# * using this method.
+# */
+# int c;
+# };
+#
+# This should be use only for struct/enum members.
+#
# You can also add additional sections. When documenting kernel functions you
# should document the "Context:" of the function, e.g. whether the functions
# can be called form interrupts. Unlike other sections you can end it with an
@@ -287,9 +311,19 @@ my $lineprefix="";
# 2 - scanning field start.
# 3 - scanning prototype.
# 4 - documentation block
+# 5 - gathering documentation outside main block
my $state;
my $in_doc_sect;

+# Split Doc State
+# 0 - Invalid (Before start or after finish)
+# 1 - Is started (the /** was found inside a struct)
+# 2 - The @parameter header was found, start accepting multi paragraph text.
+# 3 - Finished (the */ was found)
+# 4 - Error - Comment without header was found. Spit a warning as it's not
+# proper kernel-doc and ignore the rest.
+my $split_doc_state;
+
#declaration types: can be
# 'function', 'struct', 'union', 'enum', 'typedef'
my $decl_type;
@@ -304,6 +338,9 @@ my $doc_decl = $doc_com . '(\w+)';
my $doc_sect = $doc_com . '([' . $doc_special . ']?[\w\s]+):(.*)';
my $doc_content = $doc_com_body . '(.*)';
my $doc_block = $doc_com . 'DOC:\s*(.*)?';
+my $doc_split_start = '^\s*/\*\*\s*$';
+my $doc_split_sect = '\s*\*\s*(@[\w\s]+):(.*)';
+my $doc_split_end = '^\s*\*/\s*$';

my %constants;
my %parameterdescs;
@@ -2181,6 +2218,7 @@ sub reset_state {
$prototype = "";

$state = 0;
+ $split_doc_state = 0;
}

sub tracepoint_munge($) {
@@ -2453,7 +2491,6 @@ sub process_file($) {
}
$section = $newsection;
} elsif (/$doc_end/) {
-
if (($contents ne "") && ($contents ne "\n")) {
dump_section($file, $section, xml_escape($contents));
$section = $section_default;
@@ -2494,8 +2531,44 @@ sub process_file($) {
print STDERR "Warning(${file}:$.): bad line: $_";
++$warnings;
}
+ } elsif ($state == 5) { # scanning for split parameters
+ # First line (state 1) needs to be a @parameter
+ if ($split_doc_state == 1 && /$doc_split_sect/o) {
+ $section = $1;
+ $contents = $2;
+ if ($contents ne "") {
+ while ((substr($contents, 0, 1) eq " ") ||
+ substr($contents, 0, 1) eq "\t") {
+ $contents = substr($contents, 1);
+ }
+ $contents .= "\n";
+ }
+ $split_doc_state = 2;
+ # Documentation block end */
+ } elsif (/$doc_split_end/) {
+ if (($contents ne "") && ($contents ne "\n")) {
+ dump_section($file, $section, xml_escape($contents));
+ $section = $section_default;
+ $contents = "";
+ }
+ $state = 3;
+ $split_doc_state = 0;
+ # Regular text
+ } elsif (/$doc_content/) {
+ if ($split_doc_state == 2) {
+ $contents .= $1 . "\n";
+ } elsif ($split_doc_state == 1) {
+ $split_doc_state = 4;
+ print STDERR "Warning(${file}:$.): ";
+ print STDERR "Incorrect use of kernel-doc format: $_";
+ ++$warnings;
+ }
+ }
} elsif ($state == 3) { # scanning for function '{' (end of prototype)
- if ($decl_type eq 'function') {
+ if (/$doc_split_start/) {
+ $state = 5;
+ $split_doc_state = 1;
+ } elsif ($decl_type eq 'function') {
process_state3_function($_, $file);
} else {
process_state3_type($_, $file);
--
2.4.6

2015-08-06 19:13:16

by Jonathan Corbet

[permalink] [raw]
Subject: Re: [PATCH v2] scripts/kernel-doc Allow struct arguments documentation in struct body

On Tue, 4 Aug 2015 09:04:08 -0300
Danilo Cesar Lemes de Paula <[email protected]> wrote:

> Describing arguments at top of a struct definition works fine
> for small/medium size structs, but it definitely doesn't work well
> for struct with a huge list of elements.
>
> Keeping the arguments list inside the struct body makes it easier
> to maintain the documentation.

OK, this change has been merged into the docs tree.

Thanks,

jon