2019-04-23 14:54:59

by David Howells

[permalink] [raw]
Subject: Re: [PATCH v2 56/79] docs: Documentation/*.txt: rename all ReST files to *.rst

Mike Snitzer <[email protected]> wrote:

> Peter Zijlstra <[email protected]> wrote:
>
> > But yes, I have 0 motivation to learn or abide by rst. It simply doesn't
> > give me anything in return. There is no upside, only worse text files :/
>
> Right, but these changes aren't meant for our benefit. They are for
> users who get cleaner web accessible Linux kernel docs. Seems the
> decision has been made that the users' benefit, and broader
> modernization of Linux docs, outweighs the inconvenience for engineers
> who maintain the content of said documentation.

Whilst I can sympathise with Mauro and Jon - and appreciate the hard work
they've put into this, I do think that the engineers dealing directly with the
kernel code should be considered the primary audience.

There've been some changes that I've particularly objected to, such as
removing contents lists from files and replacing them with markup like:

.. contents:: :local:

This actually impedes use of the file. It should not be necessary to build
the docs to get that for ordinary use.


Anyway, the biggest doc issue in the kernel isn't addressed by the conversion
to ReST: and that is that most people don't seem interested in documenting
stuff - whether because writing documentation isn't as fun as writing code or
the fact that English isn't their native language, I don't know. I can
sympathise more with the latter.

Kerneldoc is a start - and probably means that a lot of API functions are at
least slightly documented - but too many APIs are not mentioned in the
Documentation directory at all. Remember: if you can't describe it, it's
probably wrong!

I'm not sure what we could do about this, but it would probably have to be
imposed from the top: no more undocumented APIs. Any new API must come with
documentation; changes to APIs must include changes to the documentation.

If you really want to upset people, you could add: anyone who wants to alter
an already existing undocumented API must supply documentation for the whole
API (but that could be considered a bit cruel).

And anyone who says "But the code is the documentation!" needs to consider
carefully what happens to their code after it has been trampled, generalised,
split, combined, renormalised, cthulhuised, janitorised and had parts of it
migrate.

</rant>

And now, after that, I think a fresh cup of tea is called for!

David


2019-04-23 16:55:29

by Jonathan Corbet

[permalink] [raw]
Subject: Re: [PATCH v2 56/79] docs: Documentation/*.txt: rename all ReST files to *.rst

On Tue, 23 Apr 2019 15:52:26 +0100
David Howells <[email protected]> wrote:

> There've been some changes that I've particularly objected to, such as
> removing contents lists from files and replacing them with markup like:
>
> .. contents:: :local:
>
> This actually impedes use of the file. It should not be necessary to build
> the docs to get that for ordinary use.

Usability of the text files versus that of the built docs is occasionally
something that has to be traded off. As a general rule, I want the text
files to remain useful on their own. There is a lot of value in the
built docs for a lot of people, but that should not be the only, or even
the primary, form of access

Tables of contents are certainly a place where that tradeoff makes itself
felt. Doing them by hand ensures that they are always present, but
requires that people editing the docs also maintain the TOCS - something
that experience has shown tends not to happen. That's more of a pain
than a little bit of markup, and people don't do it. An automatically
generated TOC, instead, is always correct and is linkable.

Few people complain about the biggest impediment to the readability of
text files, though: the use of kerneldoc comments. That splits the
information between the text file and multiple random-seeming locations
among tends of thousands of source files. Sometimes the solution here is
to move all of the documentation into the source, but that tends to
fragment it and make it harder to find; it's certainly not the right
place for many kinds of docs. In general, it's hard to create a coherent
story that way.

Suggestions / patches on how to improve things for *all* users of the
docs are certainly welcome!

I am, incidentally, toying with the idea of trying to put together a
documentation microconf at the Linux Plumbers Conference this year. If
anybody out there thinks that's a good idea and would like to
participate, please let me know.

> Anyway, the biggest doc issue in the kernel isn't addressed by the conversion
> to ReST: and that is that most people don't seem interested in documenting
> stuff - whether because writing documentation isn't as fun as writing code or
> the fact that English isn't their native language, I don't know. I can
> sympathise more with the latter.

I believe there's enough experience now to say that using RST increases
contributions to the documentation. Just having something that approaches
a coherent approach to docs (though we are a *long* way from that still)
makes the whole thing more accessible. It doesn't solve our
documentation issues by any stretch, but it helps.

Thanks,

jon

2019-04-23 17:14:10

by Peter Zijlstra

[permalink] [raw]
Subject: Re: [PATCH v2 56/79] docs: Documentation/*.txt: rename all ReST files to *.rst

On Tue, Apr 23, 2019 at 10:54:15AM -0600, Jonathan Corbet wrote:
> Few people complain about the biggest impediment to the readability of
> text files, though: the use of kerneldoc comments.

I just stopped using kerneldoc.

2019-04-23 19:40:20

by Mauro Carvalho Chehab

[permalink] [raw]
Subject: Re: [PATCH v2 56/79] docs: Documentation/*.txt: rename all ReST files to *.rst

Em Tue, 23 Apr 2019 15:52:26 +0100
David Howells <[email protected]> escreveu:

> Mike Snitzer <[email protected]> wrote:
>
> > Peter Zijlstra <[email protected]> wrote:
> >
> > > But yes, I have 0 motivation to learn or abide by rst. It simply doesn't
> > > give me anything in return. There is no upside, only worse text files :/
> >
> > Right, but these changes aren't meant for our benefit. They are for
> > users who get cleaner web accessible Linux kernel docs. Seems the
> > decision has been made that the users' benefit, and broader
> > modernization of Linux docs, outweighs the inconvenience for engineers
> > who maintain the content of said documentation.
>
> Whilst I can sympathise with Mauro and Jon - and appreciate the hard work
> they've put into this, I do think that the engineers dealing directly with the
> kernel code should be considered the primary audience.
>
> There've been some changes that I've particularly objected to, such as
> removing contents lists from files and replacing them with markup like:
>
> .. contents:: :local:

Actually, just:

.. contents

Is enough. The goal here is just to avoid having two indexes when
generating an html output - and sometimes the internal contents index
is broken - I got one of these during this patch series: a very
confusing internal "contents" that doesn't really match the contents
of the file - probably because some patch changed the contents but
didn't update the internal index.

>
> This actually impedes use of the file. It should not be necessary to build
> the docs to get that for ordinary use.

IMHO, replacing:

Contents

1. Introduction
...

By:


.. Contents

1. Introduction
...

won't make the document harder to read as plain text file. With html
output, the contents will be at both the index.html and at the lateral
bar, so the text "Contents" can be hidden (and that's what the two points
make: it transforms the block into a comment).

>
>
> Anyway, the biggest doc issue in the kernel isn't addressed by the conversion
> to ReST: and that is that most people don't seem interested in documenting
> stuff - whether because writing documentation isn't as fun as writing code or
> the fact that English isn't their native language, I don't know. I can
> sympathise more with the latter.

Yeah, foreign people gets afraid on writing English documents. Yet, my
personal perception is that we end by getting more contributions once
the documents get converted, as more people end reading them and ends
by sending us more contributions.

> Kerneldoc is a start - and probably means that a lot of API functions are at
> least slightly documented - but too many APIs are not mentioned in the
> Documentation directory at all. Remember: if you can't describe it, it's
> probably wrong!

Fully agreed.

> I'm not sure what we could do about this, but it would probably have to be
> imposed from the top: no more undocumented APIs. Any new API must come with
> documentation; changes to APIs must include changes to the documentation.

This is a requirement on media: we don't accept for quite a while uAPI
changes without documentation, and we're doing the same for new kAPI
changes (although it is a little more complex to track kAPI changes).

> If you really want to upset people, you could add: anyone who wants to alter
> an already existing undocumented API must supply documentation for the whole
> API (but that could be considered a bit cruel).

Well, we may politely request that ;-) If it doesn't work, we can accept
documentation for just the new stuff.

>
> And anyone who says "But the code is the documentation!" needs to consider
> carefully what happens to their code after it has been trampled, generalised,
> split, combined, renormalised, cthulhuised, janitorised and had parts of it
> migrate.

Yes, on a complex system like the Kernel, having just access to the code
is not enough, as developers don't usually have the time to read the code
for every single function call his code needs to use.

>
> </rant>
>
> And now, after that, I think a fresh cup of tea is called for!
>
> David



Thanks,
Mauro

2019-04-23 20:27:53

by Mauro Carvalho Chehab

[permalink] [raw]
Subject: Re: [PATCH v2 56/79] docs: Documentation/*.txt: rename all ReST files to *.rst

Em Tue, 23 Apr 2019 10:54:15 -0600
Jonathan Corbet <[email protected]> escreveu:

> On Tue, 23 Apr 2019 15:52:26 +0100
> David Howells <[email protected]> wrote:
>
> > There've been some changes that I've particularly objected to, such as
> > removing contents lists from files and replacing them with markup like:
> >
> > .. contents:: :local:
> >
> > This actually impedes use of the file. It should not be necessary to build
> > the docs to get that for ordinary use.
>
> Usability of the text files versus that of the built docs is occasionally
> something that has to be traded off. As a general rule, I want the text
> files to remain useful on their own. There is a lot of value in the
> built docs for a lot of people, but that should not be the only, or even
> the primary, form of access
>
> Tables of contents are certainly a place where that tradeoff makes itself
> felt. Doing them by hand ensures that they are always present, but
> requires that people editing the docs also maintain the TOCS - something
> that experience has shown tends not to happen. That's more of a pain
> than a little bit of markup, and people don't do it. An automatically
> generated TOC, instead, is always correct and is linkable.
>
> Few people complain about the biggest impediment to the readability of
> text files, though: the use of kerneldoc comments. That splits the
> information between the text file and multiple random-seeming locations
> among tends of thousands of source files. Sometimes the solution here is
> to move all of the documentation into the source, but that tends to
> fragment it and make it harder to find; it's certainly not the right
> place for many kinds of docs. In general, it's hard to create a coherent
> story that way.
>
> Suggestions / patches on how to improve things for *all* users of the
> docs are certainly welcome!
>
> I am, incidentally, toying with the idea of trying to put together a
> documentation microconf at the Linux Plumbers Conference this year. If
> anybody out there thinks that's a good idea and would like to
> participate, please let me know.

If you add a microconf to LPC, I'm in.

IMO, we made big advances with documentation, but there's a lot more
to be done. Having a microconf to discuss those things may help us
to bring new ideas about how to keep improving it.

Thanks,
Mauro

2019-04-24 12:59:24

by Mauro Carvalho Chehab

[permalink] [raw]
Subject: Re: [PATCH v2 56/79] docs: Documentation/*.txt: rename all ReST files to *.rst

Em Wed, 24 Apr 2019 14:51:26 +0300
Mike Rapoport <[email protected]> escreveu:

> On Tue, Apr 23, 2019 at 05:26:41PM -0300, Mauro Carvalho Chehab wrote:
> > Em Tue, 23 Apr 2019 10:54:15 -0600
> > Jonathan Corbet <[email protected]> escreveu:
> >
> > > On Tue, 23 Apr 2019 15:52:26 +0100
> > > David Howells <[email protected]> wrote:
> > >
> > > Suggestions / patches on how to improve things for *all* users of the
> > > docs are certainly welcome!
> > >
> > > I am, incidentally, toying with the idea of trying to put together a
> > > documentation microconf at the Linux Plumbers Conference this year. If
> > > anybody out there thinks that's a good idea and would like to
> > > participate, please let me know.
> >
> > If you add a microconf to LPC, I'm in.
>
> +1
>
> > IMO, we made big advances with documentation, but there's a lot more
> > to be done. Having a microconf to discuss those things may help us
> > to bring new ideas about how to keep improving it.
>
> The most difficult part, IMHO, is to convince people to document things ;-)

As David mentioned, maintainers could enforce merging new APIs
only with documentation, with the risk of being unpopular.

Well, maintainers are not among the most loved ones anyway ;-)
<sarcasm/>

My experience enforcing it at media subsystem is that it is not
that hard to have developers writing documentation, once it
becomes a rule, and the maintainers give the example.

The big problem is how to deal with legacy stuff. I had to do that
myself for the DVB subsystem, where the documentation were frozen back
on 2002 days, while lots of new stuff got added (and, worse than that,
with some very obscure ioctls that is used only by a single driver and
some that were used only by OOT drivers).

It take a few years to put it in sync with the code, but, once the
ReST conversion was done, it became easier to do the work.

Thanks,
Mauro

2019-04-24 18:52:57

by Mike Rapoport

[permalink] [raw]
Subject: Re: [PATCH v2 56/79] docs: Documentation/*.txt: rename all ReST files to *.rst

On Tue, Apr 23, 2019 at 05:26:41PM -0300, Mauro Carvalho Chehab wrote:
> Em Tue, 23 Apr 2019 10:54:15 -0600
> Jonathan Corbet <[email protected]> escreveu:
>
> > On Tue, 23 Apr 2019 15:52:26 +0100
> > David Howells <[email protected]> wrote:
> >
> > Suggestions / patches on how to improve things for *all* users of the
> > docs are certainly welcome!
> >
> > I am, incidentally, toying with the idea of trying to put together a
> > documentation microconf at the Linux Plumbers Conference this year. If
> > anybody out there thinks that's a good idea and would like to
> > participate, please let me know.
>
> If you add a microconf to LPC, I'm in.

+1

> IMO, we made big advances with documentation, but there's a lot more
> to be done. Having a microconf to discuss those things may help us
> to bring new ideas about how to keep improving it.

The most difficult part, IMHO, is to convince people to document things ;-)

> Thanks,
> Mauro
>

--
Sincerely yours,
Mike.