Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33;
Date:   Mon, 20 Nov 2023 11:55:26 -0800
From:   Breno Leitao <leitao@debian.org>
To:     Jakub Kicinski <kuba@kernel.org>
Cc:     leit@meta.com, Jonathan Corbet <corbet@lwn.net>,
        netdev@vger.kernel.org, donald.hunter@gmail.com,
        linux-doc@vger.kernel.org, pabeni@redhat.com, edumazet@google.com,
        linux-kernel@vger.kernel.org
Subject: Re: [PATCH v2] Documentation: Document each netlink family
Message-ID: <ZVu5rq1SdloY41nH@gmail.com>
References: <20231113202936.242308-1-leitao@debian.org>
 <87y1ew6n4x.fsf@meer.lwn.net>
 <20231117163939.2de33e83@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20231117163939.2de33e83@kernel.org>
Precedence: bulk

On Fri, Nov 17, 2023 at 04:39:39PM -0800, Jakub Kicinski wrote:
> On Fri, 17 Nov 2023 15:17:02 -0700 Jonathan Corbet wrote:
> > In principle I like this approach better.  There is one problem, though:
> > 
> > - In current kernels, on my machine, "make htmldocs" when nothing has
> >   changed takes about 6s to complete.
> > 
> > - With this patch applied, it takes a little over 5 *minutes*.
> > 
> > Without having delved into it too far, I am guessing that the
> > unconditional recreation of the netlink RST files is causing the rebuild
> > of much of the documentation.  Even so, I don't quite get it.
> > 
> > That, clearly, would need to be fixed before this can go in.
> 
> FWIW on the C code-gen side we avoid touching the files if nothing
> changed both at the Makefile level:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/tree/tools/net/ynl/generated/Makefile#n28

I was able to do something similar and it works quite well. Only
regenerate what is not up-to-date. See below what I am doing. (I needed
to change the python to adapt)

> And the tool itself actually generates to a tempfile and compares
> if the output changed:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=2b7ac0c87d985c92e519995853c52b9649ea4b07

I am not planning to do it, since I would like to trust Make. Let me
know if you think this is important and I can do it also.

--
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 2f35793acd2a..dad6e2ecf082 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -97,7 +97,22 @@ quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
                cp $(if $(patsubst /%,,$(DOCS_CSS)),$(abspath $(srctree)/$(DOCS_CSS)),$(DOCS_CSS)) $(BUILDDIR)/$3/_stati
c/; \
        fi

-htmldocs:
+YNL_TOOL:=$(srctree)/tools/net/ynl/ynl-gen-rst.py
+YNL_YAML_DIR:=$(srctree)/Documentation/netlink/specs
+YNL_RST_DIR:=$(srctree)/Documentation/networking/netlink_spec
+YNL_INDEX:=$(srctree)/Documentation/networking/netlink_spec/index.rst
+
+YNL_YAML_FILES := $(wildcard $(YNL_YAML_DIR)/*.yaml)
+YNL_RST_FILES_TMP := $(patsubst %.yaml,%.rst,$(wildcard $(YNL_YAML_DIR)/*.yaml))
+YNL_RST_FILES := $(patsubst $(YNL_YAML_DIR)%,$(YNL_RST_DIR)%, $(YNL_RST_FILES_TMP))
+
+$(YNL_INDEX): $(YNL_RST_FILES)
+       $(YNL_TOOL) -x # Generate the index
+
+%.rst: $(YNL_YAMLS_FILES)
+       $(YNL_TOOL) -i $(patsubst %.rst,%.yaml, $(@F)) # generate individual rst files
+
+htmldocs: $(YNL_INDEX)
        @$(srctree)/scripts/sphinx-pre-install --version-check
        @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))