Hi Jon,
This series contain some improvements for the building system.
I sent already several of the patches here. They're rebased on the
top of your docs-next tree:
patch 1: gets rid of a warning since version 1.8 (I guess it starts
appearing with 1.8.6);
patches 2 to 4: improve the pre-install script;
patches 5 to 8: improve the script with checks broken doc references;
patch 9: by default, use "-jauto" with Sphinx 1.7 or upper, in order
to speed up the build.
patch 10 changes the recommended Sphinx version to 1.7.9. It keeps
the minimal supported version to 1.3.
Patch 4 contains a good description of the improvements made at
the build system.
If you prefer, you can pull those patches (and the next series I'm
submitting you) from my development git tree:
https://git.linuxtv.org/mchehab/experimental.git/log/?h=fix_doc_links_v2
Regards,
Mauro
-
Mauro Carvalho Chehab (10):
docs: cdomain.py: get rid of a warning since version 1.8
scripts/sphinx-pre-install: make activate hint smarter
scripts/sphinx-pre-install: get rid of RHEL7 explicity check
scripts/sphinx-pre-install: always check if version is compatible with
build
scripts/documentation-file-ref-check: better handle translations
scripts/documentation-file-ref-check: exclude false-positives
scripts/documentation-file-ref-check: improve tools ref handling
scripts/documentation-file-ref-check: teach about .txt -> .yaml
renames
docs: by default, build docs a lot faster with Sphinx >= 1.7
docs: requirements.txt: recommend Sphinx 1.7.9
Documentation/Makefile | 7 +++
Documentation/doc-guide/sphinx.rst | 17 +++---
Documentation/sphinx/cdomain.py | 5 +-
Documentation/sphinx/requirements.txt | 4 +-
scripts/documentation-file-ref-check | 44 ++++++++++++----
scripts/sphinx-pre-install | 75 +++++++++++++++------------
6 files changed, 97 insertions(+), 55 deletions(-)
--
2.21.0
As discussed at the linux-doc ML, while we'll still support
version 1.3, it is time to recommend a more modern version.
So, let's switch the minimal requirements to Sphinx 1.7.9,
as it has the "-jauto" flag, with makes a lot faster when
building documentation.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
Documentation/doc-guide/sphinx.rst | 17 ++++++++---------
Documentation/sphinx/requirements.txt | 4 ++--
2 files changed, 10 insertions(+), 11 deletions(-)
diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index c039224b404e..4ba081f43e98 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -27,8 +27,7 @@ Sphinx Install
==============
The ReST markups currently used by the Documentation/ files are meant to be
-built with ``Sphinx`` version 1.3 or higher. If you desire to build
-PDF output, it is recommended to use version 1.4.6 or higher.
+built with ``Sphinx`` version 1.3 or higher.
There's a script that checks for the Sphinx requirements. Please see
:ref:`sphinx-pre-install` for further details.
@@ -56,13 +55,13 @@ or ``virtualenv``, depending on how your distribution packaged Python 3.
those expressions are written using LaTeX notation. It needs texlive
installed with amdfonts and amsmath in order to evaluate them.
-In summary, if you want to install Sphinx version 1.4.9, you should do::
+In summary, if you want to install Sphinx version 1.7.9, you should do::
- $ virtualenv sphinx_1.4
- $ . sphinx_1.4/bin/activate
- (sphinx_1.4) $ pip install -r Documentation/sphinx/requirements.txt
+ $ virtualenv sphinx_1.7.9
+ $ . sphinx_1.7.9/bin/activate
+ (sphinx_1.7.9) $ pip install -r Documentation/sphinx/requirements.txt
-After running ``. sphinx_1.4/bin/activate``, the prompt will change,
+After running ``. sphinx_1.7.9/bin/activate``, the prompt will change,
in order to indicate that you're using the new environment. If you
open a new shell, you need to rerun this command to enter again at
the virtual environment before building the documentation.
@@ -105,8 +104,8 @@ command line options for your distro::
You should run:
sudo dnf install -y texlive-luatex85
- /usr/bin/virtualenv sphinx_1.4
- . sphinx_1.4/bin/activate
+ /usr/bin/virtualenv sphinx_1.7.9
+ . sphinx_1.7.9/bin/activate
pip install -r Documentation/sphinx/requirements.txt
Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
diff --git a/Documentation/sphinx/requirements.txt b/Documentation/sphinx/requirements.txt
index 742be3e12619..14e29a0ae480 100644
--- a/Documentation/sphinx/requirements.txt
+++ b/Documentation/sphinx/requirements.txt
@@ -1,3 +1,3 @@
-docutils==0.12
-Sphinx==1.4.9
+docutils
+Sphinx==1.7.9
sphinx_rtd_theme
--
2.21.0
Call the script every time a make docs target is selected, on
a simplified check mode.
With this change, the script will set two vars:
$min_version - obtained from `needs_sphinx` var inside
conf.py (currently, '1.3')
$rec_version - obtained from sphinx/requirements.txt.
With those changes, a target like "make htmldocs" will do:
1) If no sphinx-build/sphinx-build3 is found, it will run
the script on normal mode as before, checking for all
system dependencies and providing install hints for the
needed programs and will abort the build;
2) If no sphinx-build/sphinx-build3 is found, but there is
a sphinx_${VER}/bin/activate file, and if
${VER} >= $min_version (string comparation), it will
run in full mode, and will recommend to activate the
virtualenv. If there are multiple virtualenvs, it
will string sort the versions, recommending the
highest version and will abort the build;
3) If Sphinx is detected but has a version lower than
$min_version, it will run in full mode - with will
recommend creating a virtual env using sphinx/requirements.txt,
and will abort the build.
4) If Sphinx is detected and version is lower than
$rec_version, it will run in full mode and will
recommend creating a virtual env using sphinx/requirements.txt.
In this case, it **won't** abort the build.
5) If Sphinx is detected and version is equal or righer than
$rec_version it will return just after detecting the
version ("quick mode"), not checking if are there any
missing dependencies.
Just like before, if one wants to install Sphinx from the
distro, it has to call the script manually and use `--no-virtualenv`
argument to get the hints for his OS:
You should run:
sudo dnf install -y python3-sphinx python3-sphinx_rtd_theme
While here, add a small help for the three optional arguments
for the script.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
Documentation/Makefile | 5 +++++
scripts/sphinx-pre-install | 46 +++++++++++++++++++++++++-------------
2 files changed, 35 insertions(+), 16 deletions(-)
diff --git a/Documentation/Makefile b/Documentation/Makefile
index e889e7cb8511..380e24053d6f 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -70,12 +70,14 @@ quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
$(abspath $(BUILDDIR)/$3/$4)
htmldocs:
+ @./scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))
linkcheckdocs:
@$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,linkcheck,$(var),,$(var)))
latexdocs:
+ @./scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var)))
ifeq ($(HAVE_PDFLATEX),0)
@@ -87,14 +89,17 @@ pdfdocs:
else # HAVE_PDFLATEX
pdfdocs: latexdocs
+ @./scripts/sphinx-pre-install --version-check
$(foreach var,$(SPHINXDIRS), $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" -C $(BUILDDIR)/$(var)/latex || exit;)
endif # HAVE_PDFLATEX
epubdocs:
+ @./scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var)))
xmldocs:
+ @./scripts/sphinx-pre-install --version-check
@+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var)))
endif # HAVE_SPHINX
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index ded3e2ef3f8d..f001fc2fcf12 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -38,6 +38,7 @@ my $min_version;
my $pdf = 1;
my $virtualenv = 1;
+my $version_check = 0;
#
# List of required texlive packages on Fedora and OpenSuse
@@ -277,20 +278,22 @@ sub check_sphinx()
die "$sphinx didn't return its version" if (!$cur_version);
- printf "Sphinx version %s (minimal: %s, recommended >= %s)\n",
- $cur_version, $min_version, $rec_version;
-
if ($cur_version lt $min_version) {
- print "Warning: Sphinx version should be >= $min_version\n\n";
+ printf "ERROR: Sphinx version is %s. It should be >= %s (recommended >= %s)\n",
+ $cur_version, $min_version, $rec_version;;
$need_sphinx = 1;
return;
}
if ($cur_version lt $rec_version) {
+ printf "Sphinx version %s\n", $cur_version;
print "Warning: It is recommended at least Sphinx version $rec_version.\n";
- print " To upgrade, use:\n\n";
$rec_sphinx_upgrade = 1;
+ return;
}
+
+ # On version check mode, just assume Sphinx has all mandatory deps
+ exit (0) if ($version_check);
}
#
@@ -575,14 +578,18 @@ sub check_distros()
sub check_needs()
{
- if ($system_release) {
- print "Detected OS: $system_release.\n";
- } else {
- print "Unknown OS\n";
- }
-
# Check for needed programs/tools
check_sphinx();
+
+ if ($system_release) {
+ print "Detected OS: $system_release.\n\n";
+ } else {
+ print "Unknown OS\n\n";
+ }
+
+ print "To upgrade Sphinx, use:\n\n" if ($rec_sphinx_upgrade);
+
+ # Check for needed programs/tools
check_perl_module("Pod::Usage", 0);
check_program("make", 0);
check_program("gcc", 0);
@@ -601,13 +608,14 @@ sub check_needs()
}
if ($need_sphinx || $rec_sphinx_upgrade) {
my $min_activate = "$ENV{'PWD'}/${virtenv_prefix}${min_version}/bin/activate";
- my @activates = glob "$ENV{'PWD'}/${virtenv_prefix}*/bin/activate";
+ my @activates = glob "$ENV{'PWD'}/${virtenv_prefix}*/bin/activate";
- @activates = sort {$b cmp $a} @activates;
+ @activates = sort {$b cmp $a} @activates;
- if (scalar @activates > 0 && $activates[0] ge $min_activate) {
- printf "\nNeed to activate virtualenv with:\n";
+ if ($need_sphinx && scalar @activates > 0 && $activates[0] ge $min_activate) {
+ printf "\nNeed to activate a compatible Sphinx version on virtualenv with:\n";
printf "\t. $activates[0]\n";
+ exit (1);
} else {
my $rec_activate = "$virtenv_dir/bin/activate";
my $virtualenv = findprog("virtualenv-3");
@@ -646,8 +654,14 @@ while (@ARGV) {
$virtualenv = 0;
} elsif ($arg eq "--no-pdf"){
$pdf = 0;
+ } elsif ($arg eq "--version-check"){
+ $version_check = 1;
} else {
- print "Usage:\n\t$0 <--no-virtualenv> <--no-pdf>\n\n";
+ print "Usage:\n\t$0 <--no-virtualenv> <--no-pdf> <--version-check>\n\n";
+ print "Where:\n";
+ print "\t--no-virtualenv\t- Recommend installing Sphinx instead of using a virtualenv\n";
+ print "\t--version-check\t- if version is compatible, don't check for missing dependencies\n";
+ print "\t--no-pdf\t- don't check for dependencies required to build PDF docs\n\n";
exit -1;
}
}
--
2.21.0
At DT, files are being renamed to jason. Teach the script how to
handle such renames when used in fix mode.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/documentation-file-ref-check | 19 ++++++++++++++-----
1 file changed, 14 insertions(+), 5 deletions(-)
diff --git a/scripts/documentation-file-ref-check b/scripts/documentation-file-ref-check
index 5d775ca7469b..ff16db269079 100755
--- a/scripts/documentation-file-ref-check
+++ b/scripts/documentation-file-ref-check
@@ -165,13 +165,22 @@ foreach my $ref (keys %broken_ref) {
# usual reason for breakage: DT file moved around
if ($ref =~ /devicetree/) {
- my $search = $new;
- $search =~ s,^.*/,,;
- $f = qx(find Documentation/devicetree/ -iname "*$search*") if ($search);
+ # usual reason for breakage: DT file renamed to .yaml
if (!$f) {
- # Manufacturer name may have changed
- $search =~ s/^.*,//;
+ my $new_ref = $ref;
+ $new_ref =~ s/\.txt$/.yaml/;
+ $f=$new_ref if (-f $new_ref);
+ }
+
+ if (!$f) {
+ my $search = $new;
+ $search =~ s,^.*/,,;
$f = qx(find Documentation/devicetree/ -iname "*$search*") if ($search);
+ if (!$f) {
+ # Manufacturer name may have changed
+ $search =~ s/^.*,//;
+ $f = qx(find Documentation/devicetree/ -iname "*$search*") if ($search);
+ }
}
}
--
2.21.0
RHEL8 was already launched. This test won't get it, and will
do the wrong thing. Ok, we could fix it, but now we check
Sphinx version to ensure that it matches the minimal (1.3),
so there's no need for an explicit check there.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/sphinx-pre-install | 13 -------------
1 file changed, 13 deletions(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 11239eb29695..ded3e2ef3f8d 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -581,19 +581,6 @@ sub check_needs()
print "Unknown OS\n";
}
- # RHEL 7.x and clones have Sphinx version 1.1.x and incomplete texlive
- if (($system_release =~ /Red Hat Enterprise Linux/) ||
- ($system_release =~ /CentOS/) ||
- ($system_release =~ /Scientific Linux/) ||
- ($system_release =~ /Oracle Linux Server/)) {
- $virtualenv = 1;
- $pdf = 0;
-
- printf("NOTE: On this distro, Sphinx and TexLive shipped versions are incompatible\n");
- printf("with doc build. So, use Sphinx via a Python virtual environment.\n\n");
- printf("This script can't install a TexLive version that would provide PDF.\n");
- }
-
# Check for needed programs/tools
check_sphinx();
check_perl_module("Pod::Usage", 0);
--
2.21.0
On Wed, 29 May 2019 20:09:22 -0300
Mauro Carvalho Chehab <[email protected]> wrote:
> This series contain some improvements for the building system.
>
> I sent already several of the patches here. They're rebased on the
> top of your docs-next tree:
The set is now applied...
> patch 1: gets rid of a warning since version 1.8 (I guess it starts
> appearing with 1.8.6);
This one I'd already picked up before.
> patches 2 to 4: improve the pre-install script;
>
> patches 5 to 8: improve the script with checks broken doc references;
>
> patch 9: by default, use "-jauto" with Sphinx 1.7 or upper, in order
> to speed up the build.
I put in the tweak we discussed here.
> patch 10 changes the recommended Sphinx version to 1.7.9. It keeps
> the minimal supported version to 1.3.
>
> Patch 4 contains a good description of the improvements made at
> the build system.
Thanks,
jon