The first three patches here address some issues with the script which
detects the requirements for running Sphinx, making it a smarter and
running it all the times, in order to allow recommending version
updates.
The 4th patch adds "-jauto" by default, if Sphinx version supports it
(e. g. version 1.7 or upper).
The final patch changes the recommendation to use at least Sphinx
version 1.7.9, as discussed at linux-doc ML.
Mauro Carvalho Chehab (5):
scripts/sphinx-pre-install: make activate hint smarter
scripts/sphinx-pre-install: get rid of RHEL7 explicity check
scripts/sphinx-pre-install: always check if version is compatible with
build
docs: by default, build docs a lot faster with Sphinx >= 1.7
docs: requirements.txt: recommend Sphinx 1.7.9
Documentation/Makefile | 7 +++
Documentation/doc-guide/sphinx.rst | 17 +++---
Documentation/sphinx/requirements.txt | 4 +-
scripts/sphinx-pre-install | 75 +++++++++++++++------------
4 files changed, 58 insertions(+), 45 deletions(-)
--
2.21.0
As discussed at the linux-doc ML, while we'll still support
version 1.3, it is time to recommend a more modern version.
So, let's switch the minimal requirements to Sphinx 1.7.9,
as it has the "-jauto" flag, with makes a lot faster when
building documentation.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
Documentation/doc-guide/sphinx.rst | 17 ++++++++---------
Documentation/sphinx/requirements.txt | 4 ++--
2 files changed, 10 insertions(+), 11 deletions(-)
diff --git a/Documentation/doc-guide/sphinx.rst b/Documentation/doc-guide/sphinx.rst
index c039224b404e..4ba081f43e98 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -27,8 +27,7 @@ Sphinx Install
==============
The ReST markups currently used by the Documentation/ files are meant to be
-built with ``Sphinx`` version 1.3 or higher. If you desire to build
-PDF output, it is recommended to use version 1.4.6 or higher.
+built with ``Sphinx`` version 1.3 or higher.
There's a script that checks for the Sphinx requirements. Please see
:ref:`sphinx-pre-install` for further details.
@@ -56,13 +55,13 @@ or ``virtualenv``, depending on how your distribution packaged Python 3.
those expressions are written using LaTeX notation. It needs texlive
installed with amdfonts and amsmath in order to evaluate them.
-In summary, if you want to install Sphinx version 1.4.9, you should do::
+In summary, if you want to install Sphinx version 1.7.9, you should do::
- $ virtualenv sphinx_1.4
- $ . sphinx_1.4/bin/activate
- (sphinx_1.4) $ pip install -r Documentation/sphinx/requirements.txt
+ $ virtualenv sphinx_1.7.9
+ $ . sphinx_1.7.9/bin/activate
+ (sphinx_1.7.9) $ pip install -r Documentation/sphinx/requirements.txt
-After running ``. sphinx_1.4/bin/activate``, the prompt will change,
+After running ``. sphinx_1.7.9/bin/activate``, the prompt will change,
in order to indicate that you're using the new environment. If you
open a new shell, you need to rerun this command to enter again at
the virtual environment before building the documentation.
@@ -105,8 +104,8 @@ command line options for your distro::
You should run:
sudo dnf install -y texlive-luatex85
- /usr/bin/virtualenv sphinx_1.4
- . sphinx_1.4/bin/activate
+ /usr/bin/virtualenv sphinx_1.7.9
+ . sphinx_1.7.9/bin/activate
pip install -r Documentation/sphinx/requirements.txt
Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468.
diff --git a/Documentation/sphinx/requirements.txt b/Documentation/sphinx/requirements.txt
index 742be3e12619..14e29a0ae480 100644
--- a/Documentation/sphinx/requirements.txt
+++ b/Documentation/sphinx/requirements.txt
@@ -1,3 +1,3 @@
-docutils==0.12
-Sphinx==1.4.9
+docutils
+Sphinx==1.7.9
sphinx_rtd_theme
--
2.21.0
RHEL8 was already launched. This test won't get it, and will
do the wrong thing. Ok, we could fix it, but now we check
Sphinx version to ensure that it matches the minimal (1.3),
so there's no need for an explicit check there.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/sphinx-pre-install | 13 -------------
1 file changed, 13 deletions(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 11239eb29695..ded3e2ef3f8d 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -581,19 +581,6 @@ sub check_needs()
print "Unknown OS\n";
}
- # RHEL 7.x and clones have Sphinx version 1.1.x and incomplete texlive
- if (($system_release =~ /Red Hat Enterprise Linux/) ||
- ($system_release =~ /CentOS/) ||
- ($system_release =~ /Scientific Linux/) ||
- ($system_release =~ /Oracle Linux Server/)) {
- $virtualenv = 1;
- $pdf = 0;
-
- printf("NOTE: On this distro, Sphinx and TexLive shipped versions are incompatible\n");
- printf("with doc build. So, use Sphinx via a Python virtual environment.\n\n");
- printf("This script can't install a TexLive version that would provide PDF.\n");
- }
-
# Check for needed programs/tools
check_sphinx();
check_perl_module("Pod::Usage", 0);
--
2.21.0
Since Sphinx version 1.7, it is possible to use "-jauto" in
order to speedup documentation builds. On older versions,
while -j was already supported, one would need to set the
number of threads manually.
So, if SPHINXOPTS is not provided, add -jauto, in order to
speed up the build. That makes it *a lot* times faster than
without -j.
If one really wants to slow things down, it can just use:
make SPHINXOPTS=-j1 htmldocs
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
Documentation/Makefile | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 380e24053d6f..794233d05789 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -28,6 +28,8 @@ ifeq ($(HAVE_SPHINX),0)
else # HAVE_SPHINX
+SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
+
# User-friendly check for pdflatex and latexmk
HAVE_PDFLATEX := $(shell if which $(PDFLATEX) >/dev/null 2>&1; then echo 1; else echo 0; fi)
HAVE_LATEXMK := $(shell if which latexmk >/dev/null 2>&1; then echo 1; else echo 0; fi)
--
2.21.0
It is possible that multiple Sphinx virtualenvs are installed
on a given kernel tree. Change the logic to get the latest
version of those, as this is probably what the user wants.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/sphinx-pre-install | 22 ++++++++++++++--------
1 file changed, 14 insertions(+), 8 deletions(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 8c2d1bcf2e02..11239eb29695 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -1,7 +1,7 @@
#!/usr/bin/perl
use strict;
-# Copyright (c) 2017 Mauro Carvalho Chehab <[email protected]>
+# Copyright (c) 2017-2019 Mauro Carvalho Chehab <[email protected]>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
@@ -15,6 +15,7 @@ use strict;
my $conf = "Documentation/conf.py";
my $requirement_file = "Documentation/sphinx/requirements.txt";
+my $virtenv_prefix = "sphinx_";
#
# Static vars
@@ -28,7 +29,8 @@ my $need_symlink = 0;
my $need_sphinx = 0;
my $rec_sphinx_upgrade = 0;
my $install = "";
-my $virtenv_dir = "sphinx_";
+my $virtenv_dir = "";
+my $min_version;
#
# Command line arguments
@@ -229,7 +231,6 @@ sub get_sphinx_fname()
sub check_sphinx()
{
- my $min_version;
my $rec_version;
my $cur_version;
@@ -255,7 +256,7 @@ sub check_sphinx()
die "Can't get recommended sphinx version from $requirement_file" if (!$min_version);
- $virtenv_dir .= $rec_version;
+ $virtenv_dir = $virtenv_prefix . $rec_version;
my $sphinx = get_sphinx_fname();
return if ($sphinx eq "");
@@ -612,18 +613,23 @@ sub check_needs()
which("sphinx-build-3");
}
if ($need_sphinx || $rec_sphinx_upgrade) {
- my $activate = "$virtenv_dir/bin/activate";
- if (-e "$ENV{'PWD'}/$activate") {
+ my $min_activate = "$ENV{'PWD'}/${virtenv_prefix}${min_version}/bin/activate";
+ my @activates = glob "$ENV{'PWD'}/${virtenv_prefix}*/bin/activate";
+
+ @activates = sort {$b cmp $a} @activates;
+
+ if (scalar @activates > 0 && $activates[0] ge $min_activate) {
printf "\nNeed to activate virtualenv with:\n";
- printf "\t. $activate\n";
+ printf "\t. $activates[0]\n";
} else {
+ my $rec_activate = "$virtenv_dir/bin/activate";
my $virtualenv = findprog("virtualenv-3");
$virtualenv = findprog("virtualenv-3.5") if (!$virtualenv);
$virtualenv = findprog("virtualenv") if (!$virtualenv);
$virtualenv = "virtualenv" if (!$virtualenv);
printf "\t$virtualenv $virtenv_dir\n";
- printf "\t. $activate\n";
+ printf "\t. $rec_activate\n";
printf "\tpip install -r $requirement_file\n";
$need++ if (!$rec_sphinx_upgrade);
--
2.21.0
On Mon, 27 May 2019 08:07:40 -0300
Mauro Carvalho Chehab <[email protected]> wrote:
> Since Sphinx version 1.7, it is possible to use "-jauto" in
> order to speedup documentation builds. On older versions,
> while -j was already supported, one would need to set the
> number of threads manually.
>
> So, if SPHINXOPTS is not provided, add -jauto, in order to
> speed up the build. That makes it *a lot* times faster than
> without -j.
>
> If one really wants to slow things down, it can just use:
>
> make SPHINXOPTS=-j1 htmldocs
>
> Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> ---
> Documentation/Makefile | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/Documentation/Makefile b/Documentation/Makefile
> index 380e24053d6f..794233d05789 100644
> --- a/Documentation/Makefile
> +++ b/Documentation/Makefile
> @@ -28,6 +28,8 @@ ifeq ($(HAVE_SPHINX),0)
>
> else # HAVE_SPHINX
>
> +SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
> +
So this totally fails to work for me with any version of sphinx, and I'm
not enough of a Perl person to figure it out. Sometimes I'll see the
sphinx-build output, i.e.:
sphinx-build 1.8.4
and sometimes (like with 2.0) I don't, but I never get -jauto regardless.
Not sure what's going on here?
Thanks,
jon
Em Wed, 29 May 2019 17:02:02 -0600
Jonathan Corbet <[email protected]> escreveu:
> On Mon, 27 May 2019 08:07:40 -0300
> Mauro Carvalho Chehab <[email protected]> wrote:
>
> > Since Sphinx version 1.7, it is possible to use "-jauto" in
> > order to speedup documentation builds. On older versions,
> > while -j was already supported, one would need to set the
> > number of threads manually.
> >
> > So, if SPHINXOPTS is not provided, add -jauto, in order to
> > speed up the build. That makes it *a lot* times faster than
> > without -j.
> >
> > If one really wants to slow things down, it can just use:
> >
> > make SPHINXOPTS=-j1 htmldocs
> >
> > Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> > ---
> > Documentation/Makefile | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/Documentation/Makefile b/Documentation/Makefile
> > index 380e24053d6f..794233d05789 100644
> > --- a/Documentation/Makefile
> > +++ b/Documentation/Makefile
> > @@ -28,6 +28,8 @@ ifeq ($(HAVE_SPHINX),0)
> >
> > else # HAVE_SPHINX
> >
> > +SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
> > +
>
> So this totally fails to work for me with any version of sphinx, and I'm
> not enough of a Perl person to figure it out. Sometimes I'll see the
> sphinx-build output, i.e.:
>
> sphinx-build 1.8.4
>
> and sometimes (like with 2.0) I don't, but I never get -jauto regardless.
Hmm... with 2.0.0 --version prints the version.
$ sphinx-build --version
sphinx-build 2.0.0
Afaikt, only too old versions will output it with a different format:
$ sphinx-build --version
Sphinx (sphinx-build) 1.2
> Not sure what's going on here?
Do you have SPHINXOPTS already set on your environment? If so, Makefile
will not override the existing environment.
Here, if I call it by hand (replacing $$1 by $1), it does the right
thing. For example:
1.8.4:
$ sphinx-build --version
sphinx-build 1.8.4
$ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
-jauto
2.0.0:
$ sphinx-build --version
sphinx-build 2.0.0
(sphinx_2.0) $ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
-jauto
Thanks,
Mauro
On Wed, 29 May 2019 20:20:05 -0300
Mauro Carvalho Chehab <[email protected]> wrote:
> > So this totally fails to work for me with any version of sphinx, and I'm
> > not enough of a Perl person to figure it out. Sometimes I'll see the
> > sphinx-build output, i.e.:
> >
> > sphinx-build 1.8.4
> >
> > and sometimes (like with 2.0) I don't, but I never get -jauto regardless.
>
> Hmm... with 2.0.0 --version prints the version.
>
> $ sphinx-build --version
> sphinx-build 2.0.0
Yup. The point is that I see the sphinx-build output *in the docs-build
output", not when I run it standalone (where it does the expected thing).
> > Not sure what's going on here?
>
> Do you have SPHINXOPTS already set on your environment? If so, Makefile
> will not override the existing environment.
Yeah, I had it set to -j1 because I want to wait as long as possible for my
docs builds :)
No, I didn't have it set separately, made a point of that.
> Here, if I call it by hand (replacing $$1 by $1), it does the right
> thing. For example:
>
> 1.8.4:
>
> $ sphinx-build --version
> sphinx-build 1.8.4
> $ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
> -jauto
$ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
sphinx-build 1.8.4
$
It works properly with 2.0.1 - but only on the command line; I still don't
get the right behavior in a docs build.
Most weird.
This is an Fedora 30 system, FWIW.
jon
Em Wed, 29 May 2019 17:47:16 -0600
Jonathan Corbet <[email protected]> escreveu:
> On Wed, 29 May 2019 20:20:05 -0300
> Mauro Carvalho Chehab <[email protected]> wrote:
>
> > > So this totally fails to work for me with any version of sphinx, and I'm
> > > not enough of a Perl person to figure it out. Sometimes I'll see the
> > > sphinx-build output, i.e.:
> > >
> > > sphinx-build 1.8.4
> > >
> > > and sometimes (like with 2.0) I don't, but I never get -jauto regardless.
> >
> > Hmm... with 2.0.0 --version prints the version.
> >
> > $ sphinx-build --version
> > sphinx-build 2.0.0
>
> Yup. The point is that I see the sphinx-build output *in the docs-build
> output", not when I run it standalone (where it does the expected thing).
Weird... could some versions of Sphinx be redirecting the output of
--version to stderr instead of stdout?
If so, something like:
perl -e 'open IN,"sphinx-build --version 2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
would make it print "-jauto" with those other versions you're trying.
>
> > > Not sure what's going on here?
> >
> > Do you have SPHINXOPTS already set on your environment? If so, Makefile
> > will not override the existing environment.
>
> Yeah, I had it set to -j1 because I want to wait as long as possible for my
> docs builds :)
>
> No, I didn't have it set separately, made a point of that.
>
> > Here, if I call it by hand (replacing $$1 by $1), it does the right
> > thing. For example:
> >
> > 1.8.4:
> >
> > $ sphinx-build --version
> > sphinx-build 1.8.4
> > $ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
> > -jauto
>
> $ perl -e 'open IN,"sphinx-build --version |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
> sphinx-build 1.8.4
> $
>
> It works properly with 2.0.1 - but only on the command line; I still don't
> get the right behavior in a docs build.
>
> Most weird.
>
> This is an Fedora 30 system, FWIW.
Yeah, really weird. Here I'm using Fedora 30 too:
python3-sphinx_rtd_theme-0.4.3-1.fc30.noarch
python3-sphinx-1.8.4-1.fc30.noarch
It works with both installed version and pip3 virtualenvs.
I didn't try the python2 versions, though.
>
> jon
Thanks,
Mauro
On Wed, 29 May 2019 22:53:05 -0300
Mauro Carvalho Chehab <[email protected]> wrote:
> > Yup. The point is that I see the sphinx-build output *in the docs-build
> > output", not when I run it standalone (where it does the expected thing).
>
> Weird... could some versions of Sphinx be redirecting the output of
> --version to stderr instead of stdout?
>
> If so, something like:
>
> perl -e 'open IN,"sphinx-build --version 2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
>
> would make it print "-jauto" with those other versions you're trying.
That does improve the behavior from the command line; it seems that
sphinx-build is indeed writing to stderr. BUT that still doesn't fix the
docs build! To get the option to take effect, I also have to explicitly
export SPHINXOPTS. So the winning combination is:
export SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version
2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
I don't have any weird version of make, so I'm not sure why you see
different results than I do here.
I can apply those tweaks to your patch if it's OK with you.
> I didn't try the python2 versions, though.
Interestingly, I would appear to have both versions installed, with
python2 winning in $PATH.
jon
Em Thu, 30 May 2019 08:54:04 -0600
Jonathan Corbet <[email protected]> escreveu:
> On Wed, 29 May 2019 22:53:05 -0300
> Mauro Carvalho Chehab <[email protected]> wrote:
>
> > > Yup. The point is that I see the sphinx-build output *in the docs-build
> > > output", not when I run it standalone (where it does the expected thing).
> >
> > Weird... could some versions of Sphinx be redirecting the output of
> > --version to stderr instead of stdout?
> >
> > If so, something like:
> >
> > perl -e 'open IN,"sphinx-build --version 2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto\n" if ($1 >= "1.7") } ;} close IN'
> >
> > would make it print "-jauto" with those other versions you're trying.
>
> That does improve the behavior from the command line; it seems that
> sphinx-build is indeed writing to stderr. BUT that still doesn't fix the
> docs build! To get the option to take effect, I also have to explicitly
> export SPHINXOPTS. So the winning combination is:
>
> export SPHINXOPTS = $(shell perl -e 'open IN,"sphinx-build --version
> 2>&1 |"; while (<IN>) { if (m/([\d\.]+)/) { print "-jauto" if ($$1 >= "1.7") } ;} close IN')
>
> I don't have any weird version of make, so I'm not sure why you see
> different results than I do here.
>
> I can apply those tweaks to your patch if it's OK with you.
Yeah, sure! With those changes it work fine here too.
So, feel free to apply the changes.
>
> > I didn't try the python2 versions, though.
>
> Interestingly, I would appear to have both versions installed, with
> python2 winning in $PATH.
It sounds that Fedora 30 is conservative with regards to python :-)
The Sphinx version detection script takes it into account,
suggesting pip3 instead of pip - or when called like:
$ ./scripts/sphinx-pre-install --no-virtualenv
Detected OS: Fedora release 30 (Thirty).
ERROR: please install "python-sphinx", otherwise, build won't work.
Warning: better to also install "sphinx_rtd_theme".
You should run:
sudo dnf install -y python3-sphinx python3-sphinx_rtd_theme
It seeks for the python3 packages on Fedora.
Regards,
Mauro