2020-02-21 22:16:13

by Bird, Tim

[permalink] [raw]
Subject: RFC: Fix for sphinx setup message

(Resend: Sorry for the dup. I forgot to include the maintainers, and I had the LKML
address wrong.)

I was trying to set up my machine to do some documentation work,
and I had some problems with the sphinx install. I figured out how to work
around the issue, but I have a question about how to add the information
to scripts/sphinx-pre-install (or whether it should go somewhere else).

Detailed messages below, but the TLl;DR is that I got the message:
-------
You should run:

sudo apt-get install dvipng fonts-noto-cjk latexmk librsvg2-bin texlive-xetex
/usr/bin/virtualenv sphinx_1.7.9
. sphinx_1.7.9/bin/activate
pip install -r ./Documentation/sphinx/requirements.txt
...
------

The pip install step didn't work, and I found that I needed to have everything
based on python3 instead. When I replaced:
/usr/bin/virtualenv sphinx_1.7.9
with
/usr/bin/virtualenv -p python3 sphinx_1.7.9
everything worked.

This message is coming from scripts/sphinx-pre-install (I believe on line 708).

Should I go ahead and submit a patch to add '-p python3' to that line?

Are there any downsides to enforcing that the virtualenv used for the
documentation build use python3 only?

Thanks,
-- Tim

Gory details:
I'm running on a machine with Ubuntu 16.04, and I have both python2 and
python3 installed (with /usr/bin/python linked to python2.7).

When I tried to do:
$ make htmldocs
I got the following messages:
...
You should run:

sudo apt-get install dvipng fonts-noto-cjk latexmk librsvg2-bin texlive-xetex

/usr/bin/virtualenv sphinx_1.7.9
. sphinx_1.7.9/bin/activate
pip install -r ./Documentation/sphinx/requirements.txt
If you want to exit the virtualenv, you can use:
deactivate
---
Following these instructions, at the 'pip install' step, I got:
$ pip install -r ./Documentation/sphinx/requirements.txt
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Collecting docutils
Downloading docutils-0.16-py2.py3-none-any.whl (548 kB)
|?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^| 548 kB 2.2 MB/s
Collecting Sphinx==1.7.9
Downloading Sphinx-1.7.9-py2.py3-none-any.whl (1.9 MB)
|?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^| 1.9 MB 30.7 MB/s
Collecting sphinx_rtd_theme
Downloading sphinx_rtd_theme-0.4.3-py2.py3-none-any.whl (6.4 MB)
|?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^| 6.4 MB 25.6 MB/s
Collecting alabaster<0.8,>=0.7
Downloading alabaster-0.7.12-py2.py3-none-any.whl (14 kB)
Collecting Pygments>=2.0
Downloading Pygments-2.5.2-py2.py3-none-any.whl (896 kB)
|?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^| 896 kB 35.1 MB/s
Collecting Jinja2>=2.3
Downloading Jinja2-2.11.1-py2.py3-none-any.whl (126 kB)
|?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^| 126 kB 26.2 MB/s
Collecting requests>=2.0.0
Downloading requests-2.23.0-py2.py3-none-any.whl (58 kB)
|?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^| 58 kB 774 kB/s
Collecting packaging
Downloading packaging-20.1-py2.py3-none-any.whl (36 kB)
Collecting snowballstemmer>=1.1
Downloading snowballstemmer-2.0.0-py2.py3-none-any.whl (97 kB)
|?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^?-^| 97 kB 1.1 MB/s
Collecting sphinxcontrib-websupport
Downloading sphinxcontrib_websupport-1.1.2-py2.py3-none-any.whl (39 kB)
Requirement already satisfied: setuptools in ./sphinx_1.7.9/lib/python2.7/site-packages (from Sphinx==1.7.9->-r ./Documentation/sphinx/requirements.txt (line 2)) (45.0.0)
ERROR: Package 'setuptools' requires a different Python: 2.7.12 not in '>=3.5'


2020-02-22 07:17:05

by Mauro Carvalho Chehab

[permalink] [raw]
Subject: Re: RFC: Fix for sphinx setup message

Hi Tim,

Em Fri, 21 Feb 2020 22:15:36 +0000
"Bird, Tim" <[email protected]> escreveu:

> (Resend: Sorry for the dup. I forgot to include the maintainers, and I had the LKML
> address wrong.)
>
> I was trying to set up my machine to do some documentation work,
> and I had some problems with the sphinx install. I figured out how to work
> around the issue, but I have a question about how to add the information
> to scripts/sphinx-pre-install (or whether it should go somewhere else).
>
> Detailed messages below, but the TLl;DR is that I got the message:
> -------
> You should run:
>
> sudo apt-get install dvipng fonts-noto-cjk latexmk librsvg2-bin texlive-xetex
> /usr/bin/virtualenv sphinx_1.7.9
> . sphinx_1.7.9/bin/activate
> pip install -r ./Documentation/sphinx/requirements.txt
> ...
> ------
>
> The pip install step didn't work, and I found that I needed to have everything
> based on python3 instead. When I replaced:
> /usr/bin/virtualenv sphinx_1.7.9
> with
> /usr/bin/virtualenv -p python3 sphinx_1.7.9
> everything worked.
>
> This message is coming from scripts/sphinx-pre-install (I believe on line 708).
>
> Should I go ahead and submit a patch to add '-p python3' to that line?
>
> Are there any downsides to enforcing that the virtualenv used for the
> documentation build use python3 only?

Actually, the script tries to detect if python3 is installed. Currently, it
does it by seeking for a python3 variant of virtualenv. If it finds, it
changes the recommendation accordingly. The actual code with does that is
this one:

my $virtualenv = findprog("virtualenv-3");
$virtualenv = findprog("virtualenv-3.5") if (!$virtualenv);
$virtualenv = findprog("virtualenv") if (!$virtualenv);
$virtualenv = "virtualenv" if (!$virtualenv);

This works fine on older Fedora distros (and probably CentOS/RHEL), where
there is a python3 variant of virtualenv. On Ubuntu (and Fedora 31), it
will just use virtualenv.

So, perhaps if we add something like this (untested):

my $python = findprog("python3");

if ($python)
$virtualenv = "$virtualenv -p $python";

it would make the trick. Please notice, however, that this could cause
troubles with some distros that might have a version of virtualenv that
won't work with the above. So, perhaps we should add something like the
above inside give_debian_hints(), and either ensure that other Debian and
Ubuntu LTS versions will work with such change, or add some checks for the
Ubuntu/Debian versions where we know this works.

Note: the version of the distribution (and its name) is already stored
at the global var $system_release.

Cheers,
Mauro

2020-02-22 07:25:05

by Mauro Carvalho Chehab

[permalink] [raw]
Subject: Re: RFC: Fix for sphinx setup message

Em Sat, 22 Feb 2020 08:16:44 +0100
Mauro Carvalho Chehab <[email protected]> escreveu:

> Hi Tim,
>
> Em Fri, 21 Feb 2020 22:15:36 +0000
> "Bird, Tim" <[email protected]> escreveu:
>
> > (Resend: Sorry for the dup. I forgot to include the maintainers, and I had the LKML
> > address wrong.)
> >
> > I was trying to set up my machine to do some documentation work,
> > and I had some problems with the sphinx install. I figured out how to work
> > around the issue, but I have a question about how to add the information
> > to scripts/sphinx-pre-install (or whether it should go somewhere else).
> >
> > Detailed messages below, but the TLl;DR is that I got the message:
> > -------
> > You should run:
> >
> > sudo apt-get install dvipng fonts-noto-cjk latexmk librsvg2-bin texlive-xetex
> > /usr/bin/virtualenv sphinx_1.7.9
> > . sphinx_1.7.9/bin/activate
> > pip install -r ./Documentation/sphinx/requirements.txt
> > ...
> > ------
> >
> > The pip install step didn't work, and I found that I needed to have everything
> > based on python3 instead. When I replaced:
> > /usr/bin/virtualenv sphinx_1.7.9
> > with
> > /usr/bin/virtualenv -p python3 sphinx_1.7.9
> > everything worked.
> >
> > This message is coming from scripts/sphinx-pre-install (I believe on line 708).
> >
> > Should I go ahead and submit a patch to add '-p python3' to that line?
> >
> > Are there any downsides to enforcing that the virtualenv used for the
> > documentation build use python3 only?
>
> Actually, the script tries to detect if python3 is installed. Currently, it
> does it by seeking for a python3 variant of virtualenv. If it finds, it
> changes the recommendation accordingly. The actual code with does that is
> this one:
>
> my $virtualenv = findprog("virtualenv-3");
> $virtualenv = findprog("virtualenv-3.5") if (!$virtualenv);
> $virtualenv = findprog("virtualenv") if (!$virtualenv);
> $virtualenv = "virtualenv" if (!$virtualenv);
>
> This works fine on older Fedora distros (and probably CentOS/RHEL), where
> there is a python3 variant of virtualenv. On Ubuntu (and Fedora 31), it
> will just use virtualenv.
>
> So, perhaps if we add something like this (untested):
>
> my $python = findprog("python3");
>
> if ($python)
> $virtualenv = "$virtualenv -p $python";
>
> it would make the trick. Please notice, however, that this could cause
> troubles with some distros that might have a version of virtualenv that
> won't work with the above. So, perhaps we should add something like the
> above inside give_debian_hints(), and either ensure that other Debian and
> Ubuntu LTS versions will work with such change, or add some checks for the
> Ubuntu/Debian versions where we know this works.

Indeed it seems that, with some versions of python, virtualenv -p python3
don't work:

https://stackoverflow.com/questions/23842713/using-python-3-in-virtualenv

This could well be something already solved on most distros, but I think it
would be safer if we only add "-p python3" if it is Ubuntu 16.04 or upper
(and doing a similar test for Debian).

>
> Note: the version of the distribution (and its name) is already stored
> at the global var $system_release.
>
> Cheers,
> Mauro




Cheers,
Mauro

2020-02-24 19:52:20

by Bird, Tim

[permalink] [raw]
Subject: RE: RFC: Fix for sphinx setup message



> -----Original Message-----
> From: Mauro Carvalho Chehab <[email protected]>
>
> Em Sat, 22 Feb 2020 08:16:44 +0100
> Mauro Carvalho Chehab <[email protected]> escreveu:
>
> > Hi Tim,
> >
> > Em Fri, 21 Feb 2020 22:15:36 +0000
> > "Bird, Tim" <[email protected]> escreveu:
> >
> > > (Resend: Sorry for the dup. I forgot to include the maintainers, and I had the LKML
> > > address wrong.)
> > >
> > > I was trying to set up my machine to do some documentation work,
> > > and I had some problems with the sphinx install. I figured out how to work
> > > around the issue, but I have a question about how to add the information
> > > to scripts/sphinx-pre-install (or whether it should go somewhere else).
> > >
> > > Detailed messages below, but the TLl;DR is that I got the message:
> > > -------
> > > You should run:
> > >
> > > sudo apt-get install dvipng fonts-noto-cjk latexmk librsvg2-bin texlive-xetex
> > > /usr/bin/virtualenv sphinx_1.7.9
> > > . sphinx_1.7.9/bin/activate
> > > pip install -r ./Documentation/sphinx/requirements.txt
> > > ...
> > > ------
> > >
> > > The pip install step didn't work, and I found that I needed to have everything
> > > based on python3 instead. When I replaced:
> > > /usr/bin/virtualenv sphinx_1.7.9
> > > with
> > > /usr/bin/virtualenv -p python3 sphinx_1.7.9
> > > everything worked.
> > >
> > > This message is coming from scripts/sphinx-pre-install (I believe on line 708).
> > >
> > > Should I go ahead and submit a patch to add '-p python3' to that line?
> > >
> > > Are there any downsides to enforcing that the virtualenv used for the
> > > documentation build use python3 only?
> >
> > Actually, the script tries to detect if python3 is installed. Currently, it
> > does it by seeking for a python3 variant of virtualenv. If it finds, it
> > changes the recommendation accordingly. The actual code with does that is
> > this one:
> >
> > my $virtualenv = findprog("virtualenv-3");
> > $virtualenv = findprog("virtualenv-3.5") if (!$virtualenv);
> > $virtualenv = findprog("virtualenv") if (!$virtualenv);
> > $virtualenv = "virtualenv" if (!$virtualenv);
> >
> > This works fine on older Fedora distros (and probably CentOS/RHEL), where
> > there is a python3 variant of virtualenv. On Ubuntu (and Fedora 31), it
> > will just use virtualenv.
> >
> > So, perhaps if we add something like this (untested):
> >
> > my $python = findprog("python3");
> >
> > if ($python)
> > $virtualenv = "$virtualenv -p $python";
> >
> > it would make the trick. Please notice, however, that this could cause
> > troubles with some distros that might have a version of virtualenv that
> > won't work with the above. So, perhaps we should add something like the
> > above inside give_debian_hints(), and either ensure that other Debian and
> > Ubuntu LTS versions will work with such change, or add some checks for the
> > Ubuntu/Debian versions where we know this works.
>
> Indeed it seems that, with some versions of python, virtualenv -p python3
> don't work:
>
> https://stackoverflow.com/questions/23842713/using-python-3-in-virtualenv
>
> This could well be something already solved on most distros, but I think it
> would be safer if we only add "-p python3" if it is Ubuntu 16.04 or upper
> (and doing a similar test for Debian).

OK - see the patch I will send shortly for what I came up with.
-- Tim

> >
> > Note: the version of the distribution (and its name) is already stored
> > at the global var $system_release.
> >
> > Cheers,
> > Mauro