2017-06-19 11:24:20

by Mauro Carvalho Chehab

[permalink] [raw]
Subject: [PATCH] changes.rst: explain the usage of virtual environment

As the Sphinx build seems very fragile, specially for
PDF output, add a notice about how to use it on a virtual
environment.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---

Documentation/process/changes.rst | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)

diff --git a/Documentation/process/changes.rst b/Documentation/process/changes.rst
index 3aed751e0cb5..8a51f37bce4a 100644
--- a/Documentation/process/changes.rst
+++ b/Documentation/process/changes.rst
@@ -322,6 +322,24 @@ PDF outputs, it is recommended to use version 1.4.6.
functionalities required for ``XeLaTex`` to work. For PDF output you'll also
need ``convert(1)`` from ImageMagick (https://www.imagemagick.org).

+Most distributions are shipped with Sphinx, but sometimes you may need
+to use a version different than the one shipped. For those cases, you
+can use Sphinx on a virtual environment. For example, if you want to
+install the Python 3 Sphinx version 1.4.9, do::
+
+ $ virtualenv-3 sphinx_1.4
+ $ . sphinx_1.4/bin/activate
+ $ pip install 'Sphinx==1.4.9'
+
+.. note::
+
+ #) It is recommended to use the RTD theme for html output. Depending
+ on the version, it should be installed in separate,
+ with ``pip install sphinx_rtd_theme``.
+
+ #) The command ``. sphinx_1.4/bin/activate`` sets the environment to
+ use the virtual environment you just install. It should be called
+ when you start a new shell, before running ``make htmldocs``.

Getting updated software
========================
--
2.9.4



2017-06-19 13:08:51

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] changes.rst: explain the usage of virtual environment

On Mon, Jun 19, 2017 at 08:24:10AM -0300, Mauro Carvalho Chehab wrote:
> As the Sphinx build seems very fragile, specially for
> PDF output, add a notice about how to use it on a virtual
> environment.

Please don't. python venv are good at one thing only, and that is
making a mess of your python module path. Don't ever use them on a
system that has proper package management.

2017-06-19 13:46:57

by Jonathan Corbet

[permalink] [raw]
Subject: Re: [PATCH] changes.rst: explain the usage of virtual environment

On Mon, 19 Jun 2017 06:08:37 -0700
Christoph Hellwig <[email protected]> wrote:

> On Mon, Jun 19, 2017 at 08:24:10AM -0300, Mauro Carvalho Chehab wrote:
> > As the Sphinx build seems very fragile, specially for
> > PDF output, add a notice about how to use it on a virtual
> > environment.
>
> Please don't. python venv are good at one thing only, and that is
> making a mess of your python module path. Don't ever use them on a
> system that has proper package management.

Well, they are also good for running specific versions of a given package
without disrupting the setup of your system as a whole, and when even a
system with proper package management may not offer the version you need.

I guess my question with the patch is whether this text really belongs in
changes.rst, or whether it should be part of the doc-guide.

jon

2017-06-19 14:12:11

by Markus Heiser

[permalink] [raw]
Subject: Re: [PATCH] changes.rst: explain the usage of virtual environment


> Am 19.06.2017 um 15:46 schrieb Jonathan Corbet <[email protected]>:
>
> On Mon, 19 Jun 2017 06:08:37 -0700
> Christoph Hellwig <[email protected]> wrote:
>
>> On Mon, Jun 19, 2017 at 08:24:10AM -0300, Mauro Carvalho Chehab wrote:
>>> As the Sphinx build seems very fragile, specially for
>>> PDF output, add a notice about how to use it on a virtual
>>> environment.
>>
>> Please don't. python venv are good at one thing only, and that is
>> making a mess of your python module path. Don't ever use them on a
>> system that has proper package management.
>
> Well, they are also good for running specific versions of a given package
> without disrupting the setup of your system as a whole, and when even a
> system with proper package management may not offer the version you need.
>
> I guess my question with the patch is whether this text really belongs in
> changes.rst, or whether it should be part of the doc-guide.

Hi, here are my 5cents:

Typically I have a PY_ENV target in my projects, building a virtualenv
in a folder named ./local. E.g. in LinuxDoc [1] I use something like this:


PY ?=3
PYTHON ?= python$(PY)
..
VIRTUALENV = virtualenv --python=$(PYTHON)
VTENV_OPTS = "--no-site-packages"
PY_ENV = ./local/py$(PY)
..
quiet_cmd_virtualenv = PYENV $@
cmd_virtualenv = \
if [ ! -d "./$(PY_ENV)" ];then \
$(VIRTUALENV) $(VIRTUALENV_VERBOSE) $(VTENV_OPTS) $2; \
else \
echo "using virtualenv from $2"; \
fi
...
# to build *local* environment, python and virtualenv from the OS is needed!
$(PY_ENV): virtualenv-exe python-exe
$(call cmd,virtualenv,$(PY_ENV))
@$(PY_ENV_BIN)/pip install $(PIP_VERBOSE) -r requirements.txt
..

And the sphinxbuild coammand is used from there::

SPHINXBUILD ?= $(PY_ENV_BIN)/sphinx-build

By this I can stick versions packages. E.g. to select last version of
RTD theme and Sphinx version 1.5 or upper, I add the following lines to
my reqierements.txt::

Sphinx>=1.5
sphinx_rtd_theme

If you are interested, I can prepare a patch, to add such functionality
(as option) to Documents/Makefile (which will be documented in the doc-guide).

[1] https://github.com/return42/linuxdoc/blob/master/utils/makefile.python

-- Markus --






>
> jon
> --
> To unsubscribe from this list: send the line "unsubscribe linux-doc" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html

2017-06-19 14:22:14

by Mauro Carvalho Chehab

[permalink] [raw]
Subject: Re: [PATCH] changes.rst: explain the usage of virtual environment

Em Mon, 19 Jun 2017 07:46:53 -0600
Jonathan Corbet <[email protected]> escreveu:

> On Mon, 19 Jun 2017 06:08:37 -0700
> Christoph Hellwig <[email protected]> wrote:
>
> > On Mon, Jun 19, 2017 at 08:24:10AM -0300, Mauro Carvalho Chehab wrote:
> > > As the Sphinx build seems very fragile, specially for
> > > PDF output, add a notice about how to use it on a virtual
> > > environment.
> >
> > Please don't. python venv are good at one thing only, and that is
> > making a mess of your python module path. Don't ever use them on a
> > system that has proper package management.
>
> Well, they are also good for running specific versions of a given package
> without disrupting the setup of your system as a whole, and when even a
> system with proper package management may not offer the version you need.

Yes. Based on my experience with Sphinx, upgrading it to a new
version is painful, as it can cause regressions on docs build.

Also, when a new version is released, the previously stable version
becomes abandoned. Not even trivial and/or compatibility fixes are
accepted against old versions.

So, at least for me, it works best if an specific version is installed
inside a virtual environment, as it won't cause problems when I update
other packages on my distro.

The one I'm using here is version 1.4.9 (the latest 1.4.x version).

Right now, vesion 1.5.x require a series of patches from Jon to
fix PDF build, and more stuff will be needed to fix PDF builds with
1.6.x.

> I guess my question with the patch is whether this text really belongs in
> changes.rst, or whether it should be part of the doc-guide.

Jon,

If you prefer, I can move such description to doc-guide, but I
suspect that we may need to specify, at changes.rst, what Sphinx
versions are known to work fine.

Thanks,
Mauro

2017-06-19 14:38:18

by Mauro Carvalho Chehab

[permalink] [raw]
Subject: Re: [PATCH] changes.rst: explain the usage of virtual environment

HI Markus,

Em Mon, 19 Jun 2017 16:11:56 +0200
Markus Heiser <[email protected]> escreveu:

> > Am 19.06.2017 um 15:46 schrieb Jonathan Corbet <[email protected]>:
> >
> > On Mon, 19 Jun 2017 06:08:37 -0700
> > Christoph Hellwig <[email protected]> wrote:
> >
> >> On Mon, Jun 19, 2017 at 08:24:10AM -0300, Mauro Carvalho Chehab wrote:
> >>> As the Sphinx build seems very fragile, specially for
> >>> PDF output, add a notice about how to use it on a virtual
> >>> environment.
> >>
> >> Please don't. python venv are good at one thing only, and that is
> >> making a mess of your python module path. Don't ever use them on a
> >> system that has proper package management.
> >
> > Well, they are also good for running specific versions of a given package
> > without disrupting the setup of your system as a whole, and when even a
> > system with proper package management may not offer the version you need.
> >
> > I guess my question with the patch is whether this text really belongs in
> > changes.rst, or whether it should be part of the doc-guide.
>
> Hi, here are my 5cents:
>
> Typically I have a PY_ENV target in my projects, building a virtualenv
> in a folder named ./local. E.g. in LinuxDoc [1] I use something like this:
>
>
> PY ?=3
> PYTHON ?= python$(PY)
> ..
> VIRTUALENV = virtualenv --python=$(PYTHON)
> VTENV_OPTS = "--no-site-packages"
> PY_ENV = ./local/py$(PY)

I would split the PATH name on a separate var. This way, if one would
like to have multiple Sphinx versions, all it would need would be to
change the directory.
I would prefer to call it as "./sphinx" (or something similar).

So, I would do:

PY_DIR = ./sphinx
PY_ENV = $(PY_DIR)/py$(PY)

Don't forget to add $PY_DIR directory to .gitignore on your patch.

> ..
> quiet_cmd_virtualenv = PYENV $@
> cmd_virtualenv = \
> if [ ! -d "./$(PY_ENV)" ];then \
> $(VIRTUALENV) $(VIRTUALENV_VERBOSE) $(VTENV_OPTS) $2; \
> else \
> echo "using virtualenv from $2"; \
> fi
> ...
> # to build *local* environment, python and virtualenv from the OS is needed!
> $(PY_ENV): virtualenv-exe python-exe
> $(call cmd,virtualenv,$(PY_ENV))
> @$(PY_ENV_BIN)/pip install $(PIP_VERBOSE) -r requirements.txt

Shouldn't it be using "pip$(PY)" instead?

Also, better to seek for requirements on a file under Documentation/sphinx/
directory.

> ..
>
> And the sphinxbuild coammand is used from there::
>
> SPHINXBUILD ?= $(PY_ENV_BIN)/sphinx-build
>
> By this I can stick versions packages. E.g. to select last version of
> RTD theme and Sphinx version 1.5 or upper, I add the following lines to
> my reqierements.txt::
>
> Sphinx>=1.5
> sphinx_rtd_theme
>
> If you are interested, I can prepare a patch, to add such functionality
> (as option) to Documents/Makefile (which will be documented in the doc-guide).

Yeah, IMHO, it makes sense to have something like that at the main build,
as an optional feature, e. g. perhaps adding a new make target, like:

$ make sphinx_virtenv

That would create and populate PY_DIR. If $PY_DIR/bin/sphinx-build exists
and it is executable file, run it. Otherwise, use the system's sphinx,
if available.

>
> [1] https://github.com/return42/linuxdoc/blob/master/utils/makefile.python
>
> -- Markus --
>
>
>
>
>
>
> >
> > jon
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-doc" in
> > the body of a message to [email protected]
> > More majordomo info at http://vger.kernel.org/majordomo-info.html
>



Thanks,
Mauro

2017-06-19 15:13:24

by Markus Heiser

[permalink] [raw]
Subject: Re: [PATCH] changes.rst: explain the usage of virtual environment


> Am 19.06.2017 um 16:38 schrieb Mauro Carvalho Chehab <[email protected]>:
>
> HI Markus,
>

Hi Mauro :)

[...]

>> Typically I have a PY_ENV target in my projects, building a virtualenv
>> in a folder named ./local. E.g. in LinuxDoc [1] I use something like this:
>>
>> PY ?=3
>> PYTHON ?= python$(PY)
>> ..
>> VIRTUALENV = virtualenv --python=$(PYTHON)
>> VTENV_OPTS = "--no-site-packages"
>> PY_ENV = ./local/py$(PY)
>
> I would split the PATH name on a separate var. This way, if one would
> like to have multiple Sphinx versions, all it would need would be to
> change the directory.
> I would prefer to call it as "./sphinx" (or something similar).

I will take this in mind.

> So, I would do:
>
> PY_DIR = ./sphinx
> PY_ENV = $(PY_DIR)/py$(PY)
>
> Don't forget to add $PY_DIR directory to .gitignore on your patch.

good point ;)

>> ..
>> quiet_cmd_virtualenv = PYENV $@
>> cmd_virtualenv = \
>> if [ ! -d "./$(PY_ENV)" ];then \
>> $(VIRTUALENV) $(VIRTUALENV_VERBOSE) $(VTENV_OPTS) $2; \
>> else \
>> echo "using virtualenv from $2"; \
>> fi
>> ...
>> # to build *local* environment, python and virtualenv from the OS is needed!
>> $(PY_ENV): virtualenv-exe python-exe
>> $(call cmd,virtualenv,$(PY_ENV))
>> @$(PY_ENV_BIN)/pip install $(PIP_VERBOSE) -r requirements.txt
>
> Shouldn't it be using "pip$(PY)" instead?

No, with @$(PY_ENV_BIN)/pip you always use the pip from the environment
and this is always named pip. E.g::

./local/py3/bin/pip
./local/py2/bin/pip

Or: same as "sphinx-build" command .. there is no sohinx-build3 ;)

> Also, better to seek for requirements on a file under Documentation/sphinx/
> directory.

right, the above was only a sloppily C&P from what I have in my project.

>> ..
>>
>> And the sphinxbuild coammand is used from there::
>>
>> SPHINXBUILD ?= $(PY_ENV_BIN)/sphinx-build
>>
>> By this I can stick versions packages. E.g. to select last version of
>> RTD theme and Sphinx version 1.5 or upper, I add the following lines to
>> my reqierements.txt::
>>
>> Sphinx>=1.5
>> sphinx_rtd_theme
>>
>> If you are interested, I can prepare a patch, to add such functionality
>> (as option) to Documents/Makefile (which will be documented in the doc-guide).
>
> Yeah, IMHO, it makes sense to have something like that at the main build,
> as an optional feature, e. g. perhaps adding a new make target, like:
>
> $ make sphinx_virtenv
>
> That would create and populate PY_DIR. If $PY_DIR/bin/sphinx-build exists
> and it is executable file, run it. Otherwise, use the system's sphinx,
> if available.

OK, I will prepare a RFC patch ... I will have time for this only on the
weekend, so have some patience with me.

-- Markus --

2017-06-20 15:04:11

by Markus Heiser

[permalink] [raw]
Subject: Re: [PATCH] changes.rst: explain the usage of virtual environment

[...]

> Am 19.06.2017 um 17:13 schrieb Markus Heiser <[email protected]>:
>
>>> Typically I have a PY_ENV target in my projects, building a virtualenv
>>> in a folder named ./local.

[...]

>> Yeah, IMHO, it makes sense to have something like that at the main build,
>> as an optional feature,

> OK, I will prepare a RFC patch ...

For those who are interested in / lets discuss further in context
of my patch:

https://www.mail-archive.com/[email protected]/msg12844.html

Thanks!

-- Markus --