The Sphinx folks are deprecating some interfaces in the upcoming 2.0
release; one immediate result of that is a bunch of warnings that show up
when building with 1.8. These two patches make those warnings go away,
but at a cost:
- It introduces a couple of Sphinx version checks, which are always
ugly, but the alternative would be to stop supporting versions
before 1.7. For now, I think we can carry that cruft.
- The second patch causes the build to fail horribly on newer
Sphinx installations. The change to switch_source_input() seems
to make the parser much more finicky, increasing warnings and
eventually failing the build altogether. In particular, it will
scream about problems in .rst files that are not included in the
TOC tree at all. The complaints appear to be legitimate, but it's
a bunch of stuff to clean up.
I've tested these with 1.4 and 1.8, but not various versions in between.
Jonathan Corbet (2):
doc: Cope with Sphinx logging deprecations
doc: Cope with the deprecation of AutoReporter
Documentation/sphinx/kerneldoc.py | 48 ++++++++++++++++++++++++-------
Documentation/sphinx/kernellog.py | 28 ++++++++++++++++++
Documentation/sphinx/kfigure.py | 38 +++++++++++++-----------
3 files changed, 87 insertions(+), 27 deletions(-)
create mode 100644 Documentation/sphinx/kernellog.py
--
2.21.0
On Tue, 21 May 2019, Jonathan Corbet <[email protected]> wrote:
> The Sphinx folks are deprecating some interfaces in the upcoming 2.0
> release; one immediate result of that is a bunch of warnings that show up
> when building with 1.8. These two patches make those warnings go away,
> but at a cost:
>
> - It introduces a couple of Sphinx version checks, which are always
> ugly, but the alternative would be to stop supporting versions
> before 1.7. For now, I think we can carry that cruft.
Frankly, I'd just require Sphinx 1.7+, available even in Debian stable
through stretch-backports.
> - The second patch causes the build to fail horribly on newer
> Sphinx installations. The change to switch_source_input() seems
> to make the parser much more finicky, increasing warnings and
> eventually failing the build altogether. In particular, it will
> scream about problems in .rst files that are not included in the
> TOC tree at all. The complaints appear to be legitimate, but it's
> a bunch of stuff to clean up.
I can understand Sphinx complaining that a file is not included in a TOC
tree, but I don't understand why it goes on to parse them anyway.
BR,
Jani.
>
> I've tested these with 1.4 and 1.8, but not various versions in between.
>
> Jonathan Corbet (2):
> doc: Cope with Sphinx logging deprecations
> doc: Cope with the deprecation of AutoReporter
>
> Documentation/sphinx/kerneldoc.py | 48 ++++++++++++++++++++++++-------
> Documentation/sphinx/kernellog.py | 28 ++++++++++++++++++
> Documentation/sphinx/kfigure.py | 38 +++++++++++++-----------
> 3 files changed, 87 insertions(+), 27 deletions(-)
> create mode 100644 Documentation/sphinx/kernellog.py
--
Jani Nikula, Intel Open Source Graphics Center
On Tue, May 21, 2019 at 03:17:12PM -0600, Jonathan Corbet wrote:
> The Sphinx folks are deprecating some interfaces in the upcoming 2.0
> release; one immediate result of that is a bunch of warnings that show up
> when building with 1.8. These two patches make those warnings go away,
> but at a cost:
A minor correction, if I may and if I understand this correctly: 2.0 is
not an upcoming release, but a current one (2.0.1, to be precise), and
this means that in some distros (like, Arch [1]) `make htmldocs` is
already broken for quite some time.
[1] https://bugs.archlinux.org/task/59688
>
> - It introduces a couple of Sphinx version checks, which are always
> ugly, but the alternative would be to stop supporting versions
> before 1.7. For now, I think we can carry that cruft.
>
> - The second patch causes the build to fail horribly on newer
> Sphinx installations. The change to switch_source_input() seems
> to make the parser much more finicky, increasing warnings and
> eventually failing the build altogether. In particular, it will
> scream about problems in .rst files that are not included in the
> TOC tree at all. The complaints appear to be legitimate, but it's
> a bunch of stuff to clean up.
>
> I've tested these with 1.4 and 1.8, but not various versions in between.
>
> Jonathan Corbet (2):
> doc: Cope with Sphinx logging deprecations
> doc: Cope with the deprecation of AutoReporter
>
> Documentation/sphinx/kerneldoc.py | 48 ++++++++++++++++++++++++-------
> Documentation/sphinx/kernellog.py | 28 ++++++++++++++++++
> Documentation/sphinx/kfigure.py | 38 +++++++++++++-----------
> 3 files changed, 87 insertions(+), 27 deletions(-)
> create mode 100644 Documentation/sphinx/kernellog.py
>
> --
> 2.21.0
>
--
Best regards,
Oleksandr Natalenko (post-factum)
Senior Software Maintenance Engineer
On Wed, May 22, 2019 at 11:43:54AM +0200, Oleksandr Natalenko wrote:
> On Tue, May 21, 2019 at 03:17:12PM -0600, Jonathan Corbet wrote:
> > The Sphinx folks are deprecating some interfaces in the upcoming 2.0
> > release; one immediate result of that is a bunch of warnings that show up
> > when building with 1.8. These two patches make those warnings go away,
> > but at a cost:
>
> A minor correction, if I may and if I understand this correctly: 2.0 is
> not an upcoming release, but a current one (2.0.1, to be precise), and
> this means that in some distros (like, Arch [1]) `make htmldocs` is
> already broken for quite some time.
>
> [1] https://bugs.archlinux.org/task/59688
^^ this was the initial Bug for introducing the doc, but it got reverted
in [2].
[2] https://git.archlinux.org/svntogit/packages.git/commit/trunk?h=packages/linux&id=cfe52e9aa8168d9571bedf8a376e2cfbd25223fd
>
> >
> > - It introduces a couple of Sphinx version checks, which are always
> > ugly, but the alternative would be to stop supporting versions
> > before 1.7. For now, I think we can carry that cruft.
> >
> > - The second patch causes the build to fail horribly on newer
> > Sphinx installations. The change to switch_source_input() seems
> > to make the parser much more finicky, increasing warnings and
> > eventually failing the build altogether. In particular, it will
> > scream about problems in .rst files that are not included in the
> > TOC tree at all. The complaints appear to be legitimate, but it's
> > a bunch of stuff to clean up.
> >
> > I've tested these with 1.4 and 1.8, but not various versions in between.
> >
> > Jonathan Corbet (2):
> > doc: Cope with Sphinx logging deprecations
> > doc: Cope with the deprecation of AutoReporter
> >
> > Documentation/sphinx/kerneldoc.py | 48 ++++++++++++++++++++++++-------
> > Documentation/sphinx/kernellog.py | 28 ++++++++++++++++++
> > Documentation/sphinx/kfigure.py | 38 +++++++++++++-----------
> > 3 files changed, 87 insertions(+), 27 deletions(-)
> > create mode 100644 Documentation/sphinx/kernellog.py
> >
> > --
> > 2.21.0
> >
>
> --
> Best regards,
> Oleksandr Natalenko (post-factum)
> Senior Software Maintenance Engineer
--
Best regards,
Oleksandr Natalenko (post-factum)
Senior Software Maintenance Engineer
Em Wed, 22 May 2019 10:36:45 +0300
Jani Nikula <[email protected]> escreveu:
> On Tue, 21 May 2019, Jonathan Corbet <[email protected]> wrote:
> > The Sphinx folks are deprecating some interfaces in the upcoming 2.0
> > release; one immediate result of that is a bunch of warnings that show up
> > when building with 1.8. These two patches make those warnings go away,
> > but at a cost:
> >
> > - It introduces a couple of Sphinx version checks, which are always
> > ugly, but the alternative would be to stop supporting versions
> > before 1.7. For now, I think we can carry that cruft.
>
> Frankly, I'd just require Sphinx 1.7+, available even in Debian stable
> through stretch-backports.
We can raise the bar and force a 1.7 or even 1.8 (Fedora 30 comes
with version 1.8.4), but I would prefer to keep support older versions,
at least while we don't depend on some new features introduced after
the version we're using, and while our extensions won't require a major
rework due to a new version.
>
> > - The second patch causes the build to fail horribly on newer
> > Sphinx installations. The change to switch_source_input() seems
> > to make the parser much more finicky, increasing warnings and
> > eventually failing the build altogether. In particular, it will
> > scream about problems in .rst files that are not included in the
> > TOC tree at all. The complaints appear to be legitimate, but it's
> > a bunch of stuff to clean up.
There is a flag to cleanup the warning about a file not included at
a TOC tree (:orphan:), but it will still try to parse it. There's also
a conf.py way of doing it. For example, you can exclude an entire dir:
exclude_patterns = ['includes/*.rst']
But using exclude_patterns will likely be too messy.
> I can understand Sphinx complaining that a file is not included in a TOC
> tree, but I don't understand why it goes on to parse them anyway.
Yeah, this is, IMHO, a very bad behavior.
>
> BR,
> Jani.
>
>
> >
> > I've tested these with 1.4 and 1.8, but not various versions in between.
> >
> > Jonathan Corbet (2):
> > doc: Cope with Sphinx logging deprecations
> > doc: Cope with the deprecation of AutoReporter
> >
> > Documentation/sphinx/kerneldoc.py | 48 ++++++++++++++++++++++++-------
> > Documentation/sphinx/kernellog.py | 28 ++++++++++++++++++
> > Documentation/sphinx/kfigure.py | 38 +++++++++++++-----------
> > 3 files changed, 87 insertions(+), 27 deletions(-)
> > create mode 100644 Documentation/sphinx/kernellog.py
>
Thanks,
Mauro
22.05.19 um 12:19 schrieb Mauro Carvalho Chehab:
> Em Wed, 22 May 2019 10:36:45 +0300
> Jani Nikula <[email protected]> escreveu:
>
>> On Tue, 21 May 2019, Jonathan Corbet <[email protected]> wrote:
>>> The Sphinx folks are deprecating some interfaces in the upcoming 2.0
>>> release; one immediate result of that is a bunch of warnings that show up
>>> when building with 1.8. These two patches make those warnings go away,
>>> but at a cost:
>>>
>>> - It introduces a couple of Sphinx version checks, which are always
>>> ugly, but the alternative would be to stop supporting versions
>>> before 1.7. For now, I think we can carry that cruft.
>>
>> Frankly, I'd just require Sphinx 1.7+, available even in Debian stable
>> through stretch-backports.
>
> We can raise the bar and force a 1.7 or even 1.8 (Fedora 30 comes
> with version 1.8.4), but I would prefer to keep support older versions,
> at least while we don't depend on some new features introduced after
> the version we're using, and while our extensions won't require a major
> rework due to a new version.
Lets use 1.7 :
- no need for Use_SSI wrapper
- new log should work with 1.7 [1] --> no need for kernellog.py and
additional imports, instead include on top of python modules ::
from sphinx.util import logging
logger = logging.getLogger('kerneldoc')
[1]
https://github.com/sphinx-doc/sphinx/commit/6d4e6454093953943e79d4db6efeb17390870e62
BTW we can drop other (old) sphinx-version issues e.g.
Documentation/conf.py which fails with version 2.0:
diff --git a/Documentation/conf.py b/Documentation/conf.py
index 72647a38b5c2..ba82715b6715 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -34,13 +34,7 @@ needs_sphinx = '1.3'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain',
'kfigure', 'sphinx.ext.ifconfig']
-
-# The name of the math extension changed on Sphinx 1.4
-if major == 1 and minor > 3:
- extensions.append("sphinx.ext.imgmath")
-else:
- extensions.append("sphinx.ext.pngmath")
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain',
'kfigure', 'sphinx.ext.ifconfig', 'sphinx.ext.imgmath']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
-- Markus --
Em Wed, 22 May 2019 09:45:59 -0600
Jonathan Corbet <[email protected]> escreveu:
> On Wed, 22 May 2019 15:25:36 +0200
> Markus Heiser <[email protected]> wrote:
>
> > Lets use 1.7 :
> >
> > - no need for Use_SSI wrapper
> > - new log should work with 1.7 [1] --> no need for kernellog.py and
> > additional imports, instead include on top of python modules ::
> >
> > from sphinx.util import logging
> > logger = logging.getLogger('kerneldoc')
>
> I think we're going to have to drag things forward at some point in the
> not-too-distant future, but I think I'd rather not do that quite yet. The
> cost of supporting older sphinx for a few releases while we warn people is
> not all that high. So I think we should:
>
> - Put in (a future version of) my hacks for now, plus whatever else might
> be needed to make 2.0 work right.
>
> - Fix the fallout with regard to out-of-toctree .rst files so that we can
> actually build again with current sphinx.
>
> - Update Documentation/sphinx/requirements.txt to ask for something a wee
> bit more recent than 1.4.9.
You should remember to also update conf.py (with currently points to 1.3):
# If your documentation needs a minimal Sphinx version, state it here.
needs_sphinx = '1.3'
Also, if you touch there, you should also touch:
./scripts/sphinx-pre-install
The change there won't be as trivial as just changing this line:
$virtenv_dir = "sphinx_1.4";
as the script should now run sphinx-build --version, in order to check
if the version is lower than the new minimal version. It probably makes
sense to make it grep the version from needs_sphinx at conf.py.
> - Add a warning when building with an older version that (say) 1.7 will
> be required as of (say) 5.5.
It probably makes sense to add such check at the pre-install script,
and add a:
SPHINXOPTS="-jauto"
somewhere if version is 1.7 or upper.
>
> Does this make sense?
It makes sense to me.
Thanks,
Mauro
Em Wed, 22 May 2019 13:04:08 -0300
Mauro Carvalho Chehab <[email protected]> escreveu:
> Em Wed, 22 May 2019 09:45:59 -0600
> Jonathan Corbet <[email protected]> escreveu:
>
> > On Wed, 22 May 2019 15:25:36 +0200
> > Markus Heiser <[email protected]> wrote:
> >
> > > Lets use 1.7 :
> > >
> > > - no need for Use_SSI wrapper
> > > - new log should work with 1.7 [1] --> no need for kernellog.py and
> > > additional imports, instead include on top of python modules ::
> > >
> > > from sphinx.util import logging
> > > logger = logging.getLogger('kerneldoc')
> >
> > I think we're going to have to drag things forward at some point in the
> > not-too-distant future, but I think I'd rather not do that quite yet. The
> > cost of supporting older sphinx for a few releases while we warn people is
> > not all that high. So I think we should:
> >
> > - Put in (a future version of) my hacks for now, plus whatever else might
> > be needed to make 2.0 work right.
> >
> > - Fix the fallout with regard to out-of-toctree .rst files so that we can
> > actually build again with current sphinx.
> >
> > - Update Documentation/sphinx/requirements.txt to ask for something a wee
> > bit more recent than 1.4.9.
>
> You should remember to also update conf.py (with currently points to 1.3):
>
> # If your documentation needs a minimal Sphinx version, state it here.
> needs_sphinx = '1.3'
>
> Also, if you touch there, you should also touch:
>
> ./scripts/sphinx-pre-install
>
> The change there won't be as trivial as just changing this line:
>
> $virtenv_dir = "sphinx_1.4";
>
> as the script should now run sphinx-build --version, in order to check
> if the version is lower than the new minimal version. It probably makes
> sense to make it grep the version from needs_sphinx at conf.py.
>
> > - Add a warning when building with an older version that (say) 1.7 will
> > be required as of (say) 5.5.
>
> It probably makes sense to add such check at the pre-install script,
> and add a:
>
> SPHINXOPTS="-jauto"
>
> somewhere if version is 1.7 or upper.
>
I'm meaning something like the enclosed patch.
(PS.: I'm still working at the patch)
Thanks,
Mauro
[RFC PATCH] scripts/sphinx-pre-install: make it handle Sphinx versions
As we want to switch to a newer Sphinx version in the future,
add some version detected logic.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index f6a5c0bae31e..8835aede4c61 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -13,7 +13,7 @@ use strict;
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
-my $virtenv_dir = "sphinx_1.4";
+my $conf = "Documentation/conf.py";
my $requirement_file = "Documentation/sphinx/requirements.txt";
#
@@ -27,6 +27,10 @@ my $optional = 0;
my $need_symlink = 0;
my $need_sphinx = 0;
my $install = "";
+my $min_version;
+my $rec_version;
+my $cur_version;
+my $virtenv_dir = "sphinx_";
#
# Command line arguments
@@ -76,6 +80,52 @@ my %texlive = (
# Subroutines that checks if a feature exists
#
+sub handle_sphinx_version()
+{
+ open IN, $conf;
+ while (<IN>) {
+ if (m/^\s*needs_sphinx\s*=\s*[\'\"]([\d\.]+)[\'\"]/) {
+ $min_version=$1;
+ last;
+ }
+ }
+ close IN;
+
+ die "Can't get needs_sphinx version from $conf" if (!$min_version);
+
+ open IN, $requirement_file;
+ while (<IN>) {
+ if (m/^\s*Sphinx\s*==\s*([\d\.]+)$/) {
+ $rec_version=$1;
+ last;
+ }
+ }
+ close IN;
+
+ open IN, "sphinx-build --version 2>&1 |";
+ while (<IN>) {
+ if (m/^\s*sphinx-build\s+([\d\.]+)$/) {
+ $cur_version=$1;
+ last;
+ }
+ }
+ close IN;
+
+ $virtenv_dir .= $rec_version;
+
+ # Sphinx is not installed
+ return if (!$cur_version);
+
+ if ($cur_version lt $min_version) {
+ print "Sphinx version older than $min_version! We recommend at least $rec_version";
+ exit -1;
+ }
+
+ if ($cur_version lt $rec_version) {
+ print "Warning: we recommend at least Sphinx version $rec_version";
+ }
+}
+
sub check_missing(%)
{
my %map = %{$_[0]};
@@ -587,6 +637,8 @@ while (@ARGV) {
}
}
+handle_sphinx_version();
+
#
# Determine the system type. There's no standard unique way that would
# work with all distros with a minimal package install. So, several
On Wed, 22 May 2019 15:25:36 +0200
Markus Heiser <[email protected]> wrote:
> Lets use 1.7 :
>
> - no need for Use_SSI wrapper
> - new log should work with 1.7 [1] --> no need for kernellog.py and
> additional imports, instead include on top of python modules ::
>
> from sphinx.util import logging
> logger = logging.getLogger('kerneldoc')
I think we're going to have to drag things forward at some point in the
not-too-distant future, but I think I'd rather not do that quite yet. The
cost of supporting older sphinx for a few releases while we warn people is
not all that high. So I think we should:
- Put in (a future version of) my hacks for now, plus whatever else might
be needed to make 2.0 work right.
- Fix the fallout with regard to out-of-toctree .rst files so that we can
actually build again with current sphinx.
- Update Documentation/sphinx/requirements.txt to ask for something a wee
bit more recent than 1.4.9.
- Add a warning when building with an older version that (say) 1.7 will
be required as of (say) 5.5.
Does this make sense?
Thanks,
jon