Hi Jon,
This series addresses your concerns related to CJK fonts that are
needed for translations.pdf.
It touches only the documentation build system, not the docs
themselves.
It ended to be bigger than I originally foreseen, as I found several issues
when running "make pdfdocs" for the distros that are recognized by
the scripts/sphinx-pre-install script.
It also took a lot of time, as I tested it with several VMs (each
one updated to latest packages):
- Fedora 30, CentOS 7, Mageia 7, ArchLinux, Ubuntu 18.04, Gentoo,
OpenSuse Tumbleweed.
Patch 1 addresses an issue that could be related to the fact that I
don't use openSUSE. Basically, I was unable to find the right package
for texlive to use CJK fonts on openSUSE. [1]. So, the first patch on this
series adds a workaround: if the needed CJK font is not found on a
system, conf.py won't use xeCjk extension. That sounds a good
thing to have, as other distros may not package it, or maybe the
one building the doc is not that interested on translations.pdf file;
[1] I actually found some, but they are not recognized with the
font name conf.py is expecting ("Noto Sans CJK SC"). Perhaps
SUSE uses a different name for those fonts?
Patch 2 fixes the logic with recognizes CentOS/RHEL;
Patch 3 is another workaround: CentOS 7 (and similar distros) don't
package all texlive packages we need. So, it just ignores PDF when
recommending packages on such distros, and point to a URL with
explains how to install TexLive outside distro-specific package
management (for the brave enough people);
Patch 4 fixes latexmk dependency on a few distros;
Patch 5 suppreses a Gentoo specific instruction if the user already
followed in the past;
Patch 6 is the one that actually does what you requested.
Patch 7 solves an issue when SPHINXDIRS is used with make pdfdocs:
right now, using it will produce a lot of warnings and won't do anything,
if a dir-specific conf.py file is not found. With the patch, latex_documents
are now properly updated when SPHINXDIRS is used.
Patch 8 is a cleanup: with patch 7 applied, we don't need to have anymore
any conf.py file due to pdfdocs.
With regard to the load_config.py extension, It keeps accepting custom
configuration. That's helpful if someone wants, for example, to have
something like:
Documentation/media/conf_nitpick.py
with would enable extra nitpick options if one wants that.
-
Jon,
Please let me know if you prefer if I submit those together with the big
pile of doc files I have, or if you prefer adding (some of?) them on your
tree after the merge window.
Regards,
Mauro
Mauro Carvalho Chehab (8):
docs: conf.py: only use CJK if the font is available
scripts/sphinx-pre-install: fix script for RHEL/CentOS
scripts/sphinx-pre-install: don't use LaTeX with CentOS 7
scripts/sphinx-pre-install: fix latexmk dependencies
scripts/sphinx-pre-install: cleanup Gentoo checks
scripts/sphinx-pre-install: seek for Noto CJK fonts for pdf output
docs: load_config.py: avoid needing a conf.py just due to LaTeX docs
docs: remove extra conf.py files
Documentation/admin-guide/conf.py | 10 ---
Documentation/conf.py | 13 ++-
Documentation/core-api/conf.py | 10 ---
Documentation/crypto/conf.py | 10 ---
Documentation/dev-tools/conf.py | 10 ---
Documentation/doc-guide/conf.py | 10 ---
Documentation/driver-api/80211/conf.py | 10 ---
Documentation/driver-api/conf.py | 10 ---
Documentation/driver-api/pm/conf.py | 10 ---
Documentation/filesystems/conf.py | 10 ---
Documentation/gpu/conf.py | 10 ---
Documentation/input/conf.py | 10 ---
Documentation/kernel-hacking/conf.py | 10 ---
Documentation/maintainer/conf.py | 10 ---
Documentation/media/conf.py | 12 ---
Documentation/networking/conf.py | 10 ---
Documentation/process/conf.py | 10 ---
Documentation/sh/conf.py | 10 ---
Documentation/sound/conf.py | 10 ---
Documentation/sphinx/load_config.py | 25 +++++-
Documentation/userspace-api/conf.py | 10 ---
Documentation/vm/conf.py | 10 ---
Documentation/x86/conf.py | 10 ---
scripts/sphinx-pre-install | 118 ++++++++++++++++++++-----
24 files changed, 131 insertions(+), 237 deletions(-)
delete mode 100644 Documentation/admin-guide/conf.py
delete mode 100644 Documentation/core-api/conf.py
delete mode 100644 Documentation/crypto/conf.py
delete mode 100644 Documentation/dev-tools/conf.py
delete mode 100644 Documentation/doc-guide/conf.py
delete mode 100644 Documentation/driver-api/80211/conf.py
delete mode 100644 Documentation/driver-api/conf.py
delete mode 100644 Documentation/driver-api/pm/conf.py
delete mode 100644 Documentation/filesystems/conf.py
delete mode 100644 Documentation/gpu/conf.py
delete mode 100644 Documentation/input/conf.py
delete mode 100644 Documentation/kernel-hacking/conf.py
delete mode 100644 Documentation/maintainer/conf.py
delete mode 100644 Documentation/media/conf.py
delete mode 100644 Documentation/networking/conf.py
delete mode 100644 Documentation/process/conf.py
delete mode 100644 Documentation/sh/conf.py
delete mode 100644 Documentation/sound/conf.py
delete mode 100644 Documentation/userspace-api/conf.py
delete mode 100644 Documentation/vm/conf.py
delete mode 100644 Documentation/x86/conf.py
--
2.21.0
There aren't enough texlive packages for LaTeX-based builds
to work on CentOS/RHEL <= 7.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/sphinx-pre-install | 68 ++++++++++++++++++++++++++++----------
1 file changed, 50 insertions(+), 18 deletions(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 101ddd00bf02..33efadd6c0b6 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -83,6 +83,17 @@ sub check_missing(%)
foreach my $prog (sort keys %missing) {
my $is_optional = $missing{$prog};
+ # At least on some LTS distros like CentOS 7, texlive doesn't
+ # provide all packages we need. When such distros are
+ # detected, we have to disable PDF output.
+ #
+ # So, we need to ignore the packages that distros would
+ # need for LaTeX to work
+ if ($is_optional == 2 && !$pdf) {
+ $optional--;
+ next;
+ }
+
if ($is_optional) {
print "Warning: better to also install \"$prog\".\n";
} else {
@@ -333,10 +344,10 @@ sub give_debian_hints()
if ($pdf) {
check_missing_file("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
- "fonts-dejavu", 1);
+ "fonts-dejavu", 2);
}
- check_program("dvipng", 1) if ($pdf);
+ check_program("dvipng", 2) if ($pdf);
check_missing(\%map);
return if (!$need && !$optional);
@@ -371,22 +382,40 @@ sub give_redhat_hints()
#
# Checks valid for RHEL/CentOS version 7.x.
#
+ my $old = 0;
+ my $rel;
+ $rel = $1 if ($system_release =~ /release\s+(\d+)/);
+
if (!($system_release =~ /Fedora/)) {
$map{"virtualenv"} = "python-virtualenv";
- }
- my $release;
+ if ($rel && $rel < 8) {
+ $old = 1;
+ $pdf = 0;
- $release = $1 if ($system_release =~ /Fedora\s+release\s+(\d+)/);
+ printf("Note: texlive packages on RHEL/CENTOS <= 7 are incomplete. Can't support PDF output\n");
+ printf("If you want to build PDF, please read:\n");
+ printf("\thttps://www.systutorials.com/241660/how-to-install-tex-live-on-centos-7-linux/\n");
+ }
+ } else {
+ if ($rel && $rel < 26) {
+ $old = 1;
+ }
+ }
+ if (!$rel) {
+ printf("Couldn't identify release number\n");
+ $old = 1;
+ $pdf = 0;
+ }
- check_rpm_missing(\@fedora26_opt_pkgs, 1) if ($pdf && $release >= 26);
- check_rpm_missing(\@fedora_tex_pkgs, 1) if ($pdf);
- check_missing_tex(1) if ($pdf);
+ check_rpm_missing(\@fedora26_opt_pkgs, 2) if ($pdf && !$old);
+ check_rpm_missing(\@fedora_tex_pkgs, 2) if ($pdf);
+ check_missing_tex(2) if ($pdf);
check_missing(\%map);
return if (!$need && !$optional);
- if ($release >= 18) {
+ if (!$old) {
# dnf, for Fedora 18+
printf("You should run:\n\n\tsudo dnf install -y $install\n");
} else {
@@ -425,8 +454,8 @@ sub give_opensuse_hints()
"texlive-zapfding",
);
- check_rpm_missing(\@suse_tex_pkgs, 1) if ($pdf);
- check_missing_tex(1) if ($pdf);
+ check_rpm_missing(\@suse_tex_pkgs, 2) if ($pdf);
+ check_missing_tex(2) if ($pdf);
check_missing(\%map);
return if (!$need && !$optional);
@@ -450,7 +479,7 @@ sub give_mageia_hints()
"texlive-fontsextra",
);
- check_rpm_missing(\@tex_pkgs, 1) if ($pdf);
+ check_rpm_missing(\@tex_pkgs, 2) if ($pdf);
check_missing(\%map);
return if (!$need && !$optional);
@@ -473,7 +502,8 @@ sub give_arch_linux_hints()
"texlive-latexextra",
"ttf-dejavu",
);
- check_pacman_missing(\@archlinux_tex_pkgs, 1) if ($pdf);
+ check_pacman_missing(\@archlinux_tex_pkgs, 2) if ($pdf);
+
check_missing(\%map);
return if (!$need && !$optional);
@@ -492,7 +522,7 @@ sub give_gentoo_hints()
);
check_missing_file("/usr/share/fonts/dejavu/DejaVuSans.ttf",
- "media-fonts/dejavu", 1) if ($pdf);
+ "media-fonts/dejavu", 2) if ($pdf);
check_missing(\%map);
@@ -560,7 +590,7 @@ sub check_distros()
my %map = (
"sphinx-build" => "sphinx"
);
- check_missing_tex(1) if ($pdf);
+ check_missing_tex(2) if ($pdf);
check_missing(\%map);
print "I don't know distro $system_release.\n";
print "So, I can't provide you a hint with the install procedure.\n";
@@ -589,11 +619,13 @@ sub check_needs()
check_program("make", 0);
check_program("gcc", 0);
check_python_module("sphinx_rtd_theme", 1) if (!$virtualenv);
- check_program("xelatex", 1) if ($pdf);
check_program("dot", 1);
check_program("convert", 1);
- check_program("rsvg-convert", 1) if ($pdf);
- check_program("latexmk", 1) if ($pdf);
+
+ # Extra PDF files - should use 2 for is_optional
+ check_program("xelatex", 2) if ($pdf);
+ check_program("rsvg-convert", 2) if ($pdf);
+ check_program("latexmk", 2) if ($pdf);
check_distros();
--
2.21.0
On Gentoo, the portage changes for ImageMagick to work are
always suggested, even if already applied. While the two
extra commands should be harmless, add a check to avoid
reporting it without need.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/sphinx-pre-install | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 8dc13fe95ffe..0a5c83aa5f44 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -533,8 +533,19 @@ sub give_gentoo_hints()
return if (!$need && !$optional);
printf("You should run:\n\n");
- printf("\tsudo su -c 'echo \"media-gfx/imagemagick svg png\" > /etc/portage/package.use/imagemagick'\n");
- printf("\tsudo su -c 'echo \"media-gfx/graphviz cairo pdf\" > /etc/portage/package.use/graphviz'\n");
+
+ my $imagemagick = "media-gfx/imagemagick svg png";
+ my $cairo = "media-gfx/graphviz cairo pdf";
+ my $portage_imagemagick = "/etc/portage/package.use/imagemagick";
+ my $portage_cairo = "/etc/portage/package.use/graphviz";
+
+ if (qx(cat $portage_imagemagick) ne "$imagemagick\n") {
+ printf("\tsudo su -c 'echo \"$imagemagick\" > $portage_imagemagick'\n")
+ }
+ if (qx(cat $portage_cairo) ne "$cairo\n") {
+ printf("\tsudo su -c 'echo \"$cairo\" > $portage_cairo'\n");
+ }
+
printf("\tsudo emerge --ask $install\n");
}
--
2.21.0
There's a missing parenthesis at the script, with causes it to
fail to detect non-Fedora releases (e. g. RHEL/CentOS).
Tested with Centos 7.6.1810.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/sphinx-pre-install | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index f230e65329a2..101ddd00bf02 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -371,7 +371,7 @@ sub give_redhat_hints()
#
# Checks valid for RHEL/CentOS version 7.x.
#
- if (! $system_release =~ /Fedora/) {
+ if (!($system_release =~ /Fedora/)) {
$map{"virtualenv"} = "python-virtualenv";
}
--
2.21.0
The name of the package with carries latexmk is different
on two distros:
- On OpenSUSE, latexmk is packaged as "texlive-latexmk-bin"
- On Mageia, latexmk is packaged at "texlive-collection-basic"
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/sphinx-pre-install | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 33efadd6c0b6..8dc13fe95ffe 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -454,6 +454,8 @@ sub give_opensuse_hints()
"texlive-zapfding",
);
+ $map{"latexmk"} = "texlive-latexmk-bin";
+
check_rpm_missing(\@suse_tex_pkgs, 2) if ($pdf);
check_missing_tex(2) if ($pdf);
check_missing(\%map);
@@ -479,6 +481,8 @@ sub give_mageia_hints()
"texlive-fontsextra",
);
+ $map{"latexmk"} = "texlive-collection-basic";
+
check_rpm_missing(\@tex_pkgs, 2) if ($pdf);
check_missing(\%map);
--
2.21.0
The translations guide need Noto CJK fonts. So, add a logic that
would suggest its install for distros.
It also fix a few other issues while testing the script
with several distributions.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/sphinx-pre-install | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/scripts/sphinx-pre-install b/scripts/sphinx-pre-install
index 0a5c83aa5f44..3b638c0e1a4f 100755
--- a/scripts/sphinx-pre-install
+++ b/scripts/sphinx-pre-install
@@ -345,6 +345,9 @@ sub give_debian_hints()
if ($pdf) {
check_missing_file("/usr/share/fonts/truetype/dejavu/DejaVuSans.ttf",
"fonts-dejavu", 2);
+
+ check_missing_file("/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc",
+ "fonts-noto-cjk", 2);
}
check_program("dvipng", 2) if ($pdf);
@@ -374,6 +377,7 @@ sub give_redhat_hints()
my @fedora_tex_pkgs = (
"texlive-collection-fontsrecommended",
"texlive-collection-latex",
+ "texlive-xecjk",
"dejavu-sans-fonts",
"dejavu-serif-fonts",
"dejavu-sans-mono-fonts",
@@ -408,6 +412,11 @@ sub give_redhat_hints()
$pdf = 0;
}
+ if ($pdf) {
+ check_missing_file("/usr/share/fonts/google-noto-cjk/NotoSansCJK-Regular.ttc",
+ "google-noto-sans-cjk-ttc-fonts", 2);
+ }
+
check_rpm_missing(\@fedora26_opt_pkgs, 2) if ($pdf && !$old);
check_rpm_missing(\@fedora_tex_pkgs, 2) if ($pdf);
check_missing_tex(2) if ($pdf);
@@ -456,6 +465,11 @@ sub give_opensuse_hints()
$map{"latexmk"} = "texlive-latexmk-bin";
+ # FIXME: add support for installing CJK fonts
+ #
+ # I tried hard, but was unable to find a way to install
+ # "Noto Sans CJK SC" on openSUSE
+
check_rpm_missing(\@suse_tex_pkgs, 2) if ($pdf);
check_missing_tex(2) if ($pdf);
check_missing(\%map);
@@ -483,6 +497,11 @@ sub give_mageia_hints()
$map{"latexmk"} = "texlive-collection-basic";
+ if ($pdf) {
+ check_missing_file("/usr/share/fonts/google-noto-cjk/NotoSansCJK-Regular.ttc",
+ "google-noto-sans-cjk-ttc-fonts", 2);
+ }
+
check_rpm_missing(\@tex_pkgs, 2) if ($pdf);
check_missing(\%map);
@@ -508,6 +527,11 @@ sub give_arch_linux_hints()
);
check_pacman_missing(\@archlinux_tex_pkgs, 2) if ($pdf);
+ if ($pdf) {
+ check_missing_file("/usr/share/fonts/noto-cjk/NotoSansCJK-Regular.ttc",
+ "noto-fonts-cjk", 2);
+ }
+
check_missing(\%map);
return if (!$need && !$optional);
@@ -528,6 +552,11 @@ sub give_gentoo_hints()
check_missing_file("/usr/share/fonts/dejavu/DejaVuSans.ttf",
"media-fonts/dejavu", 2) if ($pdf);
+ if ($pdf) {
+ check_missing_file("/usr/share/fonts/noto-cjk/NotoSansCJKsc-Regular.otf",
+ "media-fonts/noto-cjk", 2);
+ }
+
check_missing(\%map);
return if (!$need && !$optional);
--
2.21.0
Right now, for every directory that we need to have LaTeX output,
a conf.py file is required.
That causes an extra overhead and it is actually a hack, as
the latex_documents line there are usually a copy of the ones
that are there already at the main conf.py.
So, instead, re-use the global latex_documents var, just
adjusting the path to be relative ones.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
Documentation/sphinx/load_config.py | 25 +++++++++++++++++++++++--
1 file changed, 23 insertions(+), 2 deletions(-)
diff --git a/Documentation/sphinx/load_config.py b/Documentation/sphinx/load_config.py
index 301a21aa4f63..75f527ff4c95 100644
--- a/Documentation/sphinx/load_config.py
+++ b/Documentation/sphinx/load_config.py
@@ -21,6 +21,29 @@ def loadConfig(namespace):
and os.path.normpath(namespace["__file__"]) != os.path.normpath(config_file) ):
config_file = os.path.abspath(config_file)
+ # Let's avoid one conf.py file just due to latex_documents
+ start = config_file.find('Documentation/')
+ if start >= 0:
+ start = config_file.find('/', start + 1)
+
+ end = config_file.rfind('/')
+ if start >= 0 and end > 0:
+ dir = config_file[start + 1:end]
+
+ print("source directory: %s" % dir)
+ new_latex_docs = []
+ latex_documents = namespace['latex_documents']
+
+ for l in latex_documents:
+ if l[0].find(dir) == 0:
+ has = True
+ fn = l[0][len(dir) + 1:]
+ new_latex_docs.append((fn, l[1], l[2], l[3], l[4]))
+ break
+
+ namespace['latex_documents'] = new_latex_docs
+
+ # If there is an extra conf.py file, load it
if os.path.isfile(config_file):
sys.stdout.write("load additional sphinx-config: %s\n" % config_file)
config = namespace.copy()
@@ -28,5 +51,3 @@ def loadConfig(namespace):
execfile_(config_file, config)
del config['__file__']
namespace.update(config)
- else:
- sys.stderr.write("WARNING: additional sphinx-config not found: %s\n" % config_file)
--
2.21.0
Right now, for every directory that we need to have LaTeX output,
a conf.py file is required.
That causes an extra overhead and it is actually a hack, as
the latex_documents line there are usually a copy of the ones
that are there already at the main conf.py.
So, instead, re-use the global latex_documents var, just
adjusting the path to be relative ones.
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
v2: make SPHINXDIRS="foo" htmldocs now works without needing a per-subdir
conf.py.
diff --git a/Documentation/sphinx/load_config.py b/Documentation/sphinx/load_config.py
index 301a21aa4f63..e4a04f367b41 100644
--- a/Documentation/sphinx/load_config.py
+++ b/Documentation/sphinx/load_config.py
@@ -21,6 +21,29 @@ def loadConfig(namespace):
and os.path.normpath(namespace["__file__"]) != os.path.normpath(config_file) ):
config_file = os.path.abspath(config_file)
+ # Let's avoid one conf.py file just due to latex_documents
+ start = config_file.find('Documentation/')
+ if start >= 0:
+ start = config_file.find('/', start + 1)
+
+ end = config_file.rfind('/')
+ if start >= 0 and end > 0:
+ dir = config_file[start + 1:end]
+
+ print("source directory: %s" % dir)
+ new_latex_docs = []
+ latex_documents = namespace['latex_documents']
+
+ for l in latex_documents:
+ if l[0].find(dir) == 0:
+ has = True
+ fn = l[0][len(dir) + 1:]
+ new_latex_docs.append((fn, l[1], l[2], l[3], l[4]))
+ break
+
+ namespace['latex_documents'] = new_latex_docs
+
+ # If there is an extra conf.py file, load it
if os.path.isfile(config_file):
sys.stdout.write("load additional sphinx-config: %s\n" % config_file)
config = namespace.copy()
@@ -29,4 +52,6 @@ def loadConfig(namespace):
del config['__file__']
namespace.update(config)
else:
- sys.stderr.write("WARNING: additional sphinx-config not found: %s\n" % config_file)
+ config = namespace.copy()
+ config['tags'].add("subproject")
+ namespace.update(config)