This patch set improves conversions of DOT -> PDF and SVG -> PDF
for PDF docs.
* DOT -> PDF conversion
Current scheme uses "dot -Tpdf" (of graphviz).
Cons:
- openSUSE's dot(1) does not support -Tpdf.
- Other distro's dot(1) generates PDFs with unnecessarily wide
margins for inclusion into LaTeX docs.
Patch 1/4 changes the route to the following two steps:
1. DOT -> SVG by "dot -Tsvg"
2. SVG -> PDF by "rsvg-convert -f pdf" with fallback to convert(1)
Pros:
- Improved portability across distros
- Less space around graphs in final PDF documents
Con:
- On systems without rsvg-convert, generated PDF will be of raster
image.
Patch 2/4 avoids raster-image PDF by using "dot -Tpdf" on systems where
the option is available.
* SVG -> PDF conversion
Current scheme uses convert(1) (of ImageMagick)
Cons:
- Generated PDFs are of raster image. Some of them look blurry.
- Raster images tend to be large in size.
- convert(1) delegates SVG decoding to rsvg-convert(1).
It doesn't cover full range of Inkscape-specific SVG features
and fails to convert some of SVG figures properly.
Improper conversions are observed with SVGs listed below (incomplete,
conversion quality depends on the version of rsvg-convert):
- Documentation/userspace-api/media/v4l/selection.svg
- Documentation/userspace-api/media/v4l/vbi_525.svg
- Documentation/userspace-api/media/v4l/vbi_625.svg
- Documentation/userspace-api/media/v4l/vbi_hsync.svg
- Documentation/admin-guide/blockdev/drbd/DRBD-8.3-data-packets.svg
- Documentation/admin-guide/blockdev/drbd/DRBD-data-packages.svg
If you have Inkscape installed as well, convert(1) delegates SVG
decoding to inkscape(1) rather than to rsvg-convert(1) and SVGs listed
above can be rendered properly.
So if Inkscape is required for converting those SVGs properly, why not
use it directly in the first place?
Patches 3/4 and 4/4 add code to utilize inkscape(1) for SVG -> PDF
conversion when it is available. They don't modify any existing
requirements for kernel-doc.
Patch 3/4 adds the alternative route of SVG -> PDF conversion by
inkscape(1).
Patch 4/4 delegates warning messages from inkscape(1) to kernellog.verbose
as they are likely harmless in command-line uses.
Pros:
- Generated PDFs are of vector graphics.
- Vector graphics tends to be smaller in size and looks nicer when
zoomed in.
- SVGs drawn by Inkscape are fully supported.
On systems without Inkscape, no regression is expected by these two
patches.
Changes since v1 (as of Patch 5/3) [1]:
- Reorder and merge patches to reduce/eliminate regression windows of
raster-image PDF and stderr redirection.
v1 v2
1/3 1/4
4/3 2/4
2/3 3/4
3/3+5/3 4/4
- Massage kernellog.verbose/warn messages. They now show command(s)
used in DOT -> PDF conversion.
- Pass actual exit code of inkscape(1) to kernellog.warn.
FWIW, diff of v1 vs. v2 follows:
--------------------------------------------------------------
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index a275ee0fec02..24d2b2addcce 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -220,10 +220,16 @@ def setupTools(app):
if rsvg_convert_cmd:
kernellog.verbose(app, "use rsvg-convert(1) from: " + rsvg_convert_cmd)
+ kernellog.verbose(app, "use 'dot -Tsvg' and rsvg-convert(1) for DOT -> PDF conversion")
dot_Tpdf = False
else:
- kernellog.verbose(app, "rsvg-convert(1) not found, "
- "falling back to raster image conversion")
+ kernellog.verbose(app,
+ "rsvg-convert(1) not found.\n"
+ " SVG rendering of convert(1) is done by ImageMagick-native renderer.")
+ if dot_Tpdf:
+ kernellog.verbose(app, "use 'dot -Tpdf' for DOT -> PDF conversion")
+ else:
+ kernellog.verbose(app, "use 'dot -Tsvg' and convert(1) for DOT -> PDF conversion")
# integrate conversion tools
@@ -368,8 +374,10 @@ def svg2pdf(app, svg_fname, pdf_fname):
"""
cmd = [convert_cmd, svg_fname, pdf_fname]
+ cmd_name = 'convert(1)'
if inkscape_cmd:
+ cmd_name = 'inkscape(1)'
if inkscape_ver_one:
cmd = [inkscape_cmd, '-o', pdf_fname, svg_fname]
else:
@@ -380,15 +388,17 @@ def svg2pdf(app, svg_fname, pdf_fname):
exit_code = 0
except subprocess.CalledProcessError as err:
warning_msg = err.output
- exit_code = 1
+ exit_code = err.returncode
pass
if exit_code != 0:
kernellog.warn(app, "Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
- kernellog.warn(app, "Warning msg from inkscape: %s" % str(warning_msg, 'utf-8'))
- if warning_msg:
- kernellog.verbose(app, "Warning msg from inkscape (likely harmless):\n%s"
- % str(warning_msg, 'utf-8'))
+ if warning_msg:
+ kernellog.warn(app, "Warning msg from %s: %s"
+ % (cmd_name, str(warning_msg, 'utf-8')))
+ elif warning_msg:
+ kernellog.verbose(app, "Warning msg from %s (likely harmless):\n%s"
+ % (cmd_name, str(warning_msg, 'utf-8')))
return bool(exit_code == 0)
--------------------------------------------------------------
[1] v1: https://lkml.kernel.org/r/[email protected]
Thanks, Akira
--
Akira Yokosawa (4):
docs: sphinx/kfigure.py: Use rsvg-convert(1) for DOT -> PDF conversion
docs: sphinx/kfigure.py: Add check of 'dot -Tpdf'
docs: sphinx/kfigure.py: Use inkscape(1) for SVG -> PDF conversion
docs: sphinx/kfigure.py: Delegate inkscape msg to kernellog.verbose
Documentation/sphinx/kfigure.py | 134 ++++++++++++++++++++++++++++----
1 file changed, 121 insertions(+), 13 deletions(-)
base-commit: b36064425a18e29a3bad9c007b4dd1223f8aadc5
--
2.17.1
On openSUSE, dot(1) command does not support direct PDF output.
On other distros, generated PDF images have unnecessarily wide margins,
especially for small graphs.
By using dot(1) for DOT -> SVG, then rsvg-convert(1) for SVG -> PDF,
more optimal PDF images can be obtained, with the bonus of improved
portability across various distros.
Add rules in kfigure.py so that the above mentioned route is taken
when rsvg-convert(1) is available.
Note that rsvg-convert(1) is recommended by sphinx_pre_install.
So it is most likely that existing systems for building pdfdocs have
rsvg-convert(1) installed.
Note:
SVG features supported by rsvg-convert(1) vary depending on its
version and distro config.
For example, the one found on Ubuntu Bionic (version 2.40.20) does
poor job in rendering some of SVG files drawn by Inkscape.
SVG files generated by dot(1) are converted nicely even with such
old versions of rsvg-convert.
So this change does not affect the quality of such figures in any
way.
Signed-off-by: Akira Yokosawa <[email protected]>
Cc: Jonathan Corbet <[email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>
---
Documentation/sphinx/kfigure.py | 46 +++++++++++++++++++++++++++++++--
1 file changed, 44 insertions(+), 2 deletions(-)
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 3c78828330be..955e3ec5de5a 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -31,6 +31,8 @@ u"""
* ``dot(1)``: Graphviz (https://www.graphviz.org). If Graphviz is not
available, the DOT language is inserted as literal-block.
+ For conversion to PDF, ``rsvg-convert(1)`` of librsvg
+ (https://gitlab.gnome.org/GNOME/librsvg) is used when available.
* SVG to PDF: To generate PDF, you need at least one of this tools:
@@ -113,6 +115,9 @@ dot_cmd = None
# ImageMagick' convert(1) support
convert_cmd = None
+# librsvg's rsvg-convert(1) support
+rsvg_convert_cmd = None
+
def setup(app):
# check toolchain first
@@ -160,11 +165,12 @@ def setupTools(app):
This function is called once, when the builder is initiated.
"""
- global dot_cmd, convert_cmd # pylint: disable=W0603
+ global dot_cmd, convert_cmd, rsvg_convert_cmd # pylint: disable=W0603
kernellog.verbose(app, "kfigure: check installed tools ...")
dot_cmd = which('dot')
convert_cmd = which('convert')
+ rsvg_convert_cmd = which('rsvg-convert')
if dot_cmd:
kernellog.verbose(app, "use dot(1) from: " + dot_cmd)
@@ -177,6 +183,11 @@ def setupTools(app):
kernellog.warn(app,
"convert(1) not found, for SVG to PDF conversion install "
"ImageMagick (https://www.imagemagick.org)")
+ if rsvg_convert_cmd:
+ kernellog.verbose(app, "use rsvg-convert(1) from: " + rsvg_convert_cmd)
+ else:
+ kernellog.verbose(app, "rsvg-convert(1) not found, "
+ "falling back to raster image conversion")
# integrate conversion tools
@@ -266,7 +277,13 @@ def convert_image(img_node, translator, src_fname=None):
if in_ext == '.dot':
kernellog.verbose(app, 'convert DOT to: {out}/' + _name)
- ok = dot2format(app, src_fname, dst_fname)
+ if translator.builder.format == 'latex':
+ svg_fname = path.join(translator.builder.outdir, fname + '.svg')
+ ok1 = dot2format(app, src_fname, svg_fname)
+ ok2 = svg2pdf_by_rsvg(app, svg_fname, dst_fname)
+ ok = ok1 and ok2
+ else:
+ ok = dot2format(app, src_fname, dst_fname)
elif in_ext == '.svg':
kernellog.verbose(app, 'convert SVG to: {out}/' + _name)
@@ -319,6 +336,31 @@ def svg2pdf(app, svg_fname, pdf_fname):
kernellog.warn(app, "Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
return bool(exit_code == 0)
+def svg2pdf_by_rsvg(app, svg_fname, pdf_fname):
+ """Convert SVG to PDF with ``rsvg-convert(1)`` command.
+
+ * ``svg_fname`` pathname of input SVG file, including extension ``.svg``
+ * ``pdf_fname`` pathname of output PDF file, including extension ``.pdf``
+
+ Input SVG file should be the one generated by ``dot2format()``.
+ SVG -> PDF conversion is done by ``rsvg-convert(1)``.
+
+ If ``rsvg-convert(1)`` is unavailable, fall back to ``svg2pdf()``.
+
+ """
+
+ if rsvg_convert_cmd is None:
+ ok = svg2pdf(app, svg_fname, pdf_fname)
+ else:
+ cmd = [rsvg_convert_cmd, '--format=pdf', '-o', pdf_fname, svg_fname]
+ # use stdout and stderr from parent
+ exit_code = subprocess.call(cmd)
+ if exit_code != 0:
+ kernellog.warn(app, "Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
+ ok = bool(exit_code == 0)
+
+ return ok
+
# image handling
# ---------------------
--
2.17.1
To prevent any regression on existing build systems, limit the
fallback of converting DOT -> raster PDF only when both of the
following conditions are met.
o dot(1) doesn't support -Tpdf
o rsvg-convert(1) is not found
While we are here, add kernellog.verbose messages related to
rsvg-convert, 'dot -Tpdf', and 'dot -Tsvg' commands.
Suggested-by: Mauro Carvalho Chehab <[email protected]>
Signed-off-by: Akira Yokosawa <[email protected]>
Cc: Jonathan Corbet <[email protected]>
---
Documentation/sphinx/kfigure.py | 30 ++++++++++++++++++++++++++----
1 file changed, 26 insertions(+), 4 deletions(-)
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 955e3ec5de5a..77b0d15dba31 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -51,6 +51,7 @@ import os
from os import path
import subprocess
from hashlib import sha1
+import re
from docutils import nodes
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives
@@ -111,6 +112,8 @@ def pass_handle(self, node): # pylint: disable=W0613
# Graphviz's dot(1) support
dot_cmd = None
+# dot(1) -Tpdf should be used
+dot_Tpdf = False
# ImageMagick' convert(1) support
convert_cmd = None
@@ -165,7 +168,7 @@ def setupTools(app):
This function is called once, when the builder is initiated.
"""
- global dot_cmd, convert_cmd, rsvg_convert_cmd # pylint: disable=W0603
+ global dot_cmd, dot_Tpdf, convert_cmd, rsvg_convert_cmd # pylint: disable=W0603
kernellog.verbose(app, "kfigure: check installed tools ...")
dot_cmd = which('dot')
@@ -174,6 +177,16 @@ def setupTools(app):
if dot_cmd:
kernellog.verbose(app, "use dot(1) from: " + dot_cmd)
+
+ try:
+ dot_Thelp_list = subprocess.check_output([dot_cmd, '-Thelp'],
+ stderr=subprocess.STDOUT)
+ except subprocess.CalledProcessError as err:
+ dot_Thelp_list = err.output
+ pass
+
+ dot_Tpdf_ptn = b'pdf'
+ dot_Tpdf = re.search(dot_Tpdf_ptn, dot_Thelp_list)
else:
kernellog.warn(app, "dot(1) not found, for better output quality install "
"graphviz from https://www.graphviz.org")
@@ -185,9 +198,17 @@ def setupTools(app):
"ImageMagick (https://www.imagemagick.org)")
if rsvg_convert_cmd:
kernellog.verbose(app, "use rsvg-convert(1) from: " + rsvg_convert_cmd)
+ kernellog.verbose(app, "use 'dot -Tsvg' and rsvg-convert(1) for DOT -> PDF conversion")
+ dot_Tpdf = False
else:
- kernellog.verbose(app, "rsvg-convert(1) not found, "
- "falling back to raster image conversion")
+ kernellog.verbose(app,
+ "rsvg-convert(1) not found.\n"
+ " SVG -> PDF conversion by convert() can be poor quality.\n"
+ " Install librsvg (https://gitlab.gnome.org/GNOME/librsvg)")
+ if dot_Tpdf:
+ kernellog.verbose(app, "use 'dot -Tpdf' for DOT -> PDF conversion")
+ else:
+ kernellog.verbose(app, "use 'dot -Tsvg' and convert(1) for DOT -> PDF conversion")
# integrate conversion tools
@@ -277,11 +298,12 @@ def convert_image(img_node, translator, src_fname=None):
if in_ext == '.dot':
kernellog.verbose(app, 'convert DOT to: {out}/' + _name)
- if translator.builder.format == 'latex':
+ if translator.builder.format == 'latex' and not dot_Tpdf:
svg_fname = path.join(translator.builder.outdir, fname + '.svg')
ok1 = dot2format(app, src_fname, svg_fname)
ok2 = svg2pdf_by_rsvg(app, svg_fname, dst_fname)
ok = ok1 and ok2
+
else:
ok = dot2format(app, src_fname, dst_fname)
--
2.17.1
Using convert(1) of ImageMagick for SVG -> PDF conversion results in
PDFs containing raster (bitmap) images which sometimes look blurry.
Ideally speaking, SVG to PDF conversion should retain vector graphics
in SVG.
rsvg-convert(1) can do such conversions with regard to SVG files
generated by dot(1).
Unfortunately, rsvg-convert(1) does not cover some of SVG features
specific to Inkscape.
inkscape(1) of Inkscape naturally covers such SVG features.
So add a route in svg2pdf() so that inkscape(1) is used when it is
available.
Note:
After this change, if you have Inkscape installed, ImageMagick nor
librsvg are not required.
Signed-off-by: Akira Yokosawa <[email protected]>
Cc: Jonathan Corbet <[email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>
---
Documentation/sphinx/kfigure.py | 68 +++++++++++++++++++++++----------
1 file changed, 48 insertions(+), 20 deletions(-)
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 77b0d15dba31..e616e49669eb 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -37,6 +37,7 @@ u"""
* SVG to PDF: To generate PDF, you need at least one of this tools:
- ``convert(1)``: ImageMagick (https://www.imagemagick.org)
+ - ``inkscape(1)``: Inkscape (https://inkscape.org/)
List of customizations:
@@ -121,6 +122,11 @@ convert_cmd = None
# librsvg's rsvg-convert(1) support
rsvg_convert_cmd = None
+# Inkscape's inkscape(1) support
+inkscape_cmd = None
+# Inkscape prior to 1.0 uses different command options
+inkscape_ver_one = False
+
def setup(app):
# check toolchain first
@@ -169,11 +175,13 @@ def setupTools(app):
This function is called once, when the builder is initiated.
"""
global dot_cmd, dot_Tpdf, convert_cmd, rsvg_convert_cmd # pylint: disable=W0603
+ global inkscape_cmd, inkscape_ver_one # pylint: disable=W0603
kernellog.verbose(app, "kfigure: check installed tools ...")
dot_cmd = which('dot')
convert_cmd = which('convert')
rsvg_convert_cmd = which('rsvg-convert')
+ inkscape_cmd = which('inkscape')
if dot_cmd:
kernellog.verbose(app, "use dot(1) from: " + dot_cmd)
@@ -190,25 +198,37 @@ def setupTools(app):
else:
kernellog.warn(app, "dot(1) not found, for better output quality install "
"graphviz from https://www.graphviz.org")
- if convert_cmd:
- kernellog.verbose(app, "use convert(1) from: " + convert_cmd)
- else:
- kernellog.warn(app,
- "convert(1) not found, for SVG to PDF conversion install "
- "ImageMagick (https://www.imagemagick.org)")
- if rsvg_convert_cmd:
- kernellog.verbose(app, "use rsvg-convert(1) from: " + rsvg_convert_cmd)
- kernellog.verbose(app, "use 'dot -Tsvg' and rsvg-convert(1) for DOT -> PDF conversion")
+ if inkscape_cmd:
+ kernellog.verbose(app, "use inkscape(1) from: " + inkscape_cmd)
+ inkscape_ver = subprocess.check_output([inkscape_cmd, '--version'])
+ ver_one_ptn = b'Inkscape 1'
+ inkscape_ver_one = re.search(ver_one_ptn, inkscape_ver)
+ convert_cmd = None
+ rsvg_convert_cmd = None
dot_Tpdf = False
+
else:
- kernellog.verbose(app,
- "rsvg-convert(1) not found.\n"
- " SVG -> PDF conversion by convert() can be poor quality.\n"
- " Install librsvg (https://gitlab.gnome.org/GNOME/librsvg)")
- if dot_Tpdf:
- kernellog.verbose(app, "use 'dot -Tpdf' for DOT -> PDF conversion")
+ if convert_cmd:
+ kernellog.verbose(app, "use convert(1) from: " + convert_cmd)
+ else:
+ kernellog.warn(app,
+ "Neither inkscape(1) nor convert(1) found.\n"
+ "For SVG to PDF conversion, "
+ "install either Inkscape (https://inkscape.org/) (preferred) or\n"
+ "ImageMagick (https://www.imagemagick.org)")
+
+ if rsvg_convert_cmd:
+ kernellog.verbose(app, "use rsvg-convert(1) from: " + rsvg_convert_cmd)
+ kernellog.verbose(app, "use 'dot -Tsvg' and rsvg-convert(1) for DOT -> PDF conversion")
+ dot_Tpdf = False
else:
- kernellog.verbose(app, "use 'dot -Tsvg' and convert(1) for DOT -> PDF conversion")
+ kernellog.verbose(app,
+ "rsvg-convert(1) not found.\n"
+ " SVG rendering of convert(1) is done by ImageMagick-native renderer.")
+ if dot_Tpdf:
+ kernellog.verbose(app, "use 'dot -Tpdf' for DOT -> PDF conversion")
+ else:
+ kernellog.verbose(app, "use 'dot -Tsvg' and convert(1) for DOT -> PDF conversion")
# integrate conversion tools
@@ -274,7 +294,7 @@ def convert_image(img_node, translator, src_fname=None):
elif in_ext == '.svg':
if translator.builder.format == 'latex':
- if convert_cmd is None:
+ if not inkscape_cmd and convert_cmd is None:
kernellog.verbose(app,
"no SVG to PDF conversion available / include SVG raw.")
img_node.replace_self(file2literal(src_fname))
@@ -342,16 +362,24 @@ def dot2format(app, dot_fname, out_fname):
return bool(exit_code == 0)
def svg2pdf(app, svg_fname, pdf_fname):
- """Converts SVG to PDF with ``convert(1)`` command.
+ """Converts SVG to PDF with ``inkscape(1)`` or ``convert(1)`` command.
- Uses ``convert(1)`` from ImageMagick (https://www.imagemagick.org) for
- conversion. Returns ``True`` on success and ``False`` if an error occurred.
+ Uses ``inkscape(1)`` from Inkscape (https://inkscape.org/) or ``convert(1)``
+ from ImageMagick (https://www.imagemagick.org) for conversion.
+ Returns ``True`` on success and ``False`` if an error occurred.
* ``svg_fname`` pathname of the input SVG file with extension (``.svg``)
* ``pdf_name`` pathname of the output PDF file with extension (``.pdf``)
"""
cmd = [convert_cmd, svg_fname, pdf_fname]
+
+ if inkscape_cmd:
+ if inkscape_ver_one:
+ cmd = [inkscape_cmd, '-o', pdf_fname, svg_fname]
+ else:
+ cmd = [inkscape_cmd, '-z', '--export-pdf=%s' % pdf_fname, svg_fname]
+
# use stdout and stderr from parent
exit_code = subprocess.call(cmd)
if exit_code != 0:
--
2.17.1
Subject: [PATCH v2 4/4] docs: sphinx/kfigure.py: Delegate inkscape msg to kernellog.verbose
Depending on its version, distro config, and system-setup type,
inkscape(1) emits various warning messages which are harmless in
command-line uses.
List of such warning messages (incomplete, long ones wrapped):
- Gtk-Message: hh:mm:ss.nnn: Failed to load module "canberra-gtk-module"
- Unable to init server: Could not connect: Connection refused
- Failed to get connection
- ** (inkscape:xxx): CRITICAL **: hh:mm:ss.nnn: dbus_g_proxy_new_for_name:
assertion 'connection != NULL' failed
- ** (inkscape:xxx): CRITICAL **: hh:mm:ss.nnn: dbus_g_proxy_call:
assertion 'DBUS_IS_G_PROXY (proxy)' failed
- ** (inkscape:xxx): CRITICAL **: hh:mm:ss.nnn: dbus_g_connection_register_g_object:
assertion 'connection != NULL' failed
- ** (inkscape:xxx): WARNING **: hh:mm:ss.nnn:
Fonts dir '/usr/share/inkscape/fonts' does not exist and will be ignored.
To avoid unnecessary anxiety, capture the message and output it via
kernellog.verbose or kernellog.warn depending on the exit code.
Signed-off-by: Akira Yokosawa <[email protected]>
Cc: Jonathan Corbet <[email protected]>
Cc: Mauro Carvalho Chehab <[email protected]>
Cc: Randy Dunlap <[email protected]>
---
Documentation/sphinx/kfigure.py | 22 +++++++++++++++++++---
1 file changed, 19 insertions(+), 3 deletions(-)
diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index e616e49669eb..24d2b2addcce 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -200,7 +200,8 @@ def setupTools(app):
"graphviz from https://www.graphviz.org")
if inkscape_cmd:
kernellog.verbose(app, "use inkscape(1) from: " + inkscape_cmd)
- inkscape_ver = subprocess.check_output([inkscape_cmd, '--version'])
+ inkscape_ver = subprocess.check_output([inkscape_cmd, '--version'],
+ stderr=subprocess.DEVNULL)
ver_one_ptn = b'Inkscape 1'
inkscape_ver_one = re.search(ver_one_ptn, inkscape_ver)
convert_cmd = None
@@ -373,17 +374,32 @@ def svg2pdf(app, svg_fname, pdf_fname):
"""
cmd = [convert_cmd, svg_fname, pdf_fname]
+ cmd_name = 'convert(1)'
if inkscape_cmd:
+ cmd_name = 'inkscape(1)'
if inkscape_ver_one:
cmd = [inkscape_cmd, '-o', pdf_fname, svg_fname]
else:
cmd = [inkscape_cmd, '-z', '--export-pdf=%s' % pdf_fname, svg_fname]
- # use stdout and stderr from parent
- exit_code = subprocess.call(cmd)
+ try:
+ warning_msg = subprocess.check_output(cmd, stderr=subprocess.STDOUT)
+ exit_code = 0
+ except subprocess.CalledProcessError as err:
+ warning_msg = err.output
+ exit_code = err.returncode
+ pass
+
if exit_code != 0:
kernellog.warn(app, "Error #%d when calling: %s" % (exit_code, " ".join(cmd)))
+ if warning_msg:
+ kernellog.warn(app, "Warning msg from %s: %s"
+ % (cmd_name, str(warning_msg, 'utf-8')))
+ elif warning_msg:
+ kernellog.verbose(app, "Warning msg from %s (likely harmless):\n%s"
+ % (cmd_name, str(warning_msg, 'utf-8')))
+
return bool(exit_code == 0)
def svg2pdf_by_rsvg(app, svg_fname, pdf_fname):
--
2.17.1
On Wed, 29 Dec 2021 20:42:00 +0900, Akira Yokosawa wrote:
> This patch set improves conversions of DOT -> PDF and SVG -> PDF
> for PDF docs.
Gentle ping.
Mauro, any comments?
>
> * DOT -> PDF conversion
>
> Current scheme uses "dot -Tpdf" (of graphviz).
>
> Cons:
> - openSUSE's dot(1) does not support -Tpdf.
> - Other distro's dot(1) generates PDFs with unnecessarily wide
> margins for inclusion into LaTeX docs.
>
> Patch 1/4 changes the route to the following two steps:
>
> 1. DOT -> SVG by "dot -Tsvg"
> 2. SVG -> PDF by "rsvg-convert -f pdf" with fallback to convert(1)
>
> Pros:
> - Improved portability across distros
> - Less space around graphs in final PDF documents
>
> Con:
> - On systems without rsvg-convert, generated PDF will be of raster
> image.
>
> Patch 2/4 avoids raster-image PDF by using "dot -Tpdf" on systems where
> the option is available.
>
> * SVG -> PDF conversion
>
> Current scheme uses convert(1) (of ImageMagick)
I was not aware of security concerns regarding ImageMagick until
Christoph brought them up in another thread [1].
[1]: https://lore.kernel.org/linux-doc/[email protected]/
Now I can add another Con as bellow.
>
> Cons:
- ImageMagick is not allowed to read/write PDF by default under
Debian/Ubuntu and Gentoo systems. The policy is a band-aide
fix to its security issues.
> - Generated PDFs are of raster image. Some of them look blurry.
> - Raster images tend to be large in size.
> - convert(1) delegates SVG decoding to rsvg-convert(1).
> It doesn't cover full range of Inkscape-specific SVG features
> and fails to convert some of SVG figures properly.
Thanks, Akira
>
> Improper conversions are observed with SVGs listed below (incomplete,
> conversion quality depends on the version of rsvg-convert):
> - Documentation/userspace-api/media/v4l/selection.svg
> - Documentation/userspace-api/media/v4l/vbi_525.svg
> - Documentation/userspace-api/media/v4l/vbi_625.svg
> - Documentation/userspace-api/media/v4l/vbi_hsync.svg
> - Documentation/admin-guide/blockdev/drbd/DRBD-8.3-data-packets.svg
> - Documentation/admin-guide/blockdev/drbd/DRBD-data-packages.svg
>
> If you have Inkscape installed as well, convert(1) delegates SVG
> decoding to inkscape(1) rather than to rsvg-convert(1) and SVGs listed
> above can be rendered properly.
>
> So if Inkscape is required for converting those SVGs properly, why not
> use it directly in the first place?
>
> Patches 3/4 and 4/4 add code to utilize inkscape(1) for SVG -> PDF
> conversion when it is available. They don't modify any existing
> requirements for kernel-doc.
>
> Patch 3/4 adds the alternative route of SVG -> PDF conversion by
> inkscape(1).
> Patch 4/4 delegates warning messages from inkscape(1) to kernellog.verbose
> as they are likely harmless in command-line uses.
>
> Pros:
> - Generated PDFs are of vector graphics.
> - Vector graphics tends to be smaller in size and looks nicer when
> zoomed in.
> - SVGs drawn by Inkscape are fully supported.
>
> On systems without Inkscape, no regression is expected by these two
> patches.
>
> Changes since v1 (as of Patch 5/3) [1]:
>
> - Reorder and merge patches to reduce/eliminate regression windows of
> raster-image PDF and stderr redirection.
> v1 v2
> 1/3 1/4
> 4/3 2/4
> 2/3 3/4
> 3/3+5/3 4/4
>
> - Massage kernellog.verbose/warn messages. They now show command(s)
> used in DOT -> PDF conversion.
>
> - Pass actual exit code of inkscape(1) to kernellog.warn.
>
> FWIW, diff of v1 vs. v2 follows:
>
> --------------------------------------------------------------
[...]
Em Fri, 7 Jan 2022 22:45:47 +0900
Akira Yokosawa <[email protected]> escreveu:
> On Wed, 29 Dec 2021 20:42:00 +0900, Akira Yokosawa wrote:
> > This patch set improves conversions of DOT -> PDF and SVG -> PDF
> > for PDF docs.
>
> Gentle ping.
>
> Mauro, any comments?
Sorry, have been busy those days with not much time to test it,
and I'm not expecting any time to test it on the next couple of
weeks.
The main concern from my last review is that inkscape is too noisy
(well, frankly, textlive is also too noisy). If this was solved
on a nice way, and provided that the output files on both html and
pdf are working fine with those conversions, I don't have any
objections to this series.
Regards,
Mauro
>
> >
> > * DOT -> PDF conversion
> >
> > Current scheme uses "dot -Tpdf" (of graphviz).
> >
> > Cons:
> > - openSUSE's dot(1) does not support -Tpdf.
> > - Other distro's dot(1) generates PDFs with unnecessarily wide
> > margins for inclusion into LaTeX docs.
> >
> > Patch 1/4 changes the route to the following two steps:
> >
> > 1. DOT -> SVG by "dot -Tsvg"
> > 2. SVG -> PDF by "rsvg-convert -f pdf" with fallback to convert(1)
> >
> > Pros:
> > - Improved portability across distros
> > - Less space around graphs in final PDF documents
> >
> > Con:
> > - On systems without rsvg-convert, generated PDF will be of raster
> > image.
> >
> > Patch 2/4 avoids raster-image PDF by using "dot -Tpdf" on systems where
> > the option is available.
> >
> > * SVG -> PDF conversion
> >
> > Current scheme uses convert(1) (of ImageMagick)
>
> I was not aware of security concerns regarding ImageMagick until
> Christoph brought them up in another thread [1].
>
> [1]: https://lore.kernel.org/linux-doc/[email protected]/
>
> Now I can add another Con as bellow.
>
> >
> > Cons:
> - ImageMagick is not allowed to read/write PDF by default under
> Debian/Ubuntu and Gentoo systems. The policy is a band-aide
> fix to its security issues.
> > - Generated PDFs are of raster image. Some of them look blurry.
> > - Raster images tend to be large in size.
> > - convert(1) delegates SVG decoding to rsvg-convert(1).
> > It doesn't cover full range of Inkscape-specific SVG features
> > and fails to convert some of SVG figures properly.
>
> Thanks, Akira
>
> >
> > Improper conversions are observed with SVGs listed below (incomplete,
> > conversion quality depends on the version of rsvg-convert):
> > - Documentation/userspace-api/media/v4l/selection.svg
> > - Documentation/userspace-api/media/v4l/vbi_525.svg
> > - Documentation/userspace-api/media/v4l/vbi_625.svg
> > - Documentation/userspace-api/media/v4l/vbi_hsync.svg
> > - Documentation/admin-guide/blockdev/drbd/DRBD-8.3-data-packets.svg
> > - Documentation/admin-guide/blockdev/drbd/DRBD-data-packages.svg
> >
> > If you have Inkscape installed as well, convert(1) delegates SVG
> > decoding to inkscape(1) rather than to rsvg-convert(1) and SVGs listed
> > above can be rendered properly.
> >
> > So if Inkscape is required for converting those SVGs properly, why not
> > use it directly in the first place?
> >
> > Patches 3/4 and 4/4 add code to utilize inkscape(1) for SVG -> PDF
> > conversion when it is available. They don't modify any existing
> > requirements for kernel-doc.
> >
> > Patch 3/4 adds the alternative route of SVG -> PDF conversion by
> > inkscape(1).
> > Patch 4/4 delegates warning messages from inkscape(1) to kernellog.verbose
> > as they are likely harmless in command-line uses.
> >
> > Pros:
> > - Generated PDFs are of vector graphics.
> > - Vector graphics tends to be smaller in size and looks nicer when
> > zoomed in.
> > - SVGs drawn by Inkscape are fully supported.
> >
> > On systems without Inkscape, no regression is expected by these two
> > patches.
> >
> > Changes since v1 (as of Patch 5/3) [1]:
> >
> > - Reorder and merge patches to reduce/eliminate regression windows of
> > raster-image PDF and stderr redirection.
> > v1 v2
> > 1/3 1/4
> > 4/3 2/4
> > 2/3 3/4
> > 3/3+5/3 4/4
> >
> > - Massage kernellog.verbose/warn messages. They now show command(s)
> > used in DOT -> PDF conversion.
> >
> > - Pass actual exit code of inkscape(1) to kernellog.warn.
> >
> > FWIW, diff of v1 vs. v2 follows:
> >
> > --------------------------------------------------------------
> [...]
>
Thanks,
Mauro
On Fri, 14 Jan 2022 09:45:35 +0100,
Mauro Carvalho Chehab wrote:
> Em Fri, 7 Jan 2022 22:45:47 +0900
> Akira Yokosawa <[email protected]> escreveu:
>
>> On Wed, 29 Dec 2021 20:42:00 +0900, Akira Yokosawa wrote:
>>> This patch set improves conversions of DOT -> PDF and SVG -> PDF
>>> for PDF docs.
>>
>> Gentle ping.
>>
>> Mauro, any comments?
>
> Sorry, have been busy those days with not much time to test it,
> and I'm not expecting any time to test it on the next couple of
> weeks.
Mauro, no need of apologies.
We are in the middle of the v5.17 merge window, and I think of this
series as a v5.18 material.
Which means it won't be merged into doc-next until v5.17-rc5 or -rc6
(mid March or so), unless Jon thinks otherwise.
I'd like to have your Tested-by: and/or Reviewed-by: tags if you
could manage to spare time for testing.
>
> The main concern from my last review is that inkscape is too noisy
> (well, frankly, textlive is also too noisy).
You mean the harmless warning msgs delegated to kernellog.verbose()?
Or the direct redirection to /dev/null as of v1's 3/3?
> If this was solved
> on a nice way,
An excerpt of messages from Inkscape is as follows
(on Debian bullseye, with "make SPHINXOPTS=-v SPHINXDIRS=doc-guide pdfdocs"):
----------
convert SVG to: {out}/svg_image.pdf
Warning msg from inkscape(1) (likely harmless):
Unable to init server: Could not connect: Connection refused
Failed to get connection
** (inkscape:119): CRITICAL **: 01:58:03.988: dbus_g_proxy_new_for_name: assertion 'connection != NULL' failed
** (inkscape:119): CRITICAL **: 01:58:03.988: dbus_g_proxy_call: assertion 'DBUS_IS_G_PROXY (proxy)' failed
** (inkscape:119): CRITICAL **: 01:58:03.988: dbus_g_connection_register_g_object: assertion 'connection != NULL' failed
** (inkscape:119): WARNING **: 01:58:04.300: Fonts dir '/usr/share/inkscape/fonts' does not exist and will be ignored.
assert best format for: hello.dot
convert DOT to: {out}/hello.pdf
Warning msg from inkscape(1) (likely harmless):
Unable to init server: Could not connect: Connection refused
Failed to get connection
** (inkscape:129): CRITICAL **: 01:58:04.454: dbus_g_proxy_new_for_name: assertion 'connection != NULL' failed
** (inkscape:129): CRITICAL **: 01:58:04.454: dbus_g_proxy_call: assertion 'DBUS_IS_G_PROXY (proxy)' failed
** (inkscape:129): CRITICAL **: 01:58:04.454: dbus_g_connection_register_g_object: assertion 'connection != NULL' failed
** (inkscape:129): WARNING **: 01:58:04.628: Fonts dir '/usr/share/inkscape/fonts' does not exist and will be ignored.
[...]
----------
On Fedora 35, I don't see any message from Inkscape.
Is this acceptable to you?
> and provided that the output files on both html and
> pdf are working fine with those conversions, I don't have any
> objections to this series.
This series affects only PDF.
My test coverage is not perfect, but I don't expect any regression
in "make pdfdocs" or "make htmldocs".
Thanks, Akira
>
> Regards,
> Mauro
>
[...]
Akira Yokosawa <[email protected]> writes:
> We are in the middle of the v5.17 merge window, and I think of this
> series as a v5.18 material.
> Which means it won't be merged into doc-next until v5.17-rc5 or -rc6
> (mid March or so), unless Jon thinks otherwise.
I'd rather merge it rather sooner than that if possible; 5.18 stuff will
start going into docs-next shortly.
Thanks,
jon
On Sat, 15 Jan 2022 14:17:30 -0700,
Jonathan Corbet wrote:
> Akira Yokosawa <[email protected]> writes:
>
>> We are in the middle of the v5.17 merge window, and I think of this
>> series as a v5.18 material.
>> Which means it won't be merged into doc-next until v5.17-rc5 or -rc6
>> (mid March or so), unless Jon thinks otherwise.
>
> I'd rather merge it rather sooner than that if possible; 5.18 stuff will
> start going into docs-next shortly.
Jon,
Hearing no objection since mid January, I think it is time for you to
merge this series into docs-next.
Thanks, Akira
>
> Thanks,
>
> jon
Akira Yokosawa <[email protected]> writes:
> On Sat, 15 Jan 2022 14:17:30 -0700,
> Jonathan Corbet wrote:
>> Akira Yokosawa <[email protected]> writes:
>>
>>> We are in the middle of the v5.17 merge window, and I think of this
>>> series as a v5.18 material.
>>> Which means it won't be merged into doc-next until v5.17-rc5 or -rc6
>>> (mid March or so), unless Jon thinks otherwise.
>>
>> I'd rather merge it rather sooner than that if possible; 5.18 stuff will
>> start going into docs-next shortly.
>
> Jon,
>
> Hearing no objection since mid January, I think it is time for you to
> merge this series into docs-next.
That has been done - thanks for your work on this!
Since you seem to have a reasonably good understanding of the
PDF-generation side of things...I noted as I was testing this work that
we are still getting the "restricted \write18 enabled" messages. I
asked about this years ago, and nobody seemed to have a clue of what was
enabling it. I'd *really* like to turn that off if possible; do you
have any idea what's going on there?
Thanks,
jon
On Wed, 09 Feb 2022 17:30:38 -0700,
Jonathan Corbet wrote:
> Akira Yokosawa <[email protected]> writes:
>
>> On Sat, 15 Jan 2022 14:17:30 -0700,
>> Jonathan Corbet wrote:
>>> Akira Yokosawa <[email protected]> writes:
>>>
>>>> We are in the middle of the v5.17 merge window, and I think of this
>>>> series as a v5.18 material.
>>>> Which means it won't be merged into doc-next until v5.17-rc5 or -rc6
>>>> (mid March or so), unless Jon thinks otherwise.
>>>
>>> I'd rather merge it rather sooner than that if possible; 5.18 stuff will
>>> start going into docs-next shortly.
>>
>> Jon,
>>
>> Hearing no objection since mid January, I think it is time for you to
>> merge this series into docs-next.
>
> That has been done - thanks for your work on this!
You are welcome!
>
> Since you seem to have a reasonably good understanding of the
> PDF-generation side of things...I noted as I was testing this work that
> we are still getting the "restricted \write18 enabled" messages. I
> asked about this years ago, and nobody seemed to have a clue of what was
> enabling it. I'd *really* like to turn that off if possible; do you
> have any idea what's going on there?
I think I know the answer.
The message can be silenced by adding an option to xelatex.
Will send a patch soon.
Thanks, Akira
>
> Thanks,
>
> jon