LinuxLists.cc - [PATCH 00/14] Add support to generate ABI documentation at admin-guide

[permalink] [raw]

Subject: [PATCH 01/14] ABI: fix some syntax issues at the ABI database

From: Mauro Carvalho Chehab <[email protected]>

On those three files, the ABI representation described at
README are violated.

- at sysfs-bus-iio-proximity-as3935:
a ':' character is missing after "What"

- at sysfs-class-devfreq:
there's a typo at Description

- at sysfs-class-cxl, it is using the ":" character at a
file preamble, causing it to be misinterpreted as a
tag.

- On the other files, instead of "What", they use "Where".

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
Documentation/ABI/testing/pstore | 2 +-
.../sysfs-bus-event_source-devices-format | 2 +-
.../ABI/testing/sysfs-bus-i2c-devices-hm6352 | 6 ++---
.../ABI/testing/sysfs-bus-iio-distance-srf08 | 4 ++--
.../testing/sysfs-bus-iio-proximity-as3935 | 4 ++--
.../ABI/testing/sysfs-bus-pci-devices-cciss | 22 +++++++++----------
.../testing/sysfs-bus-usb-devices-usbsevseg | 12 +++++-----
Documentation/ABI/testing/sysfs-class-cxl | 6 ++---
Documentation/ABI/testing/sysfs-class-devfreq | 2 +-
.../ABI/testing/sysfs-class-powercap | 2 +-
Documentation/ABI/testing/sysfs-kernel-fscaps | 2 +-
.../ABI/testing/sysfs-kernel-vmcoreinfo | 2 +-
12 files changed, 33 insertions(+), 33 deletions(-)

diff --git a/Documentation/ABI/testing/pstore b/Documentation/ABI/testing/pstore
index 5fca9f5e10a3..8d6e48f4e8ef 100644
--- a/Documentation/ABI/testing/pstore
+++ b/Documentation/ABI/testing/pstore
@@ -1,4 +1,4 @@
-Where: /sys/fs/pstore/... (or /dev/pstore/...)
+What: /sys/fs/pstore/... (or /dev/pstore/...)
Date: March 2011
Kernel Version: 2.6.39
Contact: [email protected]
diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-format b/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
index 77f47ff5ee02..b6f8748e0200 100644
--- a/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
+++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
@@ -1,4 +1,4 @@
-Where: /sys/bus/event_source/devices/<dev>/format
+What: /sys/bus/event_source/devices/<dev>/format
Date: January 2012
Kernel Version: 3.3
Contact: Jiri Olsa <[email protected]>
diff --git a/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 b/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
index feb2e4a87075..29bd447e50a0 100644
--- a/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
+++ b/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
@@ -1,18 +1,18 @@
-Where: /sys/bus/i2c/devices/.../heading0_input
+What: /sys/bus/i2c/devices/.../heading0_input
Date: April 2010
Kernel Version: 2.6.36?
Contact: [email protected]
Description: Reports the current heading from the compass as a floating
point value in degrees.

-Where: /sys/bus/i2c/devices/.../power_state
+What: /sys/bus/i2c/devices/.../power_state
Date: April 2010
Kernel Version: 2.6.36?
Contact: [email protected]
Description: Sets the power state of the device. 0 sets the device into
sleep mode, 1 wakes it up.

-Where: /sys/bus/i2c/devices/.../calibration
+What: /sys/bus/i2c/devices/.../calibration
Date: April 2010
Kernel Version: 2.6.36?
Contact: [email protected]
diff --git a/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08 b/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
index 0a1ca1487fa9..a133fd8d081a 100644
--- a/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
+++ b/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
@@ -1,4 +1,4 @@
-What /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
+What: /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
Date: January 2017
KernelVersion: 4.11
Contact: [email protected]
@@ -6,7 +6,7 @@ Description:
Show or set the gain boost of the amp, from 0-31 range.
default 31

-What /sys/bus/iio/devices/iio:deviceX/sensor_max_range
+What: /sys/bus/iio/devices/iio:deviceX/sensor_max_range
Date: January 2017
KernelVersion: 4.11
Contact: [email protected]
diff --git a/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 b/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
index 9a17ab5036a4..c59d95346341 100644
--- a/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
+++ b/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
@@ -1,4 +1,4 @@
-What /sys/bus/iio/devices/iio:deviceX/in_proximity_input
+What: /sys/bus/iio/devices/iio:deviceX/in_proximity_input
Date: March 2014
KernelVersion: 3.15
Contact: Matt Ranostay <[email protected]>
@@ -6,7 +6,7 @@ Description:
Get the current distance in meters of storm (1km steps)
1000-40000 = distance in meters

-What /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
+What: /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
Date: March 2014
KernelVersion: 3.15
Contact: Matt Ranostay <[email protected]>
diff --git a/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss b/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
index 53d99edd1d75..eb449169c30b 100644
--- a/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
+++ b/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
@@ -1,66 +1,66 @@
-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/model
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/model
Date: March 2009
Kernel Version: 2.6.30
Contact: [email protected]
Description: Displays the SCSI INQUIRY page 0 model for logical drive
Y of controller X.

-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/rev
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/rev
Date: March 2009
Kernel Version: 2.6.30
Contact: [email protected]
Description: Displays the SCSI INQUIRY page 0 revision for logical
drive Y of controller X.

-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/unique_id
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/unique_id
Date: March 2009
Kernel Version: 2.6.30
Contact: [email protected]
Description: Displays the SCSI INQUIRY page 83 serial number for logical
drive Y of controller X.

-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/vendor
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/vendor
Date: March 2009
Kernel Version: 2.6.30
Contact: [email protected]
Description: Displays the SCSI INQUIRY page 0 vendor for logical drive
Y of controller X.

-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/block:cciss!cXdY
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/block:cciss!cXdY
Date: March 2009
Kernel Version: 2.6.30
Contact: [email protected]
Description: A symbolic link to /sys/block/cciss!cXdY

-Where: /sys/bus/pci/devices/<dev>/ccissX/rescan
+What: /sys/bus/pci/devices/<dev>/ccissX/rescan
Date: August 2009
Kernel Version: 2.6.31
Contact: [email protected]
Description: Kicks of a rescan of the controller to discover logical
drive topology changes.

-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/lunid
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/lunid
Date: August 2009
Kernel Version: 2.6.31
Contact: [email protected]
Description: Displays the 8-byte LUN ID used to address logical
drive Y of controller X.

-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/raid_level
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/raid_level
Date: August 2009
Kernel Version: 2.6.31
Contact: [email protected]
Description: Displays the RAID level of logical drive Y of
controller X.

-Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/usage_count
+What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/usage_count
Date: August 2009
Kernel Version: 2.6.31
Contact: [email protected]
Description: Displays the usage count (number of opens) of logical drive Y
of controller X.

-Where: /sys/bus/pci/devices/<dev>/ccissX/resettable
+What: /sys/bus/pci/devices/<dev>/ccissX/resettable
Date: February 2011
Kernel Version: 2.6.38
Contact: [email protected]
@@ -71,7 +71,7 @@ Description: Value of 1 indicates the controller can honor the reset_devices
a dump device, as kdump requires resetting the device in order
to work reliably.

-Where: /sys/bus/pci/devices/<dev>/ccissX/transport_mode
+What: /sys/bus/pci/devices/<dev>/ccissX/transport_mode
Date: July 2011
Kernel Version: 3.0
Contact: [email protected]
diff --git a/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg b/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
index 70d00dfa443d..f6199b314196 100644
--- a/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
+++ b/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
@@ -1,12 +1,12 @@
-Where: /sys/bus/usb/.../powered
+What: /sys/bus/usb/.../powered
Date: August 2008
Kernel Version: 2.6.26
Contact: Harrison Metzger <[email protected]>
Description: Controls whether the device's display will powered.
A value of 0 is off and a non-zero value is on.

-Where: /sys/bus/usb/.../mode_msb
-Where: /sys/bus/usb/.../mode_lsb
+What: /sys/bus/usb/.../mode_msb
+What: /sys/bus/usb/.../mode_lsb
Date: August 2008
Kernel Version: 2.6.26
Contact: Harrison Metzger <[email protected]>
@@ -16,7 +16,7 @@ Description: Controls the devices display mode.
for an 8 character display the values are
MSB 0x08; LSB 0xFF.

-Where: /sys/bus/usb/.../textmode
+What: /sys/bus/usb/.../textmode
Date: August 2008
Kernel Version: 2.6.26
Contact: Harrison Metzger <[email protected]>
@@ -25,13 +25,13 @@ Description: Controls the way the device interprets its text buffer.
hex: each character is between 0-15
ascii: each character is between '0'-'9' and 'A'-'F'.

-Where: /sys/bus/usb/.../text
+What: /sys/bus/usb/.../text
Date: August 2008
Kernel Version: 2.6.26
Contact: Harrison Metzger <[email protected]>
Description: The text (or data) for the device to display

-Where: /sys/bus/usb/.../decimals
+What: /sys/bus/usb/.../decimals
Date: August 2008
Kernel Version: 2.6.26
Contact: Harrison Metzger <[email protected]>
diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl
index bbbabffc682a..fc7c6f7c21b3 100644
--- a/Documentation/ABI/testing/sysfs-class-cxl
+++ b/Documentation/ABI/testing/sysfs-class-cxl
@@ -1,6 +1,6 @@
-Note: Attributes that are shared between devices are stored in the directory
-pointed to by the symlink device/.
-Example: The real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
+Please notice that attributes that are shared between devices are stored in
+the directory pointed to by the symlink device/.
+For example, the real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
/sys/class/cxl/afu0.0s/device/irqs_max, i.e. /sys/class/cxl/afu0.0/irqs_max.

diff --git a/Documentation/ABI/testing/sysfs-class-devfreq b/Documentation/ABI/testing/sysfs-class-devfreq
index ee39acacf6f8..01196e19afca 100644
--- a/Documentation/ABI/testing/sysfs-class-devfreq
+++ b/Documentation/ABI/testing/sysfs-class-devfreq
@@ -47,7 +47,7 @@ Description:
What: /sys/class/devfreq/.../trans_stat
Date: October 2012
Contact: MyungJoo Ham <[email protected]>
-Descrtiption:
+Description:
This ABI shows the statistics of devfreq behavior on a
specific device. It shows the time spent in each state and
the number of transitions between states.
diff --git a/Documentation/ABI/testing/sysfs-class-powercap b/Documentation/ABI/testing/sysfs-class-powercap
index db3b3ff70d84..f333a0ccc29b 100644
--- a/Documentation/ABI/testing/sysfs-class-powercap
+++ b/Documentation/ABI/testing/sysfs-class-powercap
@@ -147,6 +147,6 @@ What: /sys/class/powercap/.../<power zone>/enabled
Date: September 2013
KernelVersion: 3.13
Contact: [email protected]
-Description
+Description:
This allows to enable/disable power capping at power zone level.
This applies to current power zone and its children.
diff --git a/Documentation/ABI/testing/sysfs-kernel-fscaps b/Documentation/ABI/testing/sysfs-kernel-fscaps
index 50a3033b5e15..bcff34665192 100644
--- a/Documentation/ABI/testing/sysfs-kernel-fscaps
+++ b/Documentation/ABI/testing/sysfs-kernel-fscaps
@@ -2,7 +2,7 @@ What: /sys/kernel/fscaps
Date: February 2011
KernelVersion: 2.6.38
Contact: Ludwig Nussel <[email protected]>
-Description
+Description:
Shows whether file system capabilities are honored
when executing a binary

diff --git a/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo b/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
index 7bd81168e063..1f1087a5f075 100644
--- a/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
+++ b/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
@@ -4,7 +4,7 @@ KernelVersion: 2.6.24
Contact: Ken'ichi Ohmichi <[email protected]>
Kexec Mailing List <[email protected]>
Vivek Goyal <[email protected]>
-Description
+Description:
Shows physical address and size of vmcoreinfo ELF note.
First value contains physical address of note in hex and
second value contains the size of note in hex. This ELF
--
2.21.0

2019-06-14 02:05:17

[permalink] [raw]

Subject: [PATCH 04/14] ABI: better identificate tables

From: Mauro Carvalho Chehab <[email protected]>

When parsing via script, it is important to know if the script
should consider a description as a literal block that should
be displayed as-is, or if the description can be considered
as a normal text.

Change descriptions to ensure that the preceding line of a table
ends with a colon. That makes easy to identify the need of a
literal block.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra | 2 +-
.../ABI/testing/sysfs-class-backlight-driver-lm3533 | 6 +++---
Documentation/ABI/testing/sysfs-class-led-driver-lm3533 | 8 ++++----
Documentation/ABI/testing/sysfs-class-leds-gt683r | 4 ++--
Documentation/ABI/testing/sysfs-driver-hid-roccat-kone | 2 +-
5 files changed, 11 insertions(+), 11 deletions(-)

diff --git a/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra b/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
index 16020b31ae64..5d41ebadf15e 100644
--- a/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
+++ b/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
@@ -5,7 +5,7 @@ Description: It is possible to switch the cpi setting of the mouse with the
press of a button.
When read, this file returns the raw number of the actual cpi
setting reported by the mouse. This number has to be further
- processed to receive the real dpi value.
+ processed to receive the real dpi value:

VALUE DPI
1 400
diff --git a/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533 b/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533
index 77cf7ac949af..c0e0a9ae7b3d 100644
--- a/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533
+++ b/Documentation/ABI/testing/sysfs-class-backlight-driver-lm3533
@@ -4,7 +4,7 @@ KernelVersion: 3.5
Contact: Johan Hovold <[email protected]>
Description:
Get the ALS output channel used as input in
- ALS-current-control mode (0, 1), where
+ ALS-current-control mode (0, 1), where:

0 - out_current0 (backlight 0)
1 - out_current1 (backlight 1)
@@ -28,7 +28,7 @@ Date: April 2012
KernelVersion: 3.5
Contact: Johan Hovold <[email protected]>
Description:
- Set the brightness-mapping mode (0, 1), where
+ Set the brightness-mapping mode (0, 1), where:

0 - exponential mode
1 - linear mode
@@ -38,7 +38,7 @@ Date: April 2012
KernelVersion: 3.5
Contact: Johan Hovold <[email protected]>
Description:
- Set the PWM-input control mask (5 bits), where
+ Set the PWM-input control mask (5 bits), where:

bit 5 - PWM-input enabled in Zone 4
bit 4 - PWM-input enabled in Zone 3
diff --git a/Documentation/ABI/testing/sysfs-class-led-driver-lm3533 b/Documentation/ABI/testing/sysfs-class-led-driver-lm3533
index 620ebb3b9baa..e4c89b261546 100644
--- a/Documentation/ABI/testing/sysfs-class-led-driver-lm3533
+++ b/Documentation/ABI/testing/sysfs-class-led-driver-lm3533
@@ -4,7 +4,7 @@ KernelVersion: 3.5
Contact: Johan Hovold <[email protected]>
Description:
Set the ALS output channel to use as input in
- ALS-current-control mode (1, 2), where
+ ALS-current-control mode (1, 2), where:

1 - out_current1
2 - out_current2
@@ -22,7 +22,7 @@ Date: April 2012
KernelVersion: 3.5
Contact: Johan Hovold <[email protected]>
Description:
- Set the pattern generator fall and rise times (0..7), where
+ Set the pattern generator fall and rise times (0..7), where:

0 - 2048 us
1 - 262 ms
@@ -45,7 +45,7 @@ Date: April 2012
KernelVersion: 3.5
Contact: Johan Hovold <[email protected]>
Description:
- Set the brightness-mapping mode (0, 1), where
+ Set the brightness-mapping mode (0, 1), where:

0 - exponential mode
1 - linear mode
@@ -55,7 +55,7 @@ Date: April 2012
KernelVersion: 3.5
Contact: Johan Hovold <[email protected]>
Description:
- Set the PWM-input control mask (5 bits), where
+ Set the PWM-input control mask (5 bits), where:

bit 5 - PWM-input enabled in Zone 4
bit 4 - PWM-input enabled in Zone 3
diff --git a/Documentation/ABI/testing/sysfs-class-leds-gt683r b/Documentation/ABI/testing/sysfs-class-leds-gt683r
index e4fae6026e79..6adab27f646e 100644
--- a/Documentation/ABI/testing/sysfs-class-leds-gt683r
+++ b/Documentation/ABI/testing/sysfs-class-leds-gt683r
@@ -5,7 +5,7 @@ Contact: Janne Kanniainen <[email protected]>
Description:
Set the mode of LEDs. You should notice that changing the mode
of one LED will update the mode of its two sibling devices as
- well.
+ well. Possible values are:

0 - normal
1 - audio
@@ -13,4 +13,4 @@ Description:

Normal: LEDs are fully on when enabled
Audio: LEDs brightness depends on sound level
- Breathing: LEDs brightness varies at human breathing rate
\ No newline at end of file
+ Breathing: LEDs brightness varies at human breathing rate
diff --git a/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone b/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone
index 3ca3971109bf..8f7982c70d72 100644
--- a/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone
+++ b/Documentation/ABI/testing/sysfs-driver-hid-roccat-kone
@@ -5,7 +5,7 @@ Description: It is possible to switch the dpi setting of the mouse with the
press of a button.
When read, this file returns the raw number of the actual dpi
setting reported by the mouse. This number has to be further
- processed to receive the real dpi value.
+ processed to receive the real dpi value:

VALUE DPI
1 800
--
2.21.0

2019-06-14 02:05:31

[permalink] [raw]

Subject: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support

The parser breaks with UTF-8 characters with Sphinx 1.4.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
Documentation/sphinx/kernel_abi.py | 10 ++++++----
1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 7fa7806532dc..460cee48a245 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -1,4 +1,5 @@
-# -*- coding: utf-8; mode: python -*-
+# coding=utf-8
+#
u"""
kernel-abi
~~~~~~~~~~
@@ -28,6 +29,7 @@ u"""

"""

+import codecs
import sys
import os
from os import path
@@ -124,12 +126,12 @@ class KernelCmd(Directive):
cmd
, stdout = subprocess.PIPE
, stderr = subprocess.PIPE
- , universal_newlines = True
, **kwargs
)
out, err = proc.communicate()
- if err:
- self.warn(err)
+
+ out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
+
if proc.returncode != 0:
raise self.severe(
u"command '%s' failed with return code %d"
--
2.21.0

2019-06-14 02:05:43

[permalink] [raw]

Subject: [PATCH 03/14] ABI: sysfs-class-uwb_rc: remove a duplicated incomplete entry

From: Mauro Carvalho Chehab <[email protected]>

There are two entries for /sys/class/uwb_rc/uwbN/<EUI-48>/BPST.
The second one has a missing description.

Get rid of it.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
Documentation/ABI/testing/sysfs-class-uwb_rc | 6 ------
1 file changed, 6 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-class-uwb_rc b/Documentation/ABI/testing/sysfs-class-uwb_rc
index 85f4875d16ac..a0578751c1e3 100644
--- a/Documentation/ABI/testing/sysfs-class-uwb_rc
+++ b/Documentation/ABI/testing/sysfs-class-uwb_rc
@@ -125,12 +125,6 @@ Description:
The EUI-48 of this device in colon separated hex
octets.

-What: /sys/class/uwb_rc/uwbN/<EUI-48>/BPST
-Date: July 2008
-KernelVersion: 2.6.27
-Contact: [email protected]
-Description:
-
What: /sys/class/uwb_rc/uwbN/<EUI-48>/IEs
Date: July 2008
KernelVersion: 2.6.27
--
2.21.0

2019-06-14 02:05:47

[permalink] [raw]

Subject: [PATCH 08/14] scripts/get_abi.pl: split label naming from xref logic

From: Mauro Carvalho Chehab <[email protected]>

Instead of using a ReST compilant label while parsing,
move the label to ReST output. That makes the parsing logic
more generic, allowing it to provide other types of output.

As a side effect, now all files used to generate the output
will be output. We can later add command line arguments to
filter.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/get_abi.pl | 94 ++++++++++++++++++++++++++--------------------
1 file changed, 53 insertions(+), 41 deletions(-)

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index ba8a7466f896..d437e148b1c0 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -49,17 +49,23 @@ sub parse_abi {
my $name = $file;
$name =~ s,.*/,,;

+ my $nametag = "File $name";
+ $data{$nametag}->{what} = "File $name";
+ $data{$nametag}->{type} = "File";
+ $data{$nametag}->{file} = $name;
+ $data{$nametag}->{is_file} = 1;
+
my $type = $file;
$type =~ s,.*/(.*)/.*,$1,;

my $what;
my $new_what;
my $tag;
- my $label;
my $ln;
- my $has_file;
my $xrefs;
my $space;
+ my @labels;
+ my $label;

print STDERR "Opening $file\n" if ($debug > 1);
open IN, $file;
@@ -88,28 +94,13 @@ sub parse_abi {
parse_error($file, $ln, "What '$what' doesn't have a description", "") if ($what && !$data{$what}->{description});

$what = $content;
+ $label = $content;
$new_what = 1;
}
+ push @labels, [($content, $label)];
$tag = $new_tag;

- if ($has_file) {
- $label = "abi_" . $content . " ";
- $label =~ tr/A-Z/a-z/;
-
- # Convert special chars to "_"
- $label =~s/[\x00-\x2f]+/_/g;
- $label =~s/[\x3a-\x40]+/_/g;
- $label =~s/[\x7b-\xff]+/_/g;
- $label =~ s,_+,_,g;
- $label =~ s,_$,,;
-
- $data{$what}->{label} .= $label;
-
- # Escape special chars from content
- $content =~s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g;
-
- $xrefs .= "- :ref:`$content <$label>`\n\n";
- }
+ push @{$data{$nametag}->{xrefs}}, [($content, $label)] if ($data{$nametag}->{what});
next;
}

@@ -117,6 +108,9 @@ sub parse_abi {
$tag = $new_tag;

if ($new_what) {
+ @{$data{$what}->{label}} = @labels if ($data{$nametag}->{what});
+ @labels = ();
+ $label = "";
$new_what = 0;

$data{$what}->{type} = $type;
@@ -145,15 +139,8 @@ sub parse_abi {
}

# Store any contents before tags at the database
- if (!$tag) {
- next if (/^\n/);
-
- my $my_what = "File $name";
- $data{$my_what}->{what} = "File $name";
- $data{$my_what}->{type} = "File";
- $data{$my_what}->{file} = $name;
- $data{$my_what}->{description} .= $_;
- $has_file = 1;
+ if (!$tag && $data{$nametag}->{what}) {
+ $data{$nametag}->{description} .= $_;
next;
}

@@ -192,12 +179,8 @@ sub parse_abi {
# Everything else is error
parse_error($file, $ln, "Unexpected line:", $_);
}
+ $data{$nametag}->{description} =~ s/^\n+//;
close IN;
-
- if ($has_file) {
- my $my_what = "File $name";
- $data{$my_what}->{xrefs} = $xrefs;
- }
}

# Outputs the output on ReST format
@@ -212,11 +195,22 @@ sub output_rest {
my $bar = $w;
$bar =~ s/./-/g;

- if ($data{$what}->{label}) {
- my @labels = split(/\s/, $data{$what}->{label});
- foreach my $label (@labels) {
- printf ".. _%s:\n\n", $label;
- }
+ foreach my $p (@{$data{$what}->{label}}) {
+ my ($content, $label) = @{$p};
+ $label = "abi_" . $label . " ";
+ $label =~ tr/A-Z/a-z/;
+
+ # Convert special chars to "_"
+ $label =~s/([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff])/_/g;
+ $label =~ s,_+,_,g;
+ $label =~ s,_$,,;
+
+ $data{$what}->{label} .= $label;
+
+ printf ".. _%s:\n\n", $label;
+
+ # only one label is enough
+ last;
}

print "$w\n$bar\n\n";
@@ -243,10 +237,28 @@ sub output_rest {
print "$desc\n\n";
}
} else {
- print "DESCRIPTION MISSING for $what\n\n";
+ print "DESCRIPTION MISSING for $what\n\n" if (!$data{$what}->{is_file});
}

- printf "Has the following ABI:\n\n%s", $data{$what}->{xrefs} if ($data{$what}->{xrefs});
+ if ($data{$what}->{xrefs}) {
+ printf "Has the following ABI:\n\n";
+
+ foreach my $p(@{$data{$what}->{xrefs}}) {
+ my ($content, $label) = @{$p};
+ $label = "abi_" . $label . " ";
+ $label =~ tr/A-Z/a-z/;
+
+ # Convert special chars to "_"
+ $label =~s/([\x00-\x2f\x3a-\x40\x5b-\x60\x7b-\xff])/_/g;
+ $label =~ s,_+,_,g;
+ $label =~ s,_$,,;
+
+ # Escape special chars from content
+ $content =~s/([\x00-\x1f\x21-\x2f\x3a-\x40\x7b-\xff])/\\$1/g;
+
+ print "- :ref:`$content <$label>`\n\n";
+ }
+ }
}
}

--
2.21.0

2019-06-14 02:05:51

[permalink] [raw]

Subject: [PATCH 02/14] ABI: sysfs-driver-hid: the "What" field doesn't parse fine

From: Mauro Carvalho Chehab <[email protected]>

The What: field on this ABI description use a different
syntax than expected, causing it to not be properly parsed
by script.

Fix it.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
Documentation/ABI/testing/sysfs-driver-hid | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/Documentation/ABI/testing/sysfs-driver-hid b/Documentation/ABI/testing/sysfs-driver-hid
index 48942cacb0bf..a59533410871 100644
--- a/Documentation/ABI/testing/sysfs-driver-hid
+++ b/Documentation/ABI/testing/sysfs-driver-hid
@@ -1,6 +1,6 @@
-What: For USB devices : /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/report_descriptor
- For BT devices : /sys/class/bluetooth/hci<addr>/<hid-bus>:<vendor-id>:<product-id>.<num>/report_descriptor
- Symlink : /sys/class/hidraw/hidraw<num>/device/report_descriptor
+What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/report_descriptor
+What: /sys/class/bluetooth/hci<addr>/<hid-bus>:<vendor-id>:<product-id>.<num>/report_descriptor
+What: /sys/class/hidraw/hidraw<num>/device/report_descriptor
Date: Jan 2011
KernelVersion: 2.0.39
Contact: Alan Ott <[email protected]>
@@ -9,9 +9,9 @@ Description: When read, this file returns the device's raw binary HID
This file cannot be written.
Users: HIDAPI library (http://www.signal11.us/oss/hidapi)

-What: For USB devices : /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/country
- For BT devices : /sys/class/bluetooth/hci<addr>/<hid-bus>:<vendor-id>:<product-id>.<num>/country
- Symlink : /sys/class/hidraw/hidraw<num>/device/country
+What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/country
+What: /sys/class/bluetooth/hci<addr>/<hid-bus>:<vendor-id>:<product-id>.<num>/country
+What: /sys/class/hidraw/hidraw<num>/device/country
Date: February 2015
KernelVersion: 3.19
Contact: Olivier Gay <[email protected]>
--
2.21.0

2019-06-14 02:06:02

[permalink] [raw]

Subject: [PATCH 09/14] scripts/get_abi.pl: add support for searching for ABI symbols

From: Mauro Carvalho Chehab <[email protected]>

Change its syntax to allow switching between ReST output mode
and a new search mode, with allows to seek for ABI symbols
using regex.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/get_abi.pl | 112 +++++++++++++++++++++++++++++++++++++++++----
1 file changed, 103 insertions(+), 9 deletions(-)

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index d437e148b1c0..c202292af1a2 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -9,9 +9,11 @@ use Fcntl ':mode';
my $help;
my $man;
my $debug;
+my $prefix="Documentation/ABI";

GetOptions(
"debug|d+" => \$debug,
+ "dir=s" => \$prefix,
'help|?' => \$help,
man => \$man
) or pod2usage(2);
@@ -19,9 +21,12 @@ GetOptions(
pod2usage(1) if $help;
pod2usage(-exitstatus => 0, -verbose => 2) if $man;

-pod2usage(2) if (scalar @ARGV != 1);
+pod2usage(2) if (scalar @ARGV < 1 || @ARGV > 2);

-my ($prefix) = @ARGV;
+my ($cmd, $arg) = @ARGV;
+
+pod2usage(2) if ($cmd ne "search" && $cmd ne "rest");
+pod2usage(2) if ($cmd eq "search" && !$arg);

require Data::Dumper if ($debug);

@@ -53,6 +58,7 @@ sub parse_abi {
$data{$nametag}->{what} = "File $name";
$data{$nametag}->{type} = "File";
$data{$nametag}->{file} = $name;
+ $data{$nametag}->{filepath} = $file;
$data{$nametag}->{is_file} = 1;

my $type = $file;
@@ -115,6 +121,7 @@ sub parse_abi {

$data{$what}->{type} = $type;
$data{$what}->{file} = $name;
+ $data{$what}->{filepath} = $file;
print STDERR "\twhat: $what\n" if ($debug > 1);
}

@@ -183,7 +190,9 @@ sub parse_abi {
close IN;
}

-# Outputs the output on ReST format
+#
+# Outputs the book on ReST format
+#
sub output_rest {
foreach my $what (sort keys %data) {
my $type = $data{$what}->{type};
@@ -262,6 +271,45 @@ sub output_rest {
}
}

+#
+# Searches for ABI symbols
+#
+sub search_symbols {
+ foreach my $what (sort keys %data) {
+ next if (!($what =~ m/($arg)/));
+
+ my $type = $data{$what}->{type};
+ next if ($type eq "File");
+
+ my $file = $data{$what}->{filepath};
+
+ my $bar = $what;
+ $bar =~ s/./-/g;
+
+ print "\n$what\n$bar\n\n";
+
+ my $kernelversion = $data{$what}->{kernelversion};
+ my $contact = $data{$what}->{contact};
+ my $users = $data{$what}->{users};
+ my $date = $data{$what}->{date};
+ my $desc = $data{$what}->{description};
+ $kernelversion =~ s/^\s+//;
+ $contact =~ s/^\s+//;
+ $users =~ s/^\s+//;
+ $users =~ s/\n//g;
+ $date =~ s/^\s+//;
+ $desc =~ s/^\s+//;
+
+ printf "Kernel version:\t\t%s\n", $kernelversion if ($kernelversion);
+ printf "Date:\t\t\t%s\n", $date if ($date);
+ printf "Contact:\t\t%s\n", $contact if ($contact);
+ printf "Users:\t\t\t%s\n", $users if ($users);
+ print "Defined on file:\t$file\n\n";
+ print "Description:\n\n$desc";
+ }
+}
+
+
#
# Parses all ABI files located at $prefix dir
#
@@ -270,9 +318,13 @@ find({wanted =>\&parse_abi, no_chdir => 1}, $prefix);
print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug);

#
-# Outputs an ReST file with the ABI contents
+# Handles the command
#
-output_rest
+if ($cmd eq "rest") {
+ output_rest;
+} else {
+ search_symbols;
+}

__END__
@@ -283,12 +335,27 @@ abi_book.pl - parse the Linux ABI files and produce a ReST book.

=head1 SYNOPSIS

-B<abi_book.pl> [--debug] <ABI_DIR>]
+B<abi_book.pl> [--debug] <COMAND> [<ARGUMENT>]
+
+Where <COMMAND> can be:
+
+=over 8
+
+B<search> [SEARCH_REGEX] - search for [SEARCH_REGEX] inside ABI
+
+B<rest> - output the ABI in ReST markup language
+
+=back

=head1 OPTIONS

=over 8

+=item B<--dir>
+
+Changes the location of the ABI search. By default, it uses
+the Documentation/ABI directory.
+
=item B<--debug>

Put the script in verbose mode, useful for debugging. Can be called multiple
@@ -306,8 +373,35 @@ Prints the manual page and exits.

=head1 DESCRIPTION

-Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI)
-and produce a ReST book containing the Linux ABI.
+Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI),
+allowing to search for ABI symbols or to produce a ReST book containing
+the Linux ABI documentation.
+
+=head1 EXAMPLES
+
+Search for all stable symbols with the word "usb":
+
+=over 8
+
+$ scripts/get_abi.pl search usb --dir Documentation/ABI/stable
+
+=back
+
+Search for all symbols that match the regex expression "usb.*cap":
+
+=over 8
+
+$ scripts/get_abi.pl search usb.*cap
+
+=back
+
+Output all obsoleted symbols in ReST format
+
+=over 8
+
+$ scripts/get_abi.pl rest --dir Documentation/ABI/obsolete
+
+=back

=head1 BUGS

@@ -315,7 +409,7 @@ Report bugs to Mauro Carvalho Chehab <[email protected]>

=head1 COPYRIGHT

-Copyright (c) 2016 by Mauro Carvalho Chehab <[email protected]>.
+Copyright (c) 2016-2017 by Mauro Carvalho Chehab <[email protected]>.

License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.

--
2.21.0

2019-06-14 02:06:10

[permalink] [raw]

Subject: [PATCH 11/14] scripts/get_abi.pl: fix parse issues with some files

A few files are failing to parse:

Documentation/ABI/testing/sysfs-bus-pci-devices-aer_stats
Documentation/ABI/testing/sysfs-class-pktcdvd
Documentation/ABI/testing/sysfs-bus-nfit

On all three files, the problem is that there is a ":" character
at the initial file description.

Improve the parse in order to handle those special cases.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/get_abi.pl | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index a2861fcec745..c3ccfdfc3a5d 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -87,7 +87,7 @@ sub parse_abi {
# New "tag" is actually part of
# description. Don't consider it a tag
$new_tag = "";
- } else {
+ } elsif ($tag ne "") {
parse_error($file, $ln, "tag '$tag' is invalid", $_);
}
}
@@ -110,7 +110,7 @@ sub parse_abi {
next;
}

- if ($new_tag) {
+ if ($tag ne "" && $new_tag) {
$tag = $new_tag;

if ($new_what) {
--
2.21.0

2019-06-14 02:06:16

[permalink] [raw]

Subject: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

From: Mauro Carvalho Chehab <[email protected]>

As we don't want a generic Sphinx extension to execute commands,
change the one proposed to Markus to call the abi_book.pl
script.

Use a script to parse the Documentation/ABI directory and output
it at the admin-guide.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
Documentation/admin-guide/abi-obsolete.rst | 10 ++
Documentation/admin-guide/abi-removed.rst | 4 +
Documentation/admin-guide/abi-stable.rst | 13 ++
Documentation/admin-guide/abi-testing.rst | 19 +++
Documentation/admin-guide/abi.rst | 11 ++
Documentation/admin-guide/index.rst | 1 +
Documentation/conf.py | 2 +-
Documentation/sphinx/kernel_abi.py | 155 +++++++++++++++++++++
8 files changed, 214 insertions(+), 1 deletion(-)
create mode 100644 Documentation/admin-guide/abi-obsolete.rst
create mode 100644 Documentation/admin-guide/abi-removed.rst
create mode 100644 Documentation/admin-guide/abi-stable.rst
create mode 100644 Documentation/admin-guide/abi-testing.rst
create mode 100644 Documentation/admin-guide/abi.rst
create mode 100644 Documentation/sphinx/kernel_abi.py

diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst
new file mode 100644
index 000000000000..cda9168445a5
--- /dev/null
+++ b/Documentation/admin-guide/abi-obsolete.rst
@@ -0,0 +1,10 @@
+ABI obsolete symbols
+====================
+
+Documents interfaces that are still remaining in the kernel, but are
+marked to be removed at some later point in time.
+
+The description of the interface will document the reason why it is
+obsolete and when it can be expected to be removed.
+
+.. kernel-abi:: $srctree/Documentation/ABI/obsolete
diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst
new file mode 100644
index 000000000000..497978fc9632
--- /dev/null
+++ b/Documentation/admin-guide/abi-removed.rst
@@ -0,0 +1,4 @@
+ABI removed symbols
+===================
+
+.. kernel-abi:: $srctree/Documentation/ABI/removed
diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst
new file mode 100644
index 000000000000..7495d7a35048
--- /dev/null
+++ b/Documentation/admin-guide/abi-stable.rst
@@ -0,0 +1,13 @@
+ABI stable symbols
+==================
+
+Documents the interfaces that the developer has defined to be stable.
+
+Userspace programs are free to use these interfaces with no
+restrictions, and backward compatibility for them will be guaranteed
+for at least 2 years.
+
+Most interfaces (like syscalls) are expected to never change and always
+be available.
+
+.. kernel-abi:: $srctree/Documentation/ABI/stable
diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst
new file mode 100644
index 000000000000..5c886fc50b9e
--- /dev/null
+++ b/Documentation/admin-guide/abi-testing.rst
@@ -0,0 +1,19 @@
+ABI testing symbols
+===================
+
+Documents interfaces that are felt to be stable,
+as the main development of this interface has been completed.
+
+The interface can be changed to add new features, but the
+current interface will not break by doing this, unless grave
+errors or security problems are found in them.
+
+Userspace programs can start to rely on these interfaces, but they must
+be aware of changes that can occur before these interfaces move to
+be marked stable.
+
+Programs that use these interfaces are strongly encouraged to add their
+name to the description of these interfaces, so that the kernel
+developers can easily notify them if any changes occur.
+
+.. kernel-abi:: $srctree/Documentation/ABI/testing
diff --git a/Documentation/admin-guide/abi.rst b/Documentation/admin-guide/abi.rst
new file mode 100644
index 000000000000..3b9645c77469
--- /dev/null
+++ b/Documentation/admin-guide/abi.rst
@@ -0,0 +1,11 @@
+=====================
+Linux ABI description
+=====================
+
+.. toctree::
+ :maxdepth: 1
+
+ abi-stable
+ abi-testing
+ abi-obsolete
+ abi-removed
diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guide/index.rst
index 8001917ee012..20c3020fd73c 100644
--- a/Documentation/admin-guide/index.rst
+++ b/Documentation/admin-guide/index.rst
@@ -16,6 +16,7 @@ etc.
README
kernel-parameters
devices
+ abi

This section describes CPU vulnerabilities and their mitigations.

diff --git a/Documentation/conf.py b/Documentation/conf.py
index 7ace3f8852bd..598256fb5c98 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -34,7 +34,7 @@ needs_sphinx = '1.3'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig']
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig', 'kernel_abi']

# The name of the math extension changed on Sphinx 1.4
if (major == 1 and minor > 3) or (major > 1):
diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
new file mode 100644
index 000000000000..32ce90775d96
--- /dev/null
+++ b/Documentation/sphinx/kernel_abi.py
@@ -0,0 +1,155 @@
+# -*- coding: utf-8; mode: python -*-
+u"""
+ kernel-abi
+ ~~~~~~~~~~
+
+ Implementation of the ``kernel-abi`` reST-directive.
+
+ :copyright: Copyright (C) 2016 Markus Heiser
+ :copyright: Copyright (C) 2016 Mauro Carvalho Chehab
+ :license: GPL Version 2, June 1991 see Linux/COPYING for details.
+
+ The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
+ scripts/get_abi.pl script to parse the Kernel ABI files.
+
+ Overview of directive's argument and options.
+
+ .. code-block:: rst
+
+ .. kernel-abi:: <ABI directory location>
+ :debug:
+
+ The argument ``<ABI directory location>`` is required. It contains the
+ location of the ABI files to be parsed.
+
+ ``debug``
+ Inserts a code-block with the *raw* reST. Sometimes it is helpful to see
+ what reST is generated.
+
+"""
+
+import sys
+import os
+from os import path
+import subprocess
+
+from sphinx.ext.autodoc import AutodocReporter
+
+from docutils import nodes
+from docutils.parsers.rst import Directive, directives
+from docutils.statemachine import ViewList
+from docutils.utils.error_reporting import ErrorString
+
+
+__version__ = '1.0'
+
+# We can't assume that six is installed
+PY3 = sys.version_info[0] == 3
+PY2 = sys.version_info[0] == 2
+if PY3:
+ # pylint: disable=C0103, W0622
+ unicode = str
+ basestring = str
+
+def setup(app):
+
+ app.add_directive("kernel-abi", KernelCmd)
+ return dict(
+ version = __version__
+ , parallel_read_safe = True
+ , parallel_write_safe = True
+ )
+
+class KernelCmd(Directive):
+
+ u"""KernelABI (``kernel-abi``) directive"""
+
+ required_arguments = 1
+ optional_arguments = 0
+ has_content = False
+ final_argument_whitespace = True
+
+ option_spec = {
+ "debug" : directives.flag
+ }
+
+ def warn(self, message, **replace):
+ replace["fname"] = self.state.document.current_source
+ replace["line_no"] = replace.get("line_no", self.lineno)
+ message = ("%(fname)s:%(line_no)s: [kernel-abi WARN] : " + message) % replace
+ self.state.document.settings.env.app.warn(message, prefix="")
+
+ def run(self):
+
+ doc = self.state.document
+ if not doc.settings.file_insertion_enabled:
+ raise self.warning("docutils: file insertion disabled")
+
+ env = doc.settings.env
+ cwd = path.dirname(doc.current_source)
+ cmd = "get_abi.pl rest --dir "
+ cmd += self.arguments[0]
+
+ srctree = path.abspath(os.environ["srctree"])
+
+ fname = cmd
+
+ # extend PATH with $(srctree)/scripts
+ path_env = os.pathsep.join([
+ srctree + os.sep + "scripts",
+ os.environ["PATH"]
+ ])
+ shell_env = os.environ.copy()
+ shell_env["PATH"] = path_env
+ shell_env["srctree"] = srctree
+
+ lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
+ nodeList = self.nestedParse(lines, fname)
+ return nodeList
+
+ def runCmd(self, cmd, **kwargs):
+ u"""Run command ``cmd`` and return it's stdout as unicode."""
+
+ try:
+ proc = subprocess.Popen(
+ cmd
+ , stdout = subprocess.PIPE
+ , stderr = subprocess.PIPE
+ , universal_newlines = True
+ , **kwargs
+ )
+ out, err = proc.communicate()
+ if err:
+ self.warn(err)
+ if proc.returncode != 0:
+ raise self.severe(
+ u"command '%s' failed with return code %d"
+ % (cmd, proc.returncode)
+ )
+ except OSError as exc:
+ raise self.severe(u"problems with '%s' directive: %s."
+ % (self.name, ErrorString(exc)))
+ return unicode(out)
+
+ def nestedParse(self, lines, fname):
+ content = ViewList()
+ node = nodes.section()
+
+ if "debug" in self.options:
+ code_block = "\n\n.. code-block:: rst\n :linenos:\n"
+ for l in lines.split("\n"):
+ code_block += "\n " + l
+ lines = code_block + "\n\n"
+
+ for c, l in enumerate(lines.split("\n")):
+ content.append(l, fname, c)
+
+ buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
+ self.state.memo.title_styles = []
+ self.state.memo.section_level = 0
+ self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
+ try:
+ self.state.nested_parse(content, 0, node, match_titles=1)
+ finally:
+ self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
+ return node.children
--
2.21.0

2019-06-14 02:06:17

[permalink] [raw]

Subject: [PATCH 05/14] scripts: add an script to parse the ABI files

From: Mauro Carvalho Chehab <[email protected]>

Add a script to parse the Documentation/ABI files and produce
an output with all entries inside an ABI (sub)directory.

Right now, it outputs its contents on ReST format. It shouldn't
be hard to make it produce other kind of outputs, since the ABI
file parser is implemented in separate than the output generator.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/get_abi.pl | 212 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 212 insertions(+)
create mode 100755 scripts/get_abi.pl

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
new file mode 100755
index 000000000000..f7c9944a833c
--- /dev/null
+++ b/scripts/get_abi.pl
@@ -0,0 +1,212 @@
+#!/usr/bin/perl
+
+use strict;
+use Pod::Usage;
+use Getopt::Long;
+use File::Find;
+use Fcntl ':mode';
+
+my $help;
+my $man;
+my $debug;
+
+GetOptions(
+ "debug|d+" => \$debug,
+ 'help|?' => \$help,
+ man => \$man
+) or pod2usage(2);
+
+pod2usage(1) if $help;
+pod2usage(-exitstatus => 0, -verbose => 2) if $man;
+
+pod2usage(2) if (scalar @ARGV != 1);
+
+my ($prefix) = @ARGV;
+
+require Data::Dumper if ($debug);
+
+my %data;
+
+#
+# Displays an error message, printing file name and line
+#
+sub parse_error($$$$) {
+ my ($file, $ln, $msg, $data) = @_;
+
+ print STDERR "file $file#$ln: $msg at\n\t$data";
+}
+
+#
+# Parse an ABI file, storing its contents at %data
+#
+sub parse_abi {
+ my $file = $File::Find::name;
+
+ my $mode = (stat($file))[2];
+ return if ($mode & S_IFDIR);
+ return if ($file =~ m,/README,);
+
+ my $name = $file;
+ $name =~ s,.*/,,;
+
+ my $type = $file;
+ $type =~ s,.*/(.*)/.*,$1,;
+
+ my $what;
+ my $new_what;
+ my $tag;
+ my $ln;
+
+ print STDERR "Opening $file\n" if ($debug > 1);
+ open IN, $file;
+ while(<IN>) {
+ $ln++;
+ if (m/^(\S+):\s*(.*)/i) {
+ my $new_tag = lc($1);
+ my $content = $2;
+
+ if (!($new_tag =~ m/(what|date|kernelversion|contact|description|users)/)) {
+ if ($tag eq "description") {
+ $data{$what}->{$tag} .= "\n$content";;
+ $data{$what}->{$tag} =~ s/\n+$//;
+ next;
+ } else {
+ parse_error($file, $ln, "tag '$tag' is invalid", $_);
+ }
+ }
+
+ if ($new_tag =~ m/what/) {
+ if ($tag =~ m/what/) {
+ $what .= ", " . $content;
+ } else {
+ $what = $content;
+ $new_what = 1;
+ }
+ $tag = $new_tag;
+ next;
+ }
+
+ $tag = $new_tag;
+
+ if ($new_what) {
+ $new_what = 0;
+
+ $data{$what}->{type} = $type;
+ $data{$what}->{file} = $name;
+ print STDERR "\twhat: $what\n" if ($debug > 1);
+ }
+
+ if (!$what) {
+ parse_error($file, $ln, "'What:' should come first:", $_);
+ next;
+ }
+ $data{$what}->{$tag} = $content;
+ next;
+ }
+
+ # Silently ignore any headers before the database
+ next if (!$tag);
+
+ if (m/^\s*(.*)/) {
+ $data{$what}->{$tag} .= "\n$1";
+ $data{$what}->{$tag} =~ s/\n+$//;
+ next;
+ }
+
+ # Everything else is error
+ parse_error($file, $ln, "Unexpected line:", $_);
+ }
+ close IN;
+}
+
+# Outputs the output on ReST format
+sub output_rest {
+ foreach my $what (sort keys %data) {
+ my $type = $data{$what}->{type};
+ my $file = $data{$what}->{file};
+
+ my $w = $what;
+ $w =~ s/([\_\-\*\=\^\~\\])/\\$1/g;
+
+ print "$w\n\n";
+ print "- defined on file $file (type: $type)\n\n::\n\n";
+
+ my $desc = $data{$what}->{description};
+ $desc =~ s/^\s+//;
+
+ # Remove title markups from the description, as they won't work
+ $desc =~ s/\n[\-\*\=\^\~]+\n/\n/g;
+
+ # put everything inside a code block
+ $desc =~ s/\n/\n /g;
+
+
+ if (!($desc =~ /^\s*$/)) {
+ print " $desc\n\n";
+ } else {
+ print " DESCRIPTION MISSING\n\n";
+ }
+ }
+}
+
+#
+# Parses all ABI files located at $prefix dir
+#
+find({wanted =>\&parse_abi, no_chdir => 1}, $prefix);
+
+print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug);
+
+#
+# Outputs an ReST file with the ABI contents
+#
+output_rest
+
+
+__END__
+
+=head1 NAME
+
+abi_book.pl - parse the Linux ABI files and produce a ReST book.
+
+=head1 SYNOPSIS
+
+B<abi_book.pl> [--debug] <ABI_DIR>]
+
+=head1 OPTIONS
+
+=over 8
+
+=item B<--debug>
+
+Put the script in verbose mode, useful for debugging. Can be called multiple
+times, to increase verbosity.
+
+=item B<--help>
+
+Prints a brief help message and exits.
+
+=item B<--man>
+
+Prints the manual page and exits.
+
+=back
+
+=head1 DESCRIPTION
+
+Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI)
+and produce a ReST book containing the Linux ABI.
+
+=head1 BUGS
+
+Report bugs to Mauro Carvalho Chehab <[email protected]>
+
+=head1 COPYRIGHT
+
+Copyright (c) 2016 by Mauro Carvalho Chehab <[email protected]>.
+
+License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
+
+This is free software: you are free to change and redistribute it.
+There is NO WARRANTY, to the extent permitted by law.
+
+=cut
--
2.21.0

2019-06-14 02:06:26

[permalink] [raw]

Subject: [PATCH 10/14] scripts/get_abi.pl: represent what in tables

From: Mauro Carvalho Chehab <[email protected]>

Several entries at the ABI have multiple What: with the same
description.

Instead of showing those symbols as sections, let's show them
as tables. That makes easier to read on the final output,
and avoid too much recursion at Sphinx parsing.

We need to put file references at the end, as we don't want
non-file tables to be mangled with other entries.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/get_abi.pl | 41 ++++++++++++++++++++++++++++++++++++-----
1 file changed, 36 insertions(+), 5 deletions(-)

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index c202292af1a2..a2861fcec745 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -193,16 +193,19 @@ sub parse_abi {
#
# Outputs the book on ReST format
#
+
sub output_rest {
- foreach my $what (sort keys %data) {
+ foreach my $what (sort {
+ ($data{$a}->{type} eq "File") cmp ($data{$b}->{type} eq "File") ||
+ $a cmp $b
+ } keys %data) {
my $type = $data{$what}->{type};
my $file = $data{$what}->{file};
+ my $filepath = $data{$what}->{filepath};

my $w = $what;
$w =~ s/([\_\-\*\=\^\~\\])/\\$1/g;

- my $bar = $w;
- $bar =~ s/./-/g;

foreach my $p (@{$data{$what}->{label}}) {
my ($content, $label) = @{$p};
@@ -222,9 +225,37 @@ sub output_rest {
last;
}

- print "$w\n$bar\n\n";

- print "- defined on file $file (type: $type)\n\n" if ($type ne "File");
+ $filepath =~ s,.*/(.*/.*),\1,;;
+ $filepath =~ s,[/\-],_,g;;
+ my $fileref = "abi_file_".$filepath;
+
+ if ($type eq "File") {
+ my $bar = $w;
+ $bar =~ s/./-/g;
+
+ print ".. _$fileref:\n\n";
+ print "$w\n$bar\n\n";
+ } else {
+ my @names = split /\s*,\s*/,$w;
+
+ my $len = 0;
+
+ foreach my $name (@names) {
+ $len = length($name) if (length($name) > $len);
+ }
+
+ print "What:\n\n";
+
+ print "+-" . "-" x $len . "-+\n";
+ foreach my $name (@names) {
+ printf "| %s", $name . " " x ($len - length($name)) . " |\n";
+ print "+-" . "-" x $len . "-+\n";
+ }
+ print "\n";
+ }
+
+ print "Defined on file :ref:`$file <$fileref>`\n\n" if ($type ne "File");

my $desc = $data{$what}->{description};
$desc =~ s/^\s+//;
--
2.21.0

2019-06-14 02:06:59

[permalink] [raw]

Subject: [PATCH 07/14] scripts/get_abi.pl: avoid use literal blocks when not needed

From: Mauro Carvalho Chehab <[email protected]>

The usage of literal blocks make the document very complex,
causing the browser to take a long time to load.

On most ABI descriptions, they're a plain text, and don't
require a literal block.

So, add a logic there with identifies when a literal block
is needed.

As, on literal blocks, we need to respect the original
document space, the most complex part of this patch is
to preserve the original spacing where needed.

Signed-off-by: Mauro Carvalho Chehab <[email protected]>
Signed-off-by: Mauro Carvalho Chehab <[email protected]>
---
scripts/get_abi.pl | 104 +++++++++++++++++++++++++++++++++------------
1 file changed, 78 insertions(+), 26 deletions(-)

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index ac0f057fa3ca..ba8a7466f896 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -59,29 +59,34 @@ sub parse_abi {
my $ln;
my $has_file;
my $xrefs;
+ my $space;

print STDERR "Opening $file\n" if ($debug > 1);
open IN, $file;
while(<IN>) {
$ln++;
- if (m/^(\S+):\s*(.*)/i) {
+ if (m/^(\S+)(:\s*)(.*)/i) {
my $new_tag = lc($1);
- my $content = $2;
+ my $sep = $2;
+ my $content = $3;

if (!($new_tag =~ m/(what|date|kernelversion|contact|description|users)/)) {
if ($tag eq "description") {
- $data{$what}->{$tag} .= "\n$content";
- $data{$what}->{$tag} =~ s/\n+$//;
- next;
+ # New "tag" is actually part of
+ # description. Don't consider it a tag
+ $new_tag = "";
} else {
parse_error($file, $ln, "tag '$tag' is invalid", $_);
}
}

if ($new_tag =~ m/what/) {
+ $space = "";
if ($tag =~ m/what/) {
$what .= ", " . $content;
} else {
+ parse_error($file, $ln, "What '$what' doesn't have a description", "") if ($what && !$data{$what}->{description});
+
$what = $content;
$new_what = 1;
}
@@ -108,25 +113,38 @@ sub parse_abi {
next;
}

- $tag = $new_tag;
+ if ($new_tag) {
+ $tag = $new_tag;

- if ($new_what) {
- $new_what = 0;
+ if ($new_what) {
+ $new_what = 0;

- $data{$what}->{type} = $type;
- $data{$what}->{file} = $name;
- print STDERR "\twhat: $what\n" if ($debug > 1);
- }
+ $data{$what}->{type} = $type;
+ $data{$what}->{file} = $name;
+ print STDERR "\twhat: $what\n" if ($debug > 1);
+ }

- if (!$what) {
- parse_error($file, $ln, "'What:' should come first:", $_);
+ if (!$what) {
+ parse_error($file, $ln, "'What:' should come first:", $_);
+ next;
+ }
+ if ($tag eq "description") {
+ next if ($content =~ m/^\s*$/);
+ if ($content =~ m/^(\s*)(.*)/) {
+ my $new_content = $2;
+ $space = $new_tag . $sep . $1;
+ while ($space =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
+ $space =~ s/./ /g;
+ $data{$what}->{$tag} .= "$new_content\n";
+ }
+ } else {
+ $data{$what}->{$tag} = $content;
+ }
next;
}
- $data{$what}->{$tag} = $content;
- next;
}

- # Store any contents before the database
+ # Store any contents before tags at the database
if (!$tag) {
next if (/^\n/);

@@ -139,6 +157,32 @@ sub parse_abi {
next;
}

+ if ($tag eq "description") {
+ if (!$data{$what}->{description}) {
+ next if (m/^\s*\n/);
+ if (m/^(\s*)(.*)/) {
+ $space = $1;
+ while ($space =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
+ $data{$what}->{$tag} .= "$2\n";
+ }
+ } else {
+ my $content = $_;
+ if (m/^\s*\n/) {
+ $data{$what}->{$tag} .= $content;
+ next;
+ }
+
+ while ($content =~ s/\t+/' ' x (length($&) * 8 - length($`) % 8)/e) {}
+ $space = "" if (!($content =~ s/^($space)//));
+
+ # Compress spaces with tabs
+ $content =~ s<^ {8}> <\t>;
+ $content =~ s<^ {1,7}\t> <\t>;
+ $content =~ s< {1,7}\t> <\t>;
+ $data{$what}->{$tag} .= $content;
+ }
+ next;
+ }
if (m/^\s*(.*)/) {
$data{$what}->{$tag} .= "\n$1";
$data{$what}->{$tag} =~ s/\n+$//;
@@ -165,6 +209,9 @@ sub output_rest {
my $w = $what;
$w =~ s/([\_\-\*\=\^\~\\])/\\$1/g;

+ my $bar = $w;
+ $bar =~ s/./-/g;
+
if ($data{$what}->{label}) {
my @labels = split(/\s/, $data{$what}->{label});
foreach my $label (@labels) {
@@ -172,10 +219,9 @@ sub output_rest {
}
}

- print "$w\n\n";
+ print "$w\n$bar\n\n";

print "- defined on file $file (type: $type)\n\n" if ($type ne "File");
- print "::\n\n";

my $desc = $data{$what}->{description};
$desc =~ s/^\s+//;
@@ -183,18 +229,24 @@ sub output_rest {
# Remove title markups from the description, as they won't work
$desc =~ s/\n[\-\*\=\^\~]+\n/\n/g;

- # put everything inside a code block
- $desc =~ s/\n/\n /g;
-
-
if (!($desc =~ /^\s*$/)) {
- print " $desc\n\n";
+ if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
+ # put everything inside a code block
+ $desc =~ s/\n/\n /g;
+
+ print "::\n\n";
+ print " $desc\n\n";
+ } else {
+ # Escape any special chars from description
+ $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
+
+ print "$desc\n\n";
+ }
} else {
- print " DESCRIPTION MISSING for $what\n\n";
+ print "DESCRIPTION MISSING for $what\n\n";
}

printf "Has the following ABI:\n\n%s", $data{$what}->{xrefs} if ($data{$what}->{xrefs});
-
}
}

--
2.21.0

2019-06-14 07:21:20

by Andrew Donnellan

[permalink] [raw]

Subject: Re: [PATCH 01/14] ABI: fix some syntax issues at the ABI database

On 14/6/19 12:04 pm, Mauro Carvalho Chehab wrote:
> diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl
> index bbbabffc682a..fc7c6f7c21b3 100644
> --- a/Documentation/ABI/testing/sysfs-class-cxl
> +++ b/Documentation/ABI/testing/sysfs-class-cxl
> @@ -1,6 +1,6 @@
> -Note: Attributes that are shared between devices are stored in the directory
> -pointed to by the symlink device/.
> -Example: The real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
> +Please notice that attributes that are shared between devices are stored in

Would prefer "Please note" over "Please notice".

Acked-by: Andrew Donnellan <[email protected]> # cxl

--
Andrew Donnellan OzLabs, ADL Canberra
[email protected] IBM Australia Limited

2019-06-14 08:32:40

by Rafael J. Wysocki

[permalink] [raw]

Subject: Re: [PATCH 01/14] ABI: fix some syntax issues at the ABI database

On Fri, Jun 14, 2019 at 4:04 AM Mauro Carvalho Chehab
<[email protected]> wrote:
>
> From: Mauro Carvalho Chehab <[email protected]>
>
> On those three files, the ABI representation described at
> README are violated.
>
> - at sysfs-bus-iio-proximity-as3935:
> a ':' character is missing after "What"
>
> - at sysfs-class-devfreq:
> there's a typo at Description
>
> - at sysfs-class-cxl, it is using the ":" character at a
> file preamble, causing it to be misinterpreted as a
> tag.
>
> - On the other files, instead of "What", they use "Where".
>
> Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> Signed-off-by: Mauro Carvalho Chehab <[email protected]>

Acked-by: Rafael J. Wysocki <[email protected]>

> ---
> Documentation/ABI/testing/pstore | 2 +-
> .../sysfs-bus-event_source-devices-format | 2 +-
> .../ABI/testing/sysfs-bus-i2c-devices-hm6352 | 6 ++---
> .../ABI/testing/sysfs-bus-iio-distance-srf08 | 4 ++--
> .../testing/sysfs-bus-iio-proximity-as3935 | 4 ++--
> .../ABI/testing/sysfs-bus-pci-devices-cciss | 22 +++++++++----------
> .../testing/sysfs-bus-usb-devices-usbsevseg | 12 +++++-----
> Documentation/ABI/testing/sysfs-class-cxl | 6 ++---
> Documentation/ABI/testing/sysfs-class-devfreq | 2 +-
> .../ABI/testing/sysfs-class-powercap | 2 +-
> Documentation/ABI/testing/sysfs-kernel-fscaps | 2 +-
> .../ABI/testing/sysfs-kernel-vmcoreinfo | 2 +-
> 12 files changed, 33 insertions(+), 33 deletions(-)
>
> diff --git a/Documentation/ABI/testing/pstore b/Documentation/ABI/testing/pstore
> index 5fca9f5e10a3..8d6e48f4e8ef 100644
> --- a/Documentation/ABI/testing/pstore
> +++ b/Documentation/ABI/testing/pstore
> @@ -1,4 +1,4 @@
> -Where: /sys/fs/pstore/... (or /dev/pstore/...)
> +What: /sys/fs/pstore/... (or /dev/pstore/...)
> Date: March 2011
> Kernel Version: 2.6.39
> Contact: [email protected]
> diff --git a/Documentation/ABI/testing/sysfs-bus-event_source-devices-format b/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
> index 77f47ff5ee02..b6f8748e0200 100644
> --- a/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
> +++ b/Documentation/ABI/testing/sysfs-bus-event_source-devices-format
> @@ -1,4 +1,4 @@
> -Where: /sys/bus/event_source/devices/<dev>/format
> +What: /sys/bus/event_source/devices/<dev>/format
> Date: January 2012
> Kernel Version: 3.3
> Contact: Jiri Olsa <[email protected]>
> diff --git a/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352 b/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
> index feb2e4a87075..29bd447e50a0 100644
> --- a/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
> +++ b/Documentation/ABI/testing/sysfs-bus-i2c-devices-hm6352
> @@ -1,18 +1,18 @@
> -Where: /sys/bus/i2c/devices/.../heading0_input
> +What: /sys/bus/i2c/devices/.../heading0_input
> Date: April 2010
> Kernel Version: 2.6.36?
> Contact: [email protected]
> Description: Reports the current heading from the compass as a floating
> point value in degrees.
>
> -Where: /sys/bus/i2c/devices/.../power_state
> +What: /sys/bus/i2c/devices/.../power_state
> Date: April 2010
> Kernel Version: 2.6.36?
> Contact: [email protected]
> Description: Sets the power state of the device. 0 sets the device into
> sleep mode, 1 wakes it up.
>
> -Where: /sys/bus/i2c/devices/.../calibration
> +What: /sys/bus/i2c/devices/.../calibration
> Date: April 2010
> Kernel Version: 2.6.36?
> Contact: [email protected]
> diff --git a/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08 b/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
> index 0a1ca1487fa9..a133fd8d081a 100644
> --- a/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
> +++ b/Documentation/ABI/testing/sysfs-bus-iio-distance-srf08
> @@ -1,4 +1,4 @@
> -What /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
> +What: /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
> Date: January 2017
> KernelVersion: 4.11
> Contact: [email protected]
> @@ -6,7 +6,7 @@ Description:
> Show or set the gain boost of the amp, from 0-31 range.
> default 31
>
> -What /sys/bus/iio/devices/iio:deviceX/sensor_max_range
> +What: /sys/bus/iio/devices/iio:deviceX/sensor_max_range
> Date: January 2017
> KernelVersion: 4.11
> Contact: [email protected]
> diff --git a/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935 b/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
> index 9a17ab5036a4..c59d95346341 100644
> --- a/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
> +++ b/Documentation/ABI/testing/sysfs-bus-iio-proximity-as3935
> @@ -1,4 +1,4 @@
> -What /sys/bus/iio/devices/iio:deviceX/in_proximity_input
> +What: /sys/bus/iio/devices/iio:deviceX/in_proximity_input
> Date: March 2014
> KernelVersion: 3.15
> Contact: Matt Ranostay <[email protected]>
> @@ -6,7 +6,7 @@ Description:
> Get the current distance in meters of storm (1km steps)
> 1000-40000 = distance in meters
>
> -What /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
> +What: /sys/bus/iio/devices/iio:deviceX/sensor_sensitivity
> Date: March 2014
> KernelVersion: 3.15
> Contact: Matt Ranostay <[email protected]>
> diff --git a/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss b/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
> index 53d99edd1d75..eb449169c30b 100644
> --- a/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
> +++ b/Documentation/ABI/testing/sysfs-bus-pci-devices-cciss
> @@ -1,66 +1,66 @@
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/model
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/model
> Date: March 2009
> Kernel Version: 2.6.30
> Contact: [email protected]
> Description: Displays the SCSI INQUIRY page 0 model for logical drive
> Y of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/rev
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/rev
> Date: March 2009
> Kernel Version: 2.6.30
> Contact: [email protected]
> Description: Displays the SCSI INQUIRY page 0 revision for logical
> drive Y of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/unique_id
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/unique_id
> Date: March 2009
> Kernel Version: 2.6.30
> Contact: [email protected]
> Description: Displays the SCSI INQUIRY page 83 serial number for logical
> drive Y of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/vendor
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/vendor
> Date: March 2009
> Kernel Version: 2.6.30
> Contact: [email protected]
> Description: Displays the SCSI INQUIRY page 0 vendor for logical drive
> Y of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/block:cciss!cXdY
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/block:cciss!cXdY
> Date: March 2009
> Kernel Version: 2.6.30
> Contact: [email protected]
> Description: A symbolic link to /sys/block/cciss!cXdY
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/rescan
> +What: /sys/bus/pci/devices/<dev>/ccissX/rescan
> Date: August 2009
> Kernel Version: 2.6.31
> Contact: [email protected]
> Description: Kicks of a rescan of the controller to discover logical
> drive topology changes.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/lunid
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/lunid
> Date: August 2009
> Kernel Version: 2.6.31
> Contact: [email protected]
> Description: Displays the 8-byte LUN ID used to address logical
> drive Y of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/raid_level
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/raid_level
> Date: August 2009
> Kernel Version: 2.6.31
> Contact: [email protected]
> Description: Displays the RAID level of logical drive Y of
> controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/cXdY/usage_count
> +What: /sys/bus/pci/devices/<dev>/ccissX/cXdY/usage_count
> Date: August 2009
> Kernel Version: 2.6.31
> Contact: [email protected]
> Description: Displays the usage count (number of opens) of logical drive Y
> of controller X.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/resettable
> +What: /sys/bus/pci/devices/<dev>/ccissX/resettable
> Date: February 2011
> Kernel Version: 2.6.38
> Contact: [email protected]
> @@ -71,7 +71,7 @@ Description: Value of 1 indicates the controller can honor the reset_devices
> a dump device, as kdump requires resetting the device in order
> to work reliably.
>
> -Where: /sys/bus/pci/devices/<dev>/ccissX/transport_mode
> +What: /sys/bus/pci/devices/<dev>/ccissX/transport_mode
> Date: July 2011
> Kernel Version: 3.0
> Contact: [email protected]
> diff --git a/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg b/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
> index 70d00dfa443d..f6199b314196 100644
> --- a/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
> +++ b/Documentation/ABI/testing/sysfs-bus-usb-devices-usbsevseg
> @@ -1,12 +1,12 @@
> -Where: /sys/bus/usb/.../powered
> +What: /sys/bus/usb/.../powered
> Date: August 2008
> Kernel Version: 2.6.26
> Contact: Harrison Metzger <[email protected]>
> Description: Controls whether the device's display will powered.
> A value of 0 is off and a non-zero value is on.
>
> -Where: /sys/bus/usb/.../mode_msb
> -Where: /sys/bus/usb/.../mode_lsb
> +What: /sys/bus/usb/.../mode_msb
> +What: /sys/bus/usb/.../mode_lsb
> Date: August 2008
> Kernel Version: 2.6.26
> Contact: Harrison Metzger <[email protected]>
> @@ -16,7 +16,7 @@ Description: Controls the devices display mode.
> for an 8 character display the values are
> MSB 0x08; LSB 0xFF.
>
> -Where: /sys/bus/usb/.../textmode
> +What: /sys/bus/usb/.../textmode
> Date: August 2008
> Kernel Version: 2.6.26
> Contact: Harrison Metzger <[email protected]>
> @@ -25,13 +25,13 @@ Description: Controls the way the device interprets its text buffer.
> hex: each character is between 0-15
> ascii: each character is between '0'-'9' and 'A'-'F'.
>
> -Where: /sys/bus/usb/.../text
> +What: /sys/bus/usb/.../text
> Date: August 2008
> Kernel Version: 2.6.26
> Contact: Harrison Metzger <[email protected]>
> Description: The text (or data) for the device to display
>
> -Where: /sys/bus/usb/.../decimals
> +What: /sys/bus/usb/.../decimals
> Date: August 2008
> Kernel Version: 2.6.26
> Contact: Harrison Metzger <[email protected]>
> diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl
> index bbbabffc682a..fc7c6f7c21b3 100644
> --- a/Documentation/ABI/testing/sysfs-class-cxl
> +++ b/Documentation/ABI/testing/sysfs-class-cxl
> @@ -1,6 +1,6 @@
> -Note: Attributes that are shared between devices are stored in the directory
> -pointed to by the symlink device/.
> -Example: The real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
> +Please notice that attributes that are shared between devices are stored in
> +the directory pointed to by the symlink device/.
> +For example, the real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
> /sys/class/cxl/afu0.0s/device/irqs_max, i.e. /sys/class/cxl/afu0.0/irqs_max.
>
>
> diff --git a/Documentation/ABI/testing/sysfs-class-devfreq b/Documentation/ABI/testing/sysfs-class-devfreq
> index ee39acacf6f8..01196e19afca 100644
> --- a/Documentation/ABI/testing/sysfs-class-devfreq
> +++ b/Documentation/ABI/testing/sysfs-class-devfreq
> @@ -47,7 +47,7 @@ Description:
> What: /sys/class/devfreq/.../trans_stat
> Date: October 2012
> Contact: MyungJoo Ham <[email protected]>
> -Descrtiption:
> +Description:
> This ABI shows the statistics of devfreq behavior on a
> specific device. It shows the time spent in each state and
> the number of transitions between states.
> diff --git a/Documentation/ABI/testing/sysfs-class-powercap b/Documentation/ABI/testing/sysfs-class-powercap
> index db3b3ff70d84..f333a0ccc29b 100644
> --- a/Documentation/ABI/testing/sysfs-class-powercap
> +++ b/Documentation/ABI/testing/sysfs-class-powercap
> @@ -147,6 +147,6 @@ What: /sys/class/powercap/.../<power zone>/enabled
> Date: September 2013
> KernelVersion: 3.13
> Contact: [email protected]
> -Description
> +Description:
> This allows to enable/disable power capping at power zone level.
> This applies to current power zone and its children.
> diff --git a/Documentation/ABI/testing/sysfs-kernel-fscaps b/Documentation/ABI/testing/sysfs-kernel-fscaps
> index 50a3033b5e15..bcff34665192 100644
> --- a/Documentation/ABI/testing/sysfs-kernel-fscaps
> +++ b/Documentation/ABI/testing/sysfs-kernel-fscaps
> @@ -2,7 +2,7 @@ What: /sys/kernel/fscaps
> Date: February 2011
> KernelVersion: 2.6.38
> Contact: Ludwig Nussel <[email protected]>
> -Description
> +Description:
> Shows whether file system capabilities are honored
> when executing a binary
>
> diff --git a/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo b/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
> index 7bd81168e063..1f1087a5f075 100644
> --- a/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
> +++ b/Documentation/ABI/testing/sysfs-kernel-vmcoreinfo
> @@ -4,7 +4,7 @@ KernelVersion: 2.6.24
> Contact: Ken'ichi Ohmichi <[email protected]>
> Kexec Mailing List <[email protected]>
> Vivek Goyal <[email protected]>
> -Description
> +Description:
> Shows physical address and size of vmcoreinfo ELF note.
> First value contains physical address of note in hex and
> second value contains the size of note in hex. This ELF
> --
> 2.21.0
>

2019-06-14 13:31:06

[permalink] [raw]

Subject: Re: [PATCH 05/14] scripts: add an script to parse the ABI files

On Thu, 13 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> From: Mauro Carvalho Chehab <[email protected]>
>
> Add a script to parse the Documentation/ABI files and produce
> an output with all entries inside an ABI (sub)directory.
>
> Right now, it outputs its contents on ReST format. It shouldn't
> be hard to make it produce other kind of outputs, since the ABI
> file parser is implemented in separate than the output generator.

Hum, or just convert the ABI files to rst directly.

BR,
Jani.

>
> Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> ---
> scripts/get_abi.pl | 212 +++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 212 insertions(+)
> create mode 100755 scripts/get_abi.pl
>
> diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
> new file mode 100755
> index 000000000000..f7c9944a833c
> --- /dev/null
> +++ b/scripts/get_abi.pl
> @@ -0,0 +1,212 @@
> +#!/usr/bin/perl
> +
> +use strict;
> +use Pod::Usage;
> +use Getopt::Long;
> +use File::Find;
> +use Fcntl ':mode';
> +
> +my $help;
> +my $man;
> +my $debug;
> +
> +GetOptions(
> + "debug|d+" => \$debug,
> + 'help|?' => \$help,
> + man => \$man
> +) or pod2usage(2);
> +
> +pod2usage(1) if $help;
> +pod2usage(-exitstatus => 0, -verbose => 2) if $man;
> +
> +pod2usage(2) if (scalar @ARGV != 1);
> +
> +my ($prefix) = @ARGV;
> +
> +require Data::Dumper if ($debug);
> +
> +my %data;
> +
> +#
> +# Displays an error message, printing file name and line
> +#
> +sub parse_error($$$$) {
> + my ($file, $ln, $msg, $data) = @_;
> +
> + print STDERR "file $file#$ln: $msg at\n\t$data";
> +}
> +
> +#
> +# Parse an ABI file, storing its contents at %data
> +#
> +sub parse_abi {
> + my $file = $File::Find::name;
> +
> + my $mode = (stat($file))[2];
> + return if ($mode & S_IFDIR);
> + return if ($file =~ m,/README,);
> +
> + my $name = $file;
> + $name =~ s,.*/,,;
> +
> + my $type = $file;
> + $type =~ s,.*/(.*)/.*,$1,;
> +
> + my $what;
> + my $new_what;
> + my $tag;
> + my $ln;
> +
> + print STDERR "Opening $file\n" if ($debug > 1);
> + open IN, $file;
> + while(<IN>) {
> + $ln++;
> + if (m/^(\S+):\s*(.*)/i) {
> + my $new_tag = lc($1);
> + my $content = $2;
> +
> + if (!($new_tag =~ m/(what|date|kernelversion|contact|description|users)/)) {
> + if ($tag eq "description") {
> + $data{$what}->{$tag} .= "\n$content";;
> + $data{$what}->{$tag} =~ s/\n+$//;
> + next;
> + } else {
> + parse_error($file, $ln, "tag '$tag' is invalid", $_);
> + }
> + }
> +
> + if ($new_tag =~ m/what/) {
> + if ($tag =~ m/what/) {
> + $what .= ", " . $content;
> + } else {
> + $what = $content;
> + $new_what = 1;
> + }
> + $tag = $new_tag;
> + next;
> + }
> +
> + $tag = $new_tag;
> +
> + if ($new_what) {
> + $new_what = 0;
> +
> + $data{$what}->{type} = $type;
> + $data{$what}->{file} = $name;
> + print STDERR "\twhat: $what\n" if ($debug > 1);
> + }
> +
> + if (!$what) {
> + parse_error($file, $ln, "'What:' should come first:", $_);
> + next;
> + }
> + $data{$what}->{$tag} = $content;
> + next;
> + }
> +
> + # Silently ignore any headers before the database
> + next if (!$tag);
> +
> + if (m/^\s*(.*)/) {
> + $data{$what}->{$tag} .= "\n$1";
> + $data{$what}->{$tag} =~ s/\n+$//;
> + next;
> + }
> +
> + # Everything else is error
> + parse_error($file, $ln, "Unexpected line:", $_);
> + }
> + close IN;
> +}
> +
> +# Outputs the output on ReST format
> +sub output_rest {
> + foreach my $what (sort keys %data) {
> + my $type = $data{$what}->{type};
> + my $file = $data{$what}->{file};
> +
> + my $w = $what;
> + $w =~ s/([\_\-\*\=\^\~\\])/\\$1/g;
> +
> + print "$w\n\n";
> + print "- defined on file $file (type: $type)\n\n::\n\n";
> +
> + my $desc = $data{$what}->{description};
> + $desc =~ s/^\s+//;
> +
> + # Remove title markups from the description, as they won't work
> + $desc =~ s/\n[\-\*\=\^\~]+\n/\n/g;
> +
> + # put everything inside a code block
> + $desc =~ s/\n/\n /g;
> +
> +
> + if (!($desc =~ /^\s*$/)) {
> + print " $desc\n\n";
> + } else {
> + print " DESCRIPTION MISSING\n\n";
> + }
> + }
> +}
> +
> +#
> +# Parses all ABI files located at $prefix dir
> +#
> +find({wanted =>\&parse_abi, no_chdir => 1}, $prefix);
> +
> +print STDERR Data::Dumper->Dump([\%data], [qw(*data)]) if ($debug);
> +
> +#
> +# Outputs an ReST file with the ABI contents
> +#
> +output_rest
> +
> +
> +__END__
> +
> +=head1 NAME
> +
> +abi_book.pl - parse the Linux ABI files and produce a ReST book.
> +
> +=head1 SYNOPSIS
> +
> +B<abi_book.pl> [--debug] <ABI_DIR>]
> +
> +=head1 OPTIONS
> +
> +=over 8
> +
> +=item B<--debug>
> +
> +Put the script in verbose mode, useful for debugging. Can be called multiple
> +times, to increase verbosity.
> +
> +=item B<--help>
> +
> +Prints a brief help message and exits.
> +
> +=item B<--man>
> +
> +Prints the manual page and exits.
> +
> +=back
> +
> +=head1 DESCRIPTION
> +
> +Parse the Linux ABI files from ABI DIR (usually located at Documentation/ABI)
> +and produce a ReST book containing the Linux ABI.
> +
> +=head1 BUGS
> +
> +Report bugs to Mauro Carvalho Chehab <[email protected]>
> +
> +=head1 COPYRIGHT
> +
> +Copyright (c) 2016 by Mauro Carvalho Chehab <[email protected]>.
> +
> +License GPLv2: GNU GPL version 2 <http://gnu.org/licenses/gpl.html>.
> +
> +This is free software: you are free to change and redistribute it.
> +There is NO WARRANTY, to the extent permitted by law.
> +
> +=cut

--
Jani Nikula, Intel Open Source Graphics Center

2019-06-14 13:40:16

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

On Thu, 13 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> From: Mauro Carvalho Chehab <[email protected]>
>
> As we don't want a generic Sphinx extension to execute commands,
> change the one proposed to Markus to call the abi_book.pl
> script.
>
> Use a script to parse the Documentation/ABI directory and output
> it at the admin-guide.

We had a legacy kernel-doc perl script so we used that instead of
rewriting it in python. Just to keep it bug-for-bug compatible with the
past. That was the only reason.

I see absolutely zero reason to add a new perl monstrosity with a python
extension to call it. All of this could be better done in python,
directly.

Please don't complicate the documentation build. I know you know we all
worked hard to take apart the old DocBook Rube Goldberg machine to
replace it with Sphinx. Please don't turn the Sphinx build to another
complicated mess.

My strong preferences are, in this order:

1) Convert the ABI documentation to reStructuredText

2) Have the python extension read the ABI files directly, without an
extra pipeline.

BR,
Jani.

>
> Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> ---
> Documentation/admin-guide/abi-obsolete.rst | 10 ++
> Documentation/admin-guide/abi-removed.rst | 4 +
> Documentation/admin-guide/abi-stable.rst | 13 ++
> Documentation/admin-guide/abi-testing.rst | 19 +++
> Documentation/admin-guide/abi.rst | 11 ++
> Documentation/admin-guide/index.rst | 1 +
> Documentation/conf.py | 2 +-
> Documentation/sphinx/kernel_abi.py | 155 +++++++++++++++++++++
> 8 files changed, 214 insertions(+), 1 deletion(-)
> create mode 100644 Documentation/admin-guide/abi-obsolete.rst
> create mode 100644 Documentation/admin-guide/abi-removed.rst
> create mode 100644 Documentation/admin-guide/abi-stable.rst
> create mode 100644 Documentation/admin-guide/abi-testing.rst
> create mode 100644 Documentation/admin-guide/abi.rst
> create mode 100644 Documentation/sphinx/kernel_abi.py
>
> diff --git a/Documentation/admin-guide/abi-obsolete.rst b/Documentation/admin-guide/abi-obsolete.rst
> new file mode 100644
> index 000000000000..cda9168445a5
> --- /dev/null
> +++ b/Documentation/admin-guide/abi-obsolete.rst
> @@ -0,0 +1,10 @@
> +ABI obsolete symbols
> +====================
> +
> +Documents interfaces that are still remaining in the kernel, but are
> +marked to be removed at some later point in time.
> +
> +The description of the interface will document the reason why it is
> +obsolete and when it can be expected to be removed.
> +
> +.. kernel-abi:: $srctree/Documentation/ABI/obsolete
> diff --git a/Documentation/admin-guide/abi-removed.rst b/Documentation/admin-guide/abi-removed.rst
> new file mode 100644
> index 000000000000..497978fc9632
> --- /dev/null
> +++ b/Documentation/admin-guide/abi-removed.rst
> @@ -0,0 +1,4 @@
> +ABI removed symbols
> +===================
> +
> +.. kernel-abi:: $srctree/Documentation/ABI/removed
> diff --git a/Documentation/admin-guide/abi-stable.rst b/Documentation/admin-guide/abi-stable.rst
> new file mode 100644
> index 000000000000..7495d7a35048
> --- /dev/null
> +++ b/Documentation/admin-guide/abi-stable.rst
> @@ -0,0 +1,13 @@
> +ABI stable symbols
> +==================
> +
> +Documents the interfaces that the developer has defined to be stable.
> +
> +Userspace programs are free to use these interfaces with no
> +restrictions, and backward compatibility for them will be guaranteed
> +for at least 2 years.
> +
> +Most interfaces (like syscalls) are expected to never change and always
> +be available.
> +
> +.. kernel-abi:: $srctree/Documentation/ABI/stable
> diff --git a/Documentation/admin-guide/abi-testing.rst b/Documentation/admin-guide/abi-testing.rst
> new file mode 100644
> index 000000000000..5c886fc50b9e
> --- /dev/null
> +++ b/Documentation/admin-guide/abi-testing.rst
> @@ -0,0 +1,19 @@
> +ABI testing symbols
> +===================
> +
> +Documents interfaces that are felt to be stable,
> +as the main development of this interface has been completed.
> +
> +The interface can be changed to add new features, but the
> +current interface will not break by doing this, unless grave
> +errors or security problems are found in them.
> +
> +Userspace programs can start to rely on these interfaces, but they must
> +be aware of changes that can occur before these interfaces move to
> +be marked stable.
> +
> +Programs that use these interfaces are strongly encouraged to add their
> +name to the description of these interfaces, so that the kernel
> +developers can easily notify them if any changes occur.
> +
> +.. kernel-abi:: $srctree/Documentation/ABI/testing
> diff --git a/Documentation/admin-guide/abi.rst b/Documentation/admin-guide/abi.rst
> new file mode 100644
> index 000000000000..3b9645c77469
> --- /dev/null
> +++ b/Documentation/admin-guide/abi.rst
> @@ -0,0 +1,11 @@
> +=====================
> +Linux ABI description
> +=====================
> +
> +.. toctree::
> + :maxdepth: 1
> +
> + abi-stable
> + abi-testing
> + abi-obsolete
> + abi-removed
> diff --git a/Documentation/admin-guide/index.rst b/Documentation/admin-guide/index.rst
> index 8001917ee012..20c3020fd73c 100644
> --- a/Documentation/admin-guide/index.rst
> +++ b/Documentation/admin-guide/index.rst
> @@ -16,6 +16,7 @@ etc.
> README
> kernel-parameters
> devices
> + abi
>
> This section describes CPU vulnerabilities and their mitigations.
>
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index 7ace3f8852bd..598256fb5c98 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -34,7 +34,7 @@ needs_sphinx = '1.3'
> # Add any Sphinx extension module names here, as strings. They can be
> # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
> # ones.
> -extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig']
> +extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 'kfigure', 'sphinx.ext.ifconfig', 'kernel_abi']
>
> # The name of the math extension changed on Sphinx 1.4
> if (major == 1 and minor > 3) or (major > 1):
> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> new file mode 100644
> index 000000000000..32ce90775d96
> --- /dev/null
> +++ b/Documentation/sphinx/kernel_abi.py
> @@ -0,0 +1,155 @@
> +# -*- coding: utf-8; mode: python -*-
> +u"""
> + kernel-abi
> + ~~~~~~~~~~
> +
> + Implementation of the ``kernel-abi`` reST-directive.
> +
> + :copyright: Copyright (C) 2016 Markus Heiser
> + :copyright: Copyright (C) 2016 Mauro Carvalho Chehab
> + :license: GPL Version 2, June 1991 see Linux/COPYING for details.
> +
> + The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
> + scripts/get_abi.pl script to parse the Kernel ABI files.
> +
> + Overview of directive's argument and options.
> +
> + .. code-block:: rst
> +
> + .. kernel-abi:: <ABI directory location>
> + :debug:
> +
> + The argument ``<ABI directory location>`` is required. It contains the
> + location of the ABI files to be parsed.
> +
> + ``debug``
> + Inserts a code-block with the *raw* reST. Sometimes it is helpful to see
> + what reST is generated.
> +
> +"""
> +
> +import sys
> +import os
> +from os import path
> +import subprocess
> +
> +from sphinx.ext.autodoc import AutodocReporter
> +
> +from docutils import nodes
> +from docutils.parsers.rst import Directive, directives
> +from docutils.statemachine import ViewList
> +from docutils.utils.error_reporting import ErrorString
> +
> +
> +__version__ = '1.0'
> +
> +# We can't assume that six is installed
> +PY3 = sys.version_info[0] == 3
> +PY2 = sys.version_info[0] == 2
> +if PY3:
> + # pylint: disable=C0103, W0622
> + unicode = str
> + basestring = str
> +
> +def setup(app):
> +
> + app.add_directive("kernel-abi", KernelCmd)
> + return dict(
> + version = __version__
> + , parallel_read_safe = True
> + , parallel_write_safe = True
> + )
> +
> +class KernelCmd(Directive):
> +
> + u"""KernelABI (``kernel-abi``) directive"""
> +
> + required_arguments = 1
> + optional_arguments = 0
> + has_content = False
> + final_argument_whitespace = True
> +
> + option_spec = {
> + "debug" : directives.flag
> + }
> +
> + def warn(self, message, **replace):
> + replace["fname"] = self.state.document.current_source
> + replace["line_no"] = replace.get("line_no", self.lineno)
> + message = ("%(fname)s:%(line_no)s: [kernel-abi WARN] : " + message) % replace
> + self.state.document.settings.env.app.warn(message, prefix="")
> +
> + def run(self):
> +
> + doc = self.state.document
> + if not doc.settings.file_insertion_enabled:
> + raise self.warning("docutils: file insertion disabled")
> +
> + env = doc.settings.env
> + cwd = path.dirname(doc.current_source)
> + cmd = "get_abi.pl rest --dir "
> + cmd += self.arguments[0]
> +
> + srctree = path.abspath(os.environ["srctree"])
> +
> + fname = cmd
> +
> + # extend PATH with $(srctree)/scripts
> + path_env = os.pathsep.join([
> + srctree + os.sep + "scripts",
> + os.environ["PATH"]
> + ])
> + shell_env = os.environ.copy()
> + shell_env["PATH"] = path_env
> + shell_env["srctree"] = srctree
> +
> + lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
> + nodeList = self.nestedParse(lines, fname)
> + return nodeList
> +
> + def runCmd(self, cmd, **kwargs):
> + u"""Run command ``cmd`` and return it's stdout as unicode."""
> +
> + try:
> + proc = subprocess.Popen(
> + cmd
> + , stdout = subprocess.PIPE
> + , stderr = subprocess.PIPE
> + , universal_newlines = True
> + , **kwargs
> + )
> + out, err = proc.communicate()
> + if err:
> + self.warn(err)
> + if proc.returncode != 0:
> + raise self.severe(
> + u"command '%s' failed with return code %d"
> + % (cmd, proc.returncode)
> + )
> + except OSError as exc:
> + raise self.severe(u"problems with '%s' directive: %s."
> + % (self.name, ErrorString(exc)))
> + return unicode(out)
> +
> + def nestedParse(self, lines, fname):
> + content = ViewList()
> + node = nodes.section()
> +
> + if "debug" in self.options:
> + code_block = "\n\n.. code-block:: rst\n :linenos:\n"
> + for l in lines.split("\n"):
> + code_block += "\n " + l
> + lines = code_block + "\n\n"
> +
> + for c, l in enumerate(lines.split("\n")):
> + content.append(l, fname, c)
> +
> + buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
> + self.state.memo.title_styles = []
> + self.state.memo.section_level = 0
> + self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
> + try:
> + self.state.nested_parse(content, 0, node, match_titles=1)
> + finally:
> + self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
> + return node.children

--
Jani Nikula, Intel Open Source Graphics Center

2019-06-14 13:42:05

[permalink] [raw]

Subject: Re: [PATCH 05/14] scripts: add an script to parse the ABI files

On Fri, Jun 14, 2019 at 04:31:56PM +0300, Jani Nikula wrote:
> On Thu, 13 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> > From: Mauro Carvalho Chehab <[email protected]>
> >
> > Add a script to parse the Documentation/ABI files and produce
> > an output with all entries inside an ABI (sub)directory.
> >
> > Right now, it outputs its contents on ReST format. It shouldn't
> > be hard to make it produce other kind of outputs, since the ABI
> > file parser is implemented in separate than the output generator.
>
> Hum, or just convert the ABI files to rst directly.

And what would that look like?

thanks,

greg k-h

2019-06-14 13:58:33

[permalink] [raw]

Subject: Re: [PATCH 05/14] scripts: add an script to parse the ABI files

On Fri, 14 Jun 2019, Greg Kroah-Hartman <[email protected]> wrote:
> On Fri, Jun 14, 2019 at 04:31:56PM +0300, Jani Nikula wrote:
>> On Thu, 13 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
>> > From: Mauro Carvalho Chehab <[email protected]>
>> >
>> > Add a script to parse the Documentation/ABI files and produce
>> > an output with all entries inside an ABI (sub)directory.
>> >
>> > Right now, it outputs its contents on ReST format. It shouldn't
>> > be hard to make it produce other kind of outputs, since the ABI
>> > file parser is implemented in separate than the output generator.
>>
>> Hum, or just convert the ABI files to rst directly.
>
> And what would that look like?

That pretty much depends on the requirements we want to set on both the
ABI source files and the generated output. Obviously the requirements
can be conflicting; might be hard to produce fancy output if the input
is very limited.

At the bare minimum, you could convert the files to contain
reStructuredText field lists [1]. Add a colon at the start of the field
name, and make sure field bodies (values) are not empty.

Conversion of a file selected at random; I've only added ":" and "N/A".

diff --git a/Documentation/ABI/stable/sysfs-devices-system-cpu b/Documentation/ABI/stable/sysfs-devices-system-cpu
index 33c133e2a631..34c218b344fb 100644
--- a/Documentation/ABI/stable/sysfs-devices-system-cpu
+++ b/Documentation/ABI/stable/sysfs-devices-system-cpu
@@ -1,19 +1,20 @@
-What: /sys/devices/system/cpu/dscr_default
-Date: 13-May-2014
-KernelVersion: v3.15.0
-Contact:
-Description: Writes are equivalent to writing to
+:What: /sys/devices/system/cpu/dscr_default
+:Date: 13-May-2014
+:KernelVersion: v3.15.0
+:Contact: N/A
+:Description: Writes are equivalent to writing to
/sys/devices/system/cpu/cpuN/dscr on all CPUs.
Reads return the last written value or 0.
This value is not a global default: it is a way to set
all per-CPU defaults at the same time.
-Values: 64 bit unsigned integer (bit field)
+:Values: 64 bit unsigned integer (bit field)

-What: /sys/devices/system/cpu/cpu[0-9]+/dscr
-Date: 13-May-2014
-KernelVersion: v3.15.0
-Contact:
-Description: Default value for the Data Stream Control Register (DSCR) on
+
+:What: /sys/devices/system/cpu/cpu[0-9]+/dscr
+:Date: 13-May-2014
+:KernelVersion: v3.15.0
+:Contact: N/A
+:Description: Default value for the Data Stream Control Register (DSCR) on
a CPU.
This default value is used when the kernel is executing and
for any process that has not set the DSCR itself.
@@ -22,4 +23,4 @@ Description: Default value for the Data Stream Control Register (DSCR) on
on any CPU where it executes (overriding the value described
here).
If set by a process it will be inherited by child processes.
-Values: 64 bit unsigned integer (bit field)
+:Values: 64 bit unsigned integer (bit field)

---

Of course, you'd still need to add higher level files to include the ABI
files.

At the other end, you could add structure and syntax to your heart's
content, and make the output fancier too.

BR,
Jani.

[1] http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html#field-lists

--
Jani Nikula, Intel Open Source Graphics Center

2019-06-14 14:00:11

[permalink] [raw]

Subject: Re: [PATCH 05/14] scripts: add an script to parse the ABI files

Em Fri, 14 Jun 2019 15:39:33 +0200
Greg Kroah-Hartman <[email protected]> escreveu:

> On Fri, Jun 14, 2019 at 04:31:56PM +0300, Jani Nikula wrote:
> > On Thu, 13 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> > > From: Mauro Carvalho Chehab <[email protected]>
> > >
> > > Add a script to parse the Documentation/ABI files and produce
> > > an output with all entries inside an ABI (sub)directory.
> > >
> > > Right now, it outputs its contents on ReST format. It shouldn't
> > > be hard to make it produce other kind of outputs, since the ABI
> > > file parser is implemented in separate than the output generator.
> >
> > Hum, or just convert the ABI files to rst directly.

Converting ABI files to rst could be easily done using a modified
version of my script (plus the produced files will likely need some
manual review).

There is a drawback though: we'll lose the capability of being able
to parse ABI files via an script, as people would be freed to use a
different syntax.

We could minimize it by using things like:

:What:
:Kernel Version:
...

> And what would that look like?

That's the big question :-)

If you prefer something like that, I can write the patches.

Thanks,
Mauro

2019-06-14 14:07:39

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
> On Thu, 13 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> > From: Mauro Carvalho Chehab <[email protected]>
> >
> > As we don't want a generic Sphinx extension to execute commands,
> > change the one proposed to Markus to call the abi_book.pl
> > script.
> >
> > Use a script to parse the Documentation/ABI directory and output
> > it at the admin-guide.
>
> We had a legacy kernel-doc perl script so we used that instead of
> rewriting it in python. Just to keep it bug-for-bug compatible with the
> past. That was the only reason.
>
> I see absolutely zero reason to add a new perl monstrosity with a python
> extension to call it. All of this could be better done in python,
> directly.
>
> Please don't complicate the documentation build. I know you know we all
> worked hard to take apart the old DocBook Rube Goldberg machine to
> replace it with Sphinx. Please don't turn the Sphinx build to another
> complicated mess.
>
> My strong preferences are, in this order:
>
> 1) Convert the ABI documentation to reStructuredText

What would that exactly look like? What would it require for new
developers for when they write new entries? Why not rely on a helper
script, that allows us to validate things better?

> 2) Have the python extension read the ABI files directly, without an
> extra pipeline.

He who writes the script, get's to dictate the language of the script :)

Personally, this looks sane to me, I'm going to apply the ABI fixups to
my tree at least, and then see how the script works out. The script can
always be replaced with a different one in a different language at a
later point in time of people think it really mattes.

thanks,

greg k-h

2019-06-14 14:11:43

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

Am 14.06.19 um 15:42 schrieb Jani Nikula:
> On Thu, 13 Jun 2019, Mauro Carvalho Chehab<[email protected]> wrote:
>> From: Mauro Carvalho Chehab<[email protected]>
>>
>> As we don't want a generic Sphinx extension to execute commands,
>> change the one proposed to Markus to call the abi_book.pl
>> script.
>>
>> Use a script to parse the Documentation/ABI directory and output
>> it at the admin-guide.
> We had a legacy kernel-doc perl script so we used that instead of
> rewriting it in python. Just to keep it bug-for-bug compatible with the
> past. That was the only reason.
>
> I see absolutely zero reason to add a new perl monstrosity with a python
> extension to call it. All of this could be better done in python,
> directly.
>
> Please don't complicate the documentation build. I know you know we all
> worked hard to take apart the old DocBook Rube Goldberg machine to
> replace it with Sphinx. Please don't turn the Sphinx build to another
> complicated mess.
>
> My strong preferences are, in this order:
>
> 1) Convert the ABI documentation to reStructuredText
>
> 2) Have the python extension read the ABI files directly, without an
> extra pipeline.
>

I agree with Jani. No matter how the decision ends, since I can't help here, I'd
rather not show up in the copyright.

-- Markus --

2019-06-14 14:16:16

[permalink] [raw]

Subject: Re: [PATCH 05/14] scripts: add an script to parse the ABI files

On Fri, 14 Jun 2019 17:00:55 +0300
Jani Nikula <[email protected]> wrote:

> On Fri, 14 Jun 2019, Greg Kroah-Hartman <[email protected]> wrote:
> > On Fri, Jun 14, 2019 at 04:31:56PM +0300, Jani Nikula wrote:
> >> On Thu, 13 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> >> > From: Mauro Carvalho Chehab <[email protected]>
> >> >
> >> > Add a script to parse the Documentation/ABI files and produce
> >> > an output with all entries inside an ABI (sub)directory.
> >> >
> >> > Right now, it outputs its contents on ReST format. It shouldn't
> >> > be hard to make it produce other kind of outputs, since the ABI
> >> > file parser is implemented in separate than the output generator.
> >>
> >> Hum, or just convert the ABI files to rst directly.
> >
> > And what would that look like?
>
> That pretty much depends on the requirements we want to set on both the
> ABI source files and the generated output. Obviously the requirements
> can be conflicting; might be hard to produce fancy output if the input
> is very limited.

The real question, I guess, is: is there anything else that depends on the
current format of those files? That could make reformatting them harder.

Otherwise it seems like Jani's fieldlist idea might have some potential.
We could consider a *really* simple preprocessing step if we really wanted
("prepend a colon on relevant lines and join with the following line if
need be") to keep the current format nearly unchanged.

jon

2019-06-14 14:17:40

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

On Fri, 14 Jun 2019 16:10:31 +0200
Markus Heiser <[email protected]> wrote:

> I agree with Jani. No matter how the decision ends, since I can't help here, I'd
> rather not show up in the copyright.

Is there something specific you are asking us to do here?

Thanks,

jon

2019-06-14 15:28:33

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

Em Fri, 14 Jun 2019 16:06:03 +0200
Greg Kroah-Hartman <[email protected]> escreveu:

> On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
> > On Thu, 13 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> > > From: Mauro Carvalho Chehab <[email protected]>
> > >
> > > As we don't want a generic Sphinx extension to execute commands,
> > > change the one proposed to Markus to call the abi_book.pl
> > > script.
> > >
> > > Use a script to parse the Documentation/ABI directory and output
> > > it at the admin-guide.
> >
> > We had a legacy kernel-doc perl script so we used that instead of
> > rewriting it in python. Just to keep it bug-for-bug compatible with the
> > past. That was the only reason.
> >
> > I see absolutely zero reason to add a new perl monstrosity with a python
> > extension to call it. All of this could be better done in python,
> > directly.
> >
> > Please don't complicate the documentation build. I know you know we all
> > worked hard to take apart the old DocBook Rube Goldberg machine to
> > replace it with Sphinx. Please don't turn the Sphinx build to another
> > complicated mess.
> >
> > My strong preferences are, in this order:
> >
> > 1) Convert the ABI documentation to reStructuredText
>
> What would that exactly look like? What would it require for new
> developers for when they write new entries? Why not rely on a helper
> script, that allows us to validate things better?

Funny enough, this e-mail arrived here after Greg's reply, and my
reply over his one :-)

-

With regards to the script, my idea is to have it run on a new
"validate" mode, when the Kernel is built with COMPILE_TEST:

https://git.linuxtv.org/mchehab/experimental.git/log/?h=abi_patches_v4

NB: the last patch is not yet... somehow, the building system is not
passing CONFIG_WARN_ABI_ERRORS to Documentation/Makefile. I'm
debugging it.

Personally, I would prefer to keep it the way it is, with two
additions:

1) I would add a SPDX header at the fist line of each file there;

2) It would make sense to have a new field - or indicator - to let
add ReST markups at the description.

The advantage of using a parseable ABI file is that it is possible
to parse it, for example, to search for a symbol:

$ ./scripts/get_abi.pl voltage_max

/sys/class/power_supply/<supply_name>/voltage_max
-------------------------------------------------

Date: January 2008
Contact: [email protected]
Defined on file: Documentation/ABI/testing/sysfs-class-power

Description:

Reports the maximum VBUS voltage the supply can support.

Access: Read
Valid values: Represented in microvolts

...

>
> > 2) Have the python extension read the ABI files directly, without an
> > extra pipeline.
>
> He who writes the script, get's to dictate the language of the script :)

No idea about how much time it would take if written in python,
but this perl script is really fast:

$ time ./scripts/get_abi.pl search voltage_max >/dev/null
real 0m0,139s
user 0m0,132s
sys 0m0,006s

That's the time it takes here (SSD disks) to read all files under
Documentation/ABI, parse them and seek for a string.

That's about half of the time a python script takes to just import the
the sphinx modules and print its version, running at the same machine:

$ time sphinx-build --version >/dev/null

real 0m0,224s
user 0m0,199s
sys 0m0,024s

> Personally, this looks sane to me, I'm going to apply the ABI fixups to
> my tree at least, and then see how the script works out. The script can
> always be replaced with a different one in a different language at a
> later point in time of people think it really mattes.

Thanks,
Mauro

2019-06-14 16:16:50

[permalink] [raw]

Subject: Re: [PATCH 01/14] ABI: fix some syntax issues at the ABI database

On Fri, Jun 14, 2019 at 05:20:29PM +1000, Andrew Donnellan wrote:
> On 14/6/19 12:04 pm, Mauro Carvalho Chehab wrote:
> > diff --git a/Documentation/ABI/testing/sysfs-class-cxl b/Documentation/ABI/testing/sysfs-class-cxl
> > index bbbabffc682a..fc7c6f7c21b3 100644
> > --- a/Documentation/ABI/testing/sysfs-class-cxl
> > +++ b/Documentation/ABI/testing/sysfs-class-cxl
> > @@ -1,6 +1,6 @@
> > -Note: Attributes that are shared between devices are stored in the directory
> > -pointed to by the symlink device/.
> > -Example: The real path of the attribute /sys/class/cxl/afu0.0s/irqs_max is
> > +Please notice that attributes that are shared between devices are stored in
>
> Would prefer "Please note" over "Please notice".

Now changed, but that's a minor grammer thing, both are correct. :)

> Acked-by: Andrew Donnellan <[email protected]> # cxl

thanks,

greg k-h

2019-06-14 16:19:10

[permalink] [raw]

Subject: Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support

On Thu, Jun 13, 2019 at 11:04:20PM -0300, Mauro Carvalho Chehab wrote:
> The parser breaks with UTF-8 characters with Sphinx 1.4.
>
> Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> ---
> Documentation/sphinx/kernel_abi.py | 10 ++++++----
> 1 file changed, 6 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> index 7fa7806532dc..460cee48a245 100644
> --- a/Documentation/sphinx/kernel_abi.py
> +++ b/Documentation/sphinx/kernel_abi.py
> @@ -1,4 +1,5 @@
> -# -*- coding: utf-8; mode: python -*-
> +# coding=utf-8
> +#

Is this an emacs vs. vim fight?

Why change this?

thanks,

greg k-h

2019-06-14 16:21:24

[permalink] [raw]

Subject: Re: [PATCH 00/14] Add support to generate ABI documentation at admin-guide

On Thu, Jun 13, 2019 at 11:04:06PM -0300, Mauro Carvalho Chehab wrote:
> Greg,
>
> As promised, I'm resending the patch series with adds the Kernel ABI to
> Documentation/admin-guide.
>
> Those patches are basically the version 3 patchset I sent back in 2017,
> rebased on the top of linux-next (next-20190613), and with some fixes
> in order for it to work.
>
> - The 4 initial patches to fix some ABI descriptions that are violating
> the syntax described at Documentation/ABI/README;

These 4 are now applied to my driver-core tree, thanks.

greg k-h

2019-06-14 16:25:55

[permalink] [raw]

Subject: Re: [PATCH 05/14] scripts: add an script to parse the ABI files

On Fri, Jun 14, 2019 at 05:00:55PM +0300, Jani Nikula wrote:
> On Fri, 14 Jun 2019, Greg Kroah-Hartman <[email protected]> wrote:
> > On Fri, Jun 14, 2019 at 04:31:56PM +0300, Jani Nikula wrote:
> >> On Thu, 13 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> >> > From: Mauro Carvalho Chehab <[email protected]>
> >> >
> >> > Add a script to parse the Documentation/ABI files and produce
> >> > an output with all entries inside an ABI (sub)directory.
> >> >
> >> > Right now, it outputs its contents on ReST format. It shouldn't
> >> > be hard to make it produce other kind of outputs, since the ABI
> >> > file parser is implemented in separate than the output generator.
> >>
> >> Hum, or just convert the ABI files to rst directly.
> >
> > And what would that look like?
>
> That pretty much depends on the requirements we want to set on both the
> ABI source files and the generated output. Obviously the requirements
> can be conflicting; might be hard to produce fancy output if the input
> is very limited.
>
> At the bare minimum, you could convert the files to contain
> reStructuredText field lists [1]. Add a colon at the start of the field
> name, and make sure field bodies (values) are not empty.
>
> Conversion of a file selected at random; I've only added ":" and "N/A".

N/A should be allowed to just drop the line entirely, right?

And what does this end up looking like?

I also hate "flag days" where all of a chunk of stuff needs to be
converted into another style. Also it doesn't deal with merges from the
100+ different trees that all end up adding stuff to this directory over
time (slowly though, unfortunately)

So ideally, I'd like to keep the original format if at all possible.
Having the tool here allows people to do nice things like search for a
specific file easily, or a device type, which is something that I know I
have wanted, and others have asked for in the past as well.

It also might allow us to find out where we are missing documentation, a
long-term goal of mine.

thanks,

greg k-h

2019-06-14 16:27:19

[permalink] [raw]

Subject: Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support

Em Fri, 14 Jun 2019 18:18:37 +0200
Greg Kroah-Hartman <[email protected]> escreveu:

> On Thu, Jun 13, 2019 at 11:04:20PM -0300, Mauro Carvalho Chehab wrote:
> > The parser breaks with UTF-8 characters with Sphinx 1.4.
> >
> > Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> > ---
> > Documentation/sphinx/kernel_abi.py | 10 ++++++----
> > 1 file changed, 6 insertions(+), 4 deletions(-)
> >
> > diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> > index 7fa7806532dc..460cee48a245 100644
> > --- a/Documentation/sphinx/kernel_abi.py
> > +++ b/Documentation/sphinx/kernel_abi.py
> > @@ -1,4 +1,5 @@
> > -# -*- coding: utf-8; mode: python -*-
> > +# coding=utf-8
> > +#
>
> Is this an emacs vs. vim fight?

No. This is a python-specific thing:

https://www.python.org/dev/peps/pep-0263/

>
> Why change this?

Just to keep the "header" part of the script closer to kerneldoc.py.

You may keep the previous syntax if you want, as both ways are
equally recognized, as python actually checks for anything that
matches this regex at the first or second line:

^[ \t\f]*#.*?coding[:=][ \t]*([-_.a-zA-Z0-9]+)

Thanks,
Mauro

2019-06-15 06:19:42

[permalink] [raw]

Subject: Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support

On Fri, Jun 14, 2019 at 01:25:30PM -0300, Mauro Carvalho Chehab wrote:
> Em Fri, 14 Jun 2019 18:18:37 +0200
> Greg Kroah-Hartman <[email protected]> escreveu:
>
> > On Thu, Jun 13, 2019 at 11:04:20PM -0300, Mauro Carvalho Chehab wrote:
> > > The parser breaks with UTF-8 characters with Sphinx 1.4.
> > >
> > > Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> > > ---
> > > Documentation/sphinx/kernel_abi.py | 10 ++++++----
> > > 1 file changed, 6 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> > > index 7fa7806532dc..460cee48a245 100644
> > > --- a/Documentation/sphinx/kernel_abi.py
> > > +++ b/Documentation/sphinx/kernel_abi.py
> > > @@ -1,4 +1,5 @@
> > > -# -*- coding: utf-8; mode: python -*-
> > > +# coding=utf-8
> > > +#
> >
> > Is this an emacs vs. vim fight?
>
> No. This is a python-specific thing:
>
> https://www.python.org/dev/peps/pep-0263/

Ah, thanks, didn't know that.

greg k-h

2019-06-16 15:44:33

[permalink] [raw]

Subject: Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support

Am 14.06.19 um 18:25 schrieb Mauro Carvalho Chehab:
> Em Fri, 14 Jun 2019 18:18:37 +0200
> Greg Kroah-Hartman <[email protected]> escreveu:
>
>> On Thu, Jun 13, 2019 at 11:04:20PM -0300, Mauro Carvalho Chehab wrote:
>>> The parser breaks with UTF-8 characters with Sphinx 1.4.
>>>
>>> Signed-off-by: Mauro Carvalho Chehab <[email protected]>
>>> ---
>>> Documentation/sphinx/kernel_abi.py | 10 ++++++----
>>> 1 file changed, 6 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
>>> index 7fa7806532dc..460cee48a245 100644
>>> --- a/Documentation/sphinx/kernel_abi.py
>>> +++ b/Documentation/sphinx/kernel_abi.py
>>> @@ -1,4 +1,5 @@
>>> -# -*- coding: utf-8; mode: python -*-
>>> +# coding=utf-8
>>> +#
>>
>> Is this an emacs vs. vim fight?
>
> No. This is a python-specific thing:
>
> https://www.python.org/dev/peps/pep-0263/

No need to change, the emacs notation is also OK, see your link

"""or (using formats recognized by popular editors):"""

https://www.python.org/dev/peps/pep-0263/#defining-the-encoding

I prefer emacs notation, this is also evaluated by many other editors / tools.

-- Markus --

2019-06-16 16:04:26

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

Am 14.06.19 um 16:15 schrieb Jonathan Corbet:
> On Fri, 14 Jun 2019 16:10:31 +0200
> Markus Heiser <[email protected]> wrote:
>
>> I agree with Jani. No matter how the decision ends, since I can't help here, I'd
>> rather not show up in the copyright.
>
> Is there something specific you are asking us to do here?
>

I have lost the overview, but there was a patch Mauro added a
kernel_abi.py. There was my name (Markus Heiser) listed with a
copyright notation.

I guess Mauro picked up some old RFC or an other old patch of
mine from 2016 and made some C&P .. whatever .. ATM I do not have
time to give any support on parsing ABI and I'am not interested
in holding copyrights on a C&P of a old source ;)

-- Markus --

2019-06-17 09:12:27

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

Em Sun, 16 Jun 2019 18:04:01 +0200
Markus Heiser <[email protected]> escreveu:

> Am 14.06.19 um 16:15 schrieb Jonathan Corbet:
> > On Fri, 14 Jun 2019 16:10:31 +0200
> > Markus Heiser <[email protected]> wrote:
> >
> >> I agree with Jani. No matter how the decision ends, since I can't help here, I'd
> >> rather not show up in the copyright.
> >
> > Is there something specific you are asking us to do here?
> >
>
>
> I have lost the overview, but there was a patch Mauro added a
> kernel_abi.py. There was my name (Markus Heiser) listed with a
> copyright notation.
>
> I guess Mauro picked up some old RFC or an other old patch of
> mine from 2016 and made some C&P .. whatever .. ATM I do not have
> time to give any support on parsing ABI and I'am not interested
> in holding copyrights on a C&P of a old source ;)

Well, the code was basically written by you :-)

It was written to be a script capable of running a generic
script. On that time, my contribution to it was basically
to hardcode it to run "get_abi.pl".

This came from an old branch where the last change was back in 2017.
It was resurrected due to a discussion at KS ML.

There, the discussion was related to what's left to be converted
to ReST.

While I can't simply remove your copyright, would you be happy
with something like that?

Thanks,
Mauro

diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
index 2d5d582207f7..ef91b1e1ff4b 100644
--- a/Documentation/sphinx/kernel_abi.py
+++ b/Documentation/sphinx/kernel_abi.py
@@ -7,7 +7,8 @@ u"""
Implementation of the ``kernel-abi`` reST-directive.

:copyright: Copyright (C) 2016 Markus Heiser
- :copyright: Copyright (C) 2016 Mauro Carvalho Chehab
+ :copyright: Copyright (C) 2016-2019 Mauro Carvalho Chehab
+ :maintained-by: Mauro Carvalho Chehab <[email protected]>
:license: GPL Version 2, June 1991 see Linux/COPYING for details.

The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the

2019-06-17 09:18:44

[permalink] [raw]

Subject: Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support

Em Sun, 16 Jun 2019 17:43:50 +0200
Markus Heiser <[email protected]> escreveu:

> Am 14.06.19 um 18:25 schrieb Mauro Carvalho Chehab:
> > Em Fri, 14 Jun 2019 18:18:37 +0200
> > Greg Kroah-Hartman <[email protected]> escreveu:
> >
> >> On Thu, Jun 13, 2019 at 11:04:20PM -0300, Mauro Carvalho Chehab wrote:
> >>> The parser breaks with UTF-8 characters with Sphinx 1.4.
> >>>
> >>> Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> >>> ---
> >>> Documentation/sphinx/kernel_abi.py | 10 ++++++----
> >>> 1 file changed, 6 insertions(+), 4 deletions(-)
> >>>
> >>> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> >>> index 7fa7806532dc..460cee48a245 100644
> >>> --- a/Documentation/sphinx/kernel_abi.py
> >>> +++ b/Documentation/sphinx/kernel_abi.py
> >>> @@ -1,4 +1,5 @@
> >>> -# -*- coding: utf-8; mode: python -*-
> >>> +# coding=utf-8
> >>> +#
> >>
> >> Is this an emacs vs. vim fight?
> >
> > No. This is a python-specific thing:
> >
> > https://www.python.org/dev/peps/pep-0263/
>
> No need to change, the emacs notation is also OK, see your link
>
> """or (using formats recognized by popular editors):"""
>
> https://www.python.org/dev/peps/pep-0263/#defining-the-encoding
>
> I prefer emacs notation, this is also evaluated by many other editors / tools.

The usage of emacs notation is something that we don't like at the
Linux Kernel. With ~4K developers per release, if we add tags to
every single editor people use, it would be really messy, as one
developer would be adding a tag and the next one replacing it by its
some other favorite editor's tag.

There's even an item at Documentation/process/coding-style.rst
due to that (item 9).

Thanks,
Mauro

2019-06-17 12:33:45

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

On Fri, 14 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> Em Fri, 14 Jun 2019 16:06:03 +0200
> Greg Kroah-Hartman <[email protected]> escreveu:
>
>> On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
>> > 2) Have the python extension read the ABI files directly, without an
>> > extra pipeline.
>>
>> He who writes the script, get's to dictate the language of the script :)

The point is, it's an extension to a python based tool, written in perl,
using pipes for communication, and losing any advantages of integrating
with the tool it's extending.

I doubt you'd want to see system() to be used to subsequently extend the
perl tool.

I think it's just sad to see the documentation system slowly drift
further away from the ideals we had, and towards the old ways we worked
so hard to fix.

> No idea about how much time it would take if written in python,
> but this perl script is really fast:
>
> $ time ./scripts/get_abi.pl search voltage_max >/dev/null
> real 0m0,139s
> user 0m0,132s
> sys 0m0,006s
>
> That's the time it takes here (SSD disks) to read all files under
> Documentation/ABI, parse them and seek for a string.
>
> That's about half of the time a python script takes to just import the
> the sphinx modules and print its version, running at the same machine:
>
> $ time sphinx-build --version >/dev/null
>
> real 0m0,224s
> user 0m0,199s
> sys 0m0,024s

Please at least use fair and sensible comparisons. If you want to make
the extension usable standalone on the command-line, bypassing Sphinx,
you can do that. No need to factor in Sphinx to your comparisons.

BR,
Jani.

--
Jani Nikula, Intel Open Source Graphics Center

2019-06-17 12:56:26

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

On Mon, Jun 17, 2019 at 03:36:17PM +0300, Jani Nikula wrote:
> On Fri, 14 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> > Em Fri, 14 Jun 2019 16:06:03 +0200
> > Greg Kroah-Hartman <[email protected]> escreveu:
> >
> >> On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
> >> > 2) Have the python extension read the ABI files directly, without an
> >> > extra pipeline.
> >>
> >> He who writes the script, get's to dictate the language of the script :)
>
> The point is, it's an extension to a python based tool, written in perl,
> using pipes for communication, and losing any advantages of integrating
> with the tool it's extending.
>
> I doubt you'd want to see system() to be used to subsequently extend the
> perl tool.
>
> I think it's just sad to see the documentation system slowly drift
> further away from the ideals we had, and towards the old ways we worked
> so hard to fix.

What are those ideals?

I thought the goal was to be able to write documentation in a as much
as a normal text file as possible and have automation turn those files
into "pretty" documentation that we can all use.

And I think that fits with the way this patch set goes, right? We are
not on a quest for purity of scripts to generate the documentation at
the expense of having to force hundreds, or thousands, of developers to
change their ways, or to force a "flag day" conversion of existing
documentation resulting in a huge merge mess.

So, we are stuck with the current structure that I totally made up for
Documentation/ABI/. Turns out it is almost parsable, as Mauro's tool
shows. His tool also validates the existing text, which is great, and
has caused fixes for it.

If someone wants to write that tool in some other language, like python,
wonderful, I have no objection, but as it is, this is a useful tool
already, allowing us to validate, and search, existing documentation
entries that we have never been able to do before. It also provides an
output that can be turned into pretty html/pdf/whatever files by other
tools in the pipeline, a totally bonus benefit.

So what is going backwards here?

Maybe the processing pipeline isn't as nice as you would like, but
remember to view this from a normal developer's point of view, not a
documentation pipeline developer's point of view please.

So, in short, my requirements are:
- keep Documentation/ABI/ file formats as close as possible to
what we have today, preventing any flag-day issues or merge
problems
- be able to query and validate Documentation/ABI/
- be able to turn Documentation/ABI into pretty documentation.

If you object to the mechanics of the last requirement here, I don't
object either, provide something else that works better. But don't
throw away the whole thing just because you don't like how things are
hooked up here.

I'm going to go apply most of the rest of these patches to my
driver-core tree, stopping at the "hook it up to the kernel
documentation" point. Is that ok?

thanks,

greg k-h

2019-06-17 13:48:57

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

On Mon, 17 Jun 2019 14:54:38 +0200
Greg Kroah-Hartman <[email protected]> wrote:

> > I think it's just sad to see the documentation system slowly drift
> > further away from the ideals we had, and towards the old ways we worked
> > so hard to fix.
>
> What are those ideals?
>
> I thought the goal was to be able to write documentation in a as much
> as a normal text file as possible and have automation turn those files
> into "pretty" documentation that we can all use.

That was indeed one of the goals. Another was to replace the incredible
pile of fragile duct tape that the docs build system had become with
something more robust, understandable, and maintainable. We did that, to
an extent at least, and life is better.

Jani worries that we have been regressing toward duct-tape mode, and I
suspect he may be right. I'm certainly as guilty as anybody of tossing
stuff in because it's expedient right now. It is right to ask whether we
should continue in that direction.

Can we slow down just a bit on the ABI files? It may be that Mauro's
solution is the best one, but I would really like to think a bit about
how all this stuff fits together, and life isn't really even giving me
time to tie my shoes these days. I don't think that this is screamingly
urgent right now.

Thanks,

jon

2019-06-17 13:49:37

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

On Mon, 17 Jun 2019, Greg Kroah-Hartman <[email protected]> wrote:
> On Mon, Jun 17, 2019 at 03:36:17PM +0300, Jani Nikula wrote:
>> On Fri, 14 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
>> > Em Fri, 14 Jun 2019 16:06:03 +0200
>> > Greg Kroah-Hartman <[email protected]> escreveu:
>> >
>> >> On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
>> >> > 2) Have the python extension read the ABI files directly, without an
>> >> > extra pipeline.
>> >>
>> >> He who writes the script, get's to dictate the language of the script :)
>>
>> The point is, it's an extension to a python based tool, written in perl,
>> using pipes for communication, and losing any advantages of integrating
>> with the tool it's extending.
>>
>> I doubt you'd want to see system() to be used to subsequently extend the
>> perl tool.
>>
>> I think it's just sad to see the documentation system slowly drift
>> further away from the ideals we had, and towards the old ways we worked
>> so hard to fix.
>
> What are those ideals?

For example, have a single coherent system, instead of a fragile pipe
with each stage written in a different language, each having its own
idiosynchracies, each step losing something in translation.

Have a system that a normal developer can actually look at and
understand. It didn't use to be that way.

> I thought the goal was to be able to write documentation in a as much
> as a normal text file as possible and have automation turn those files
> into "pretty" documentation that we can all use.
>
> And I think that fits with the way this patch set goes, right? We are
> not on a quest for purity of scripts to generate the documentation at
> the expense of having to force hundreds, or thousands, of developers to
> change their ways, or to force a "flag day" conversion of existing
> documentation resulting in a huge merge mess.

Fair enough, let's dismiss the thought of changing the ABI files. But I
never meant that would somehow be for the "purity of scripts", or that
those two would somehow be at odds here.

> So, we are stuck with the current structure that I totally made up for
> Documentation/ABI/. Turns out it is almost parsable, as Mauro's tool
> shows. His tool also validates the existing text, which is great, and
> has caused fixes for it.
>
> If someone wants to write that tool in some other language, like python,
> wonderful, I have no objection, but as it is, this is a useful tool
> already, allowing us to validate, and search, existing documentation
> entries that we have never been able to do before. It also provides an
> output that can be turned into pretty html/pdf/whatever files by other
> tools in the pipeline, a totally bonus benefit.
>
> So what is going backwards here?
>
> Maybe the processing pipeline isn't as nice as you would like, but
> remember to view this from a normal developer's point of view, not a
> documentation pipeline developer's point of view please.
>
> So, in short, my requirements are:
> - keep Documentation/ABI/ file formats as close as possible to
> what we have today, preventing any flag-day issues or merge
> problems
> - be able to query and validate Documentation/ABI/
> - be able to turn Documentation/ABI into pretty documentation.
>
> If you object to the mechanics of the last requirement here, I don't
> object either, provide something else that works better. But don't
> throw away the whole thing just because you don't like how things are
> hooked up here.
>
> I'm going to go apply most of the rest of these patches to my
> driver-core tree, stopping at the "hook it up to the kernel
> documentation" point. Is that ok?

I'll leave it all up to Jon's discretion; I trust he'll understand my
concerns. I have no authority beyond the opinion I've voiced here
anyway.

BR,
Jani.

--
Jani Nikula, Intel Open Source Graphics Center

2019-06-17 13:52:49

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

Em Mon, 17 Jun 2019 15:36:17 +0300
Jani Nikula <[email protected]> escreveu:

> On Fri, 14 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> > Em Fri, 14 Jun 2019 16:06:03 +0200
> > Greg Kroah-Hartman <[email protected]> escreveu:
> >
> >> On Fri, Jun 14, 2019 at 04:42:20PM +0300, Jani Nikula wrote:
> >> > 2) Have the python extension read the ABI files directly, without an
> >> > extra pipeline.
> >>
> >> He who writes the script, get's to dictate the language of the script :)
>
> The point is, it's an extension to a python based tool, written in perl,
> using pipes for communication, and losing any advantages of integrating
> with the tool it's extending.
>
> I doubt you'd want to see system() to be used to subsequently extend the
> perl tool.
>
> I think it's just sad to see the documentation system slowly drift
> further away from the ideals we had, and towards the old ways we worked
> so hard to fix.

Actually, it is a perl script that can be used standalone (just like
get_maintainers.pl and kernel-doc) with have some features including
producing a ReST output. We could easily get rid of the python extension,
if we add this to the Makefile (adjusted to work with O= option):

./scripts/get_api.pl rest > Documentation/output/admin-guide/abi.rst

>
> > No idea about how much time it would take if written in python,
> > but this perl script is really fast:
> >
> > $ time ./scripts/get_abi.pl search voltage_max >/dev/null
> > real 0m0,139s
> > user 0m0,132s
> > sys 0m0,006s
> >
> > That's the time it takes here (SSD disks) to read all files under
> > Documentation/ABI, parse them and seek for a string.
> >
> > That's about half of the time a python script takes to just import the
> > the sphinx modules and print its version, running at the same machine:
> >
> > $ time sphinx-build --version >/dev/null
> >
> > real 0m0,224s
> > user 0m0,199s
> > sys 0m0,024s
>
> Please at least use fair and sensible comparisons. If you want to make
> the extension usable standalone on the command-line, bypassing Sphinx,
> you can do that. No need to factor in Sphinx to your comparisons.

Yeah, I guess it should be possible to do that. How a python script
can identify if it was called by Sphinx, or if it was called directly?

Thanks,
Mauro

2019-06-17 13:59:10

[permalink] [raw]

Subject: Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support

On Mon, 17 Jun 2019 06:16:59 -0300
Mauro Carvalho Chehab <[email protected]> wrote:

> > No need to change, the emacs notation is also OK, see your link
> >
> > """or (using formats recognized by popular editors):"""
> >
> > https://www.python.org/dev/peps/pep-0263/#defining-the-encoding
> >
> > I prefer emacs notation, this is also evaluated by many other editors / tools.
>
> The usage of emacs notation is something that we don't like at the
> Linux Kernel. With ~4K developers per release, if we add tags to
> every single editor people use, it would be really messy, as one
> developer would be adding a tag and the next one replacing it by its
> some other favorite editor's tag.

So "we" like a language-specific notation instead? That seems a little
strange to me. Lots of things understand the Emacs notation, it doesn't
seem like something that needs to be actively avoided here.

Thanks,

jon

2019-06-17 15:57:39

[permalink] [raw]

Subject: Re: [PATCH 14/14] docs: sphinx/kernel_abi.py: fix UTF-8 support

Em Mon, 17 Jun 2019 07:56:08 -0600
Jonathan Corbet <[email protected]> escreveu:

> On Mon, 17 Jun 2019 06:16:59 -0300
> Mauro Carvalho Chehab <[email protected]> wrote:
>
> > > No need to change, the emacs notation is also OK, see your link
> > >
> > > """or (using formats recognized by popular editors):"""
> > >
> > > https://www.python.org/dev/peps/pep-0263/#defining-the-encoding
> > >
> > > I prefer emacs notation, this is also evaluated by many other editors / tools.
> >
> > The usage of emacs notation is something that we don't like at the
> > Linux Kernel. With ~4K developers per release, if we add tags to
> > every single editor people use, it would be really messy, as one
> > developer would be adding a tag and the next one replacing it by its
> > some other favorite editor's tag.
>
> So "we" like a language-specific notation instead? That seems a little
> strange to me. Lots of things understand the Emacs notation, it doesn't
> seem like something that needs to be actively avoided here.

From my side, I don't have any strong preference. Just saying that
people usually complain when e-macs or vim specific tags appear at the
Kernel. That's why I would prefer an editor-agnostic macro.

It won't make any difference for me, anyway, as the editors I use
don't recognize it.

Whatever you want is OK to me, provided that we use the same notation on
all Sphinx extensions... right now there's a mix of notations.

Thanks,
Mauro

2019-06-17 16:32:52

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

Am 17.06.19 um 11:11 schrieb Mauro Carvalho Chehab:
> Em Sun, 16 Jun 2019 18:04:01 +0200
> Markus Heiser <[email protected]> escreveu:
>
>> Am 14.06.19 um 16:15 schrieb Jonathan Corbet:
>>> On Fri, 14 Jun 2019 16:10:31 +0200
>>> Markus Heiser <[email protected]> wrote:
>>>
>>>> I agree with Jani. No matter how the decision ends, since I can't help here, I'd
>>>> rather not show up in the copyright.
>>>
>>> Is there something specific you are asking us to do here?
>>>
>>
>>
>> I have lost the overview, but there was a patch Mauro added a
>> kernel_abi.py. There was my name (Markus Heiser) listed with a
>> copyright notation.
>>
>> I guess Mauro picked up some old RFC or an other old patch of
>> mine from 2016 and made some C&P .. whatever .. ATM I do not have
>> time to give any support on parsing ABI and I'am not interested
>> in holding copyrights on a C&P of a old source ;)
>
> Well, the code was basically written by you :-)
>
> It was written to be a script capable of running a generic
> script. On that time, my contribution to it was basically
> to hardcode it to run "get_abi.pl".

Thanks for clarifying.

>
> This came from an old branch where the last change was back in 2017.
> It was resurrected due to a discussion at KS ML.
>
> There, the discussion was related to what's left to be converted
> to ReST.
>
> While I can't simply remove your copyright, would you be happy
> with something like that?

Yes, but basically I share Jani's and Jon's doubts about this solution.

-- Markus --

>
>
> Thanks,
> Mauro
>
> diff --git a/Documentation/sphinx/kernel_abi.py b/Documentation/sphinx/kernel_abi.py
> index 2d5d582207f7..ef91b1e1ff4b 100644
> --- a/Documentation/sphinx/kernel_abi.py
> +++ b/Documentation/sphinx/kernel_abi.py
> @@ -7,7 +7,8 @@ u"""
> Implementation of the ``kernel-abi`` reST-directive.
>
> :copyright: Copyright (C) 2016 Markus Heiser
> - :copyright: Copyright (C) 2016 Mauro Carvalho Chehab
> + :copyright: Copyright (C) 2016-2019 Mauro Carvalho Chehab
> + :maintained-by: Mauro Carvalho Chehab <[email protected]>
> :license: GPL Version 2, June 1991 see Linux/COPYING for details.
>
> The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
>
>

2019-06-18 08:45:06

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

On Mon, 17 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> Yeah, I guess it should be possible to do that. How a python script
> can identify if it was called by Sphinx, or if it was called directly?

if __name__ == '__main__':
# run on the command-line, not imported

BR,
Jani.

--
Jani Nikula, Intel Open Source Graphics Center

2019-06-19 12:52:54

by Johan Hovold

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> From: Mauro Carvalho Chehab <[email protected]>
>
> When parsing via script, it is important to know if the script
> should consider a description as a literal block that should
> be displayed as-is, or if the description can be considered
> as a normal text.
>
> Change descriptions to ensure that the preceding line of a table
> ends with a colon. That makes easy to identify the need of a
> literal block.

In the cover letter you say that the first four patches of this series,
including this one, "fix some ABI descriptions that are violating the
syntax described at Documentation/ABI/README". This seems a bit harsh,
given that it's you that is now *introducing* a new syntax requirement
to assist your script.

Specifically, this new requirement isn't documented anywhere AFAICT, so
how will anyone adding new ABI descriptions learn about it?

> Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> Signed-off-by: Mauro Carvalho Chehab <[email protected]>
> ---
> Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra | 2 +-
> .../ABI/testing/sysfs-class-backlight-driver-lm3533 | 6 +++---
> Documentation/ABI/testing/sysfs-class-led-driver-lm3533 | 8 ++++----
> Documentation/ABI/testing/sysfs-class-leds-gt683r | 4 ++--
> Documentation/ABI/testing/sysfs-driver-hid-roccat-kone | 2 +-
> 5 files changed, 11 insertions(+), 11 deletions(-)
>
> diff --git a/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra b/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
> index 16020b31ae64..5d41ebadf15e 100644
> --- a/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
> +++ b/Documentation/ABI/obsolete/sysfs-driver-hid-roccat-pyra
> @@ -5,7 +5,7 @@ Description: It is possible to switch the cpi setting of the mouse with the
> press of a button.
> When read, this file returns the raw number of the actual cpi
> setting reported by the mouse. This number has to be further
> - processed to receive the real dpi value.
> + processed to receive the real dpi value:
>
> VALUE DPI
> 1 400

Johan

2019-06-19 13:56:59

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

Hi Johan,

Em Wed, 19 Jun 2019 14:51:35 +0200
Johan Hovold <[email protected]> escreveu:

> On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > From: Mauro Carvalho Chehab <[email protected]>
> >
> > When parsing via script, it is important to know if the script
> > should consider a description as a literal block that should
> > be displayed as-is, or if the description can be considered
> > as a normal text.
> >
> > Change descriptions to ensure that the preceding line of a table
> > ends with a colon. That makes easy to identify the need of a
> > literal block.
>
> In the cover letter you say that the first four patches of this series,
> including this one, "fix some ABI descriptions that are violating the
> syntax described at Documentation/ABI/README". This seems a bit harsh,
> given that it's you that is now *introducing* a new syntax requirement
> to assist your script.

Yeah, what's there at the cover letter doesn't apply to this specific
patch. The thing is that I wrote this series a lot of time ago (2016/17).

I revived those per a request at KS ML, as we still need to expose the
ABI content on some book that will be used by userspace people.

So, I just rebased it on the top of curent Kernel, add a cover letter
with the things I remembered and re-sent.

In the specific case of this patch, the ":" there actually makes sense
for someone that it is reading it as a text file, and it is an easy
hack to make it parse better.

> Specifically, this new requirement isn't documented anywhere AFAICT, so
> how will anyone adding new ABI descriptions learn about it?

Yeah, either that or provide an alternative to "Description" tag, to be
used with more complex ABI descriptions.

One of the things that occurred to me, back on 2017, is that we should
have a way to to specify that an specific ABI description would have
a rich format. Something like:

What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
Date: August 2010
Contact: Stefan Achatz <[email protected]>
RST-Description:
It is possible to switch the cpi setting of the mouse with the
press of a button.
When read, this file returns the raw number of the actual cpi
setting reported by the mouse. This number has to be further
processed to receive the real dpi value:

===== =====
VALUE DPI
===== =====
1 400
2 800
4 1600
===== =====

With that, the script will know that the description contents will be using
the ReST markup, and parse it accordingly. Right now, what it does, instead,
is to place the description on a code-block, e. g. it will produce this
output for the description:

::

It is possible to switch the cpi setting of the mouse with the
press of a button.
When read, this file returns the raw number of the actual cpi
setting reported by the mouse. This number has to be further
processed to receive the real dpi value:

VALUE DPI
1 400
2 800
4 1600

Greg,

what do you think?

Thanks,
Mauro

2019-06-19 15:02:35

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> Hi Johan,
>
> Em Wed, 19 Jun 2019 14:51:35 +0200
> Johan Hovold <[email protected]> escreveu:
>
> > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > From: Mauro Carvalho Chehab <[email protected]>
> > >
> > > When parsing via script, it is important to know if the script
> > > should consider a description as a literal block that should
> > > be displayed as-is, or if the description can be considered
> > > as a normal text.
> > >
> > > Change descriptions to ensure that the preceding line of a table
> > > ends with a colon. That makes easy to identify the need of a
> > > literal block.
> >
> > In the cover letter you say that the first four patches of this series,
> > including this one, "fix some ABI descriptions that are violating the
> > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > given that it's you that is now *introducing* a new syntax requirement
> > to assist your script.
>
> Yeah, what's there at the cover letter doesn't apply to this specific
> patch. The thing is that I wrote this series a lot of time ago (2016/17).
>
> I revived those per a request at KS ML, as we still need to expose the
> ABI content on some book that will be used by userspace people.
>
> So, I just rebased it on the top of curent Kernel, add a cover letter
> with the things I remembered and re-sent.
>
> In the specific case of this patch, the ":" there actually makes sense
> for someone that it is reading it as a text file, and it is an easy
> hack to make it parse better.
>
> > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > how will anyone adding new ABI descriptions learn about it?
>
> Yeah, either that or provide an alternative to "Description" tag, to be
> used with more complex ABI descriptions.
>
> One of the things that occurred to me, back on 2017, is that we should
> have a way to to specify that an specific ABI description would have
> a rich format. Something like:
>
> What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
> Date: August 2010
> Contact: Stefan Achatz <[email protected]>
> RST-Description:
> It is possible to switch the cpi setting of the mouse with the
> press of a button.
> When read, this file returns the raw number of the actual cpi
> setting reported by the mouse. This number has to be further
> processed to receive the real dpi value:
>
> ===== =====
> VALUE DPI
> ===== =====
> 1 400
> 2 800
> 4 1600
> ===== =====
>
> With that, the script will know that the description contents will be using
> the ReST markup, and parse it accordingly. Right now, what it does, instead,
> is to place the description on a code-block, e. g. it will produce this
> output for the description:
>
> ::
>
> It is possible to switch the cpi setting of the mouse with the
> press of a button.
> When read, this file returns the raw number of the actual cpi
> setting reported by the mouse. This number has to be further
> processed to receive the real dpi value:
>
> VALUE DPI
> 1 400
> 2 800
> 4 1600
>
>
> Greg,
>
> what do you think?

I don't know when "Description" and "RST-Description" would be used.
Why not just parse "Description" like rst text and if things are "messy"
we fix them up as found, like you did with the ":" here? It doesn't
have to be complex, we can always fix them up after-the-fact if new
stuff gets added that doesn't quite parse properly.

Just like we do for most kernel-doc formatting :)

thanks,

greg k-h

2019-06-19 16:15:04

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

Em Wed, 19 Jun 2019 17:02:07 +0200
Greg Kroah-Hartman <[email protected]> escreveu:

> On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> > Hi Johan,
> >
> > Em Wed, 19 Jun 2019 14:51:35 +0200
> > Johan Hovold <[email protected]> escreveu:
> >
> > > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > > From: Mauro Carvalho Chehab <[email protected]>
> > > >
> > > > When parsing via script, it is important to know if the script
> > > > should consider a description as a literal block that should
> > > > be displayed as-is, or if the description can be considered
> > > > as a normal text.
> > > >
> > > > Change descriptions to ensure that the preceding line of a table
> > > > ends with a colon. That makes easy to identify the need of a
> > > > literal block.
> > >
> > > In the cover letter you say that the first four patches of this series,
> > > including this one, "fix some ABI descriptions that are violating the
> > > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > > given that it's you that is now *introducing* a new syntax requirement
> > > to assist your script.
> >
> > Yeah, what's there at the cover letter doesn't apply to this specific
> > patch. The thing is that I wrote this series a lot of time ago (2016/17).
> >
> > I revived those per a request at KS ML, as we still need to expose the
> > ABI content on some book that will be used by userspace people.
> >
> > So, I just rebased it on the top of curent Kernel, add a cover letter
> > with the things I remembered and re-sent.
> >
> > In the specific case of this patch, the ":" there actually makes sense
> > for someone that it is reading it as a text file, and it is an easy
> > hack to make it parse better.
> >
> > > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > > how will anyone adding new ABI descriptions learn about it?
> >
> > Yeah, either that or provide an alternative to "Description" tag, to be
> > used with more complex ABI descriptions.
> >
> > One of the things that occurred to me, back on 2017, is that we should
> > have a way to to specify that an specific ABI description would have
> > a rich format. Something like:
> >
> > What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
> > Date: August 2010
> > Contact: Stefan Achatz <[email protected]>
> > RST-Description:
> > It is possible to switch the cpi setting of the mouse with the
> > press of a button.
> > When read, this file returns the raw number of the actual cpi
> > setting reported by the mouse. This number has to be further
> > processed to receive the real dpi value:
> >
> > ===== =====
> > VALUE DPI
> > ===== =====
> > 1 400
> > 2 800
> > 4 1600
> > ===== =====
> >
> > With that, the script will know that the description contents will be using
> > the ReST markup, and parse it accordingly. Right now, what it does, instead,
> > is to place the description on a code-block, e. g. it will produce this
> > output for the description:
> >
> > ::
> >
> > It is possible to switch the cpi setting of the mouse with the
> > press of a button.
> > When read, this file returns the raw number of the actual cpi
> > setting reported by the mouse. This number has to be further
> > processed to receive the real dpi value:
> >
> > VALUE DPI
> > 1 400
> > 2 800
> > 4 1600
> >
> >
> > Greg,
> >
> > what do you think?
>
> I don't know when "Description" and "RST-Description" would be used.
> Why not just parse "Description" like rst text and if things are "messy"
> we fix them up as found, like you did with the ":" here? It doesn't
> have to be complex, we can always fix them up after-the-fact if new
> stuff gets added that doesn't quite parse properly.
>
> Just like we do for most kernel-doc formatting :)

Works for me. Yet, I guess I tried that, back on 2017.

If I'm not mistaken, the initial patchset to solve the broken things
won't be small, and will be require a lot of attention in order to
identify what's broken and where.

Btw, one thing is to pass at ReST validation. Another thing is to
produce something that people can read.

Right now, the pertinent logic at the script I wrote (scripts/get_abi.pl)
is here:

if (!($desc =~ /^\s*$/)) {
if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
# put everything inside a code block
$desc =~ s/\n/\n /g;

print "::\n\n";
print " $desc\n\n";
} else {
# Escape any special chars from description
$desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;

print "$desc\n\n";
}
}

If it discovers something weird enough, it just places everything
into a comment block. Otherwise, it assumes that it is a plain
text and that any special characters should be escaped.

If the above block is replaced by a simple:

print "$desc\n\n";

The description content will be handled as a ReST file.

I don't have any time right now to do this change and to handle the
warnings that will start to popup.

Btw, a single replace there is enough to show the amount of problems that
it will rise, as it will basically break Sphinx build with:

reading sources... [ 1%] admin-guide/abi-testing
reST markup error:
get_abi.pl rest --dir $srctree/Documentation/ABI/testing:45261: (SEVERE/4) Missing matching underline for section title overline.

==========================
PCIe Device AER statistics
These attributes show up under all the devices that are AER capable. These

Thanks,
Mauro

diff --git a/scripts/get_abi.pl b/scripts/get_abi.pl
index 7bc619b6890c..e75f441afdcd 100755
--- a/scripts/get_abi.pl
+++ b/scripts/get_abi.pl
@@ -288,18 +288,18 @@ sub output_rest {
$desc =~ s/\n[\-\*\=\^\~]+\n/\n/g;

if (!($desc =~ /^\s*$/)) {
- if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
- # put everything inside a code block
- $desc =~ s/\n/\n /g;
+# if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
+# # put everything inside a code block
+# $desc =~ s/\n/\n /g;

- print "::\n\n";
- print " $desc\n\n";
- } else {
- # Escape any special chars from description
- $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
+# print "::\n\n";
+# print " $desc\n\n";
+# } else {
+# # Escape any special chars from description
+# $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;

print "$desc\n\n";
- }
+# }
} else {
print "DESCRIPTION MISSING for $what\n\n" if (!$data{$what}->{is_file});
}

2019-06-19 16:38:26

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

Em Tue, 18 Jun 2019 11:47:32 +0300
Jani Nikula <[email protected]> escreveu:

> On Mon, 17 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> > Yeah, I guess it should be possible to do that. How a python script
> > can identify if it was called by Sphinx, or if it was called directly?
>
> if __name__ == '__main__':
> # run on the command-line, not imported

Ok, when I have some spare time, I may try to convert one script
to python and see how it behaves.

Thanks,
Mauro

2019-06-20 12:02:33

by Johan Hovold

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

On Wed, Jun 19, 2019 at 05:02:07PM +0200, Greg Kroah-Hartman wrote:
> On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> > Hi Johan,
> >
> > Em Wed, 19 Jun 2019 14:51:35 +0200
> > Johan Hovold <[email protected]> escreveu:
> >
> > > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > > From: Mauro Carvalho Chehab <[email protected]>
> > > >
> > > > When parsing via script, it is important to know if the script
> > > > should consider a description as a literal block that should
> > > > be displayed as-is, or if the description can be considered
> > > > as a normal text.
> > > >
> > > > Change descriptions to ensure that the preceding line of a table
> > > > ends with a colon. That makes easy to identify the need of a
> > > > literal block.
> > >
> > > In the cover letter you say that the first four patches of this series,
> > > including this one, "fix some ABI descriptions that are violating the
> > > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > > given that it's you that is now *introducing* a new syntax requirement
> > > to assist your script.
> >
> > Yeah, what's there at the cover letter doesn't apply to this specific
> > patch. The thing is that I wrote this series a lot of time ago (2016/17).

Got it, thanks.

[...]

> > In the specific case of this patch, the ":" there actually makes sense
> > for someone that it is reading it as a text file, and it is an easy
> > hack to make it parse better.

Human readers probably depend on more on tabulation and white space.
When the preceding description wasn't using a colon to begin with (and
you just replace s/\./:/) it can even look weird, but no big deal.

> > > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > > how will anyone adding new ABI descriptions learn about it?
> >
> > Yeah, either that or provide an alternative to "Description" tag, to be
> > used with more complex ABI descriptions.
> >
> > One of the things that occurred to me, back on 2017, is that we should
> > have a way to to specify that an specific ABI description would have
> > a rich format. Something like:

[...]

> I don't know when "Description" and "RST-Description" would be used.
> Why not just parse "Description" like rst text and if things are "messy"
> we fix them up as found, like you did with the ":" here? It doesn't
> have to be complex, we can always fix them up after-the-fact if new
> stuff gets added that doesn't quite parse properly.
>
> Just like we do for most kernel-doc formatting :)

But kernel-doc has a documented format, which was sort of the point I
was trying to make. If the new get_abi.pl scripts expects a colon I
think it should be mentioned somewhere (e.g. Documentation/ABI/README).

Grepping for attribute entries in linux-next still reveals a number
descriptions that still lack that colon and use varying formatting. More
are bound to be added later, but perhaps that's ok depending on what
you're aiming at here.

Johan

2019-06-20 12:54:43

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

On Thu, Jun 20, 2019 at 02:01:50PM +0200, Johan Hovold wrote:
> > I don't know when "Description" and "RST-Description" would be used.
> > Why not just parse "Description" like rst text and if things are "messy"
> > we fix them up as found, like you did with the ":" here? It doesn't
> > have to be complex, we can always fix them up after-the-fact if new
> > stuff gets added that doesn't quite parse properly.
> >
> > Just like we do for most kernel-doc formatting :)
>
> But kernel-doc has a documented format, which was sort of the point I
> was trying to make. If the new get_abi.pl scripts expects a colon I
> think it should be mentioned somewhere (e.g. Documentation/ABI/README).
>
> Grepping for attribute entries in linux-next still reveals a number
> descriptions that still lack that colon and use varying formatting. More
> are bound to be added later, but perhaps that's ok depending on what
> you're aiming at here.

I'm aiming for "good enough" to start with, and then we can work through
the exceptions.

But given that Mauro hasn't resent the script that does the conversion
of the files, I don't know if that will even matter... {hint}

thanks,

greg k-h

2019-06-20 14:21:15

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

Em Thu, 20 Jun 2019 14:54:13 +0200
Greg Kroah-Hartman <[email protected]> escreveu:

> On Thu, Jun 20, 2019 at 02:01:50PM +0200, Johan Hovold wrote:
> > > I don't know when "Description" and "RST-Description" would be used.
> > > Why not just parse "Description" like rst text and if things are "messy"
> > > we fix them up as found, like you did with the ":" here? It doesn't
> > > have to be complex, we can always fix them up after-the-fact if new
> > > stuff gets added that doesn't quite parse properly.
> > >
> > > Just like we do for most kernel-doc formatting :)
> >
> > But kernel-doc has a documented format, which was sort of the point I
> > was trying to make. If the new get_abi.pl scripts expects a colon I
> > think it should be mentioned somewhere (e.g. Documentation/ABI/README).
> >
> > Grepping for attribute entries in linux-next still reveals a number
> > descriptions that still lack that colon and use varying formatting. More
> > are bound to be added later, but perhaps that's ok depending on what
> > you're aiming at here.
>
> I'm aiming for "good enough" to start with, and then we can work through
> the exceptions.
>
> But given that Mauro hasn't resent the script that does the conversion
> of the files, I don't know if that will even matter... {hint}

It sounds I missed something... are you expecting a new version?

If so, what changes are you expecting?

Thanks,
Mauro

2019-06-20 14:28:14

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

Em Wed, 19 Jun 2019 13:14:08 -0300
Mauro Carvalho Chehab <[email protected]> escreveu:

> Em Wed, 19 Jun 2019 17:02:07 +0200
> Greg Kroah-Hartman <[email protected]> escreveu:
>
> > On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> > > Hi Johan,
> > >
> > > Em Wed, 19 Jun 2019 14:51:35 +0200
> > > Johan Hovold <[email protected]> escreveu:
> > >
> > > > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > > > From: Mauro Carvalho Chehab <[email protected]>
> > > > >
> > > > > When parsing via script, it is important to know if the script
> > > > > should consider a description as a literal block that should
> > > > > be displayed as-is, or if the description can be considered
> > > > > as a normal text.
> > > > >
> > > > > Change descriptions to ensure that the preceding line of a table
> > > > > ends with a colon. That makes easy to identify the need of a
> > > > > literal block.
> > > >
> > > > In the cover letter you say that the first four patches of this series,
> > > > including this one, "fix some ABI descriptions that are violating the
> > > > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > > > given that it's you that is now *introducing* a new syntax requirement
> > > > to assist your script.
> > >
> > > Yeah, what's there at the cover letter doesn't apply to this specific
> > > patch. The thing is that I wrote this series a lot of time ago (2016/17).
> > >
> > > I revived those per a request at KS ML, as we still need to expose the
> > > ABI content on some book that will be used by userspace people.
> > >
> > > So, I just rebased it on the top of curent Kernel, add a cover letter
> > > with the things I remembered and re-sent.
> > >
> > > In the specific case of this patch, the ":" there actually makes sense
> > > for someone that it is reading it as a text file, and it is an easy
> > > hack to make it parse better.
> > >
> > > > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > > > how will anyone adding new ABI descriptions learn about it?
> > >
> > > Yeah, either that or provide an alternative to "Description" tag, to be
> > > used with more complex ABI descriptions.
> > >
> > > One of the things that occurred to me, back on 2017, is that we should
> > > have a way to to specify that an specific ABI description would have
> > > a rich format. Something like:
> > >
> > > What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
> > > Date: August 2010
> > > Contact: Stefan Achatz <[email protected]>
> > > RST-Description:
> > > It is possible to switch the cpi setting of the mouse with the
> > > press of a button.
> > > When read, this file returns the raw number of the actual cpi
> > > setting reported by the mouse. This number has to be further
> > > processed to receive the real dpi value:
> > >
> > > ===== =====
> > > VALUE DPI
> > > ===== =====
> > > 1 400
> > > 2 800
> > > 4 1600
> > > ===== =====
> > >
> > > With that, the script will know that the description contents will be using
> > > the ReST markup, and parse it accordingly. Right now, what it does, instead,
> > > is to place the description on a code-block, e. g. it will produce this
> > > output for the description:
> > >
> > > ::
> > >
> > > It is possible to switch the cpi setting of the mouse with the
> > > press of a button.
> > > When read, this file returns the raw number of the actual cpi
> > > setting reported by the mouse. This number has to be further
> > > processed to receive the real dpi value:
> > >
> > > VALUE DPI
> > > 1 400
> > > 2 800
> > > 4 1600
> > >
> > >
> > > Greg,
> > >
> > > what do you think?
> >
> > I don't know when "Description" and "RST-Description" would be used.
> > Why not just parse "Description" like rst text and if things are "messy"
> > we fix them up as found, like you did with the ":" here? It doesn't
> > have to be complex, we can always fix them up after-the-fact if new
> > stuff gets added that doesn't quite parse properly.
> >
> > Just like we do for most kernel-doc formatting :)
>
> Works for me. Yet, I guess I tried that, back on 2017.
>
> If I'm not mistaken, the initial patchset to solve the broken things
> won't be small, and will be require a lot of attention in order to
> identify what's broken and where.
>
> Btw, one thing is to pass at ReST validation. Another thing is to
> produce something that people can read.
>
> Right now, the pertinent logic at the script I wrote (scripts/get_abi.pl)
> is here:
>
> if (!($desc =~ /^\s*$/)) {
> if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
> # put everything inside a code block
> $desc =~ s/\n/\n /g;
>
> print "::\n\n";
> print " $desc\n\n";
> } else {
> # Escape any special chars from description
> $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
>
> print "$desc\n\n";
> }
> }
>
> If it discovers something weird enough, it just places everything
> into a comment block. Otherwise, it assumes that it is a plain
> text and that any special characters should be escaped.
>
> If the above block is replaced by a simple:
>
> print "$desc\n\n";
>
> The description content will be handled as a ReST file.
>
> I don't have any time right now to do this change and to handle the
> warnings that will start to popup.
>
> Btw, a single replace there is enough to show the amount of problems that
> it will rise, as it will basically break Sphinx build with:
>
> reading sources... [ 1%] admin-guide/abi-testing
> reST markup error:
> get_abi.pl rest --dir $srctree/Documentation/ABI/testing:45261: (SEVERE/4) Missing matching underline for section title overline.
>
> ==========================
> PCIe Device AER statistics
> These attributes show up under all the devices that are AER capable. These

To be clear here: the problem with the above is that ReST has zero
tolerance and actually behaves like a crash, if it receives something like:

==========================
PCIe Device AER statistics
==========================

For it to work, it has to have zero spaces before ===..=== line, e. g.:

==========================
PCIe Device AER statistics
==========================

Ok, maybe we could try to teach the parser a way to identify the initial
spacing of the first description line and remove that amount of
spaces/tabs for the following lines, but it may require some heuristics.

Thanks,
Mauro

2019-06-20 16:31:56

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

On Thu, Jun 20, 2019 at 11:20:34AM -0300, Mauro Carvalho Chehab wrote:
> Em Thu, 20 Jun 2019 14:54:13 +0200
> Greg Kroah-Hartman <[email protected]> escreveu:
>
> > On Thu, Jun 20, 2019 at 02:01:50PM +0200, Johan Hovold wrote:
> > > > I don't know when "Description" and "RST-Description" would be used.
> > > > Why not just parse "Description" like rst text and if things are "messy"
> > > > we fix them up as found, like you did with the ":" here? It doesn't
> > > > have to be complex, we can always fix them up after-the-fact if new
> > > > stuff gets added that doesn't quite parse properly.
> > > >
> > > > Just like we do for most kernel-doc formatting :)
> > >
> > > But kernel-doc has a documented format, which was sort of the point I
> > > was trying to make. If the new get_abi.pl scripts expects a colon I
> > > think it should be mentioned somewhere (e.g. Documentation/ABI/README).
> > >
> > > Grepping for attribute entries in linux-next still reveals a number
> > > descriptions that still lack that colon and use varying formatting. More
> > > are bound to be added later, but perhaps that's ok depending on what
> > > you're aiming at here.
> >
> > I'm aiming for "good enough" to start with, and then we can work through
> > the exceptions.
> >
> > But given that Mauro hasn't resent the script that does the conversion
> > of the files, I don't know if that will even matter... {hint}
>
> It sounds I missed something... are you expecting a new version?

Yes, the last round of patches didn't have a SPDX header on the script,
so I couldn't add it to the tree :(

thanks,

greg k-h

2019-06-20 17:18:11

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

Em Thu, 20 Jun 2019 18:29:45 +0200
Greg Kroah-Hartman <[email protected]> escreveu:

> On Thu, Jun 20, 2019 at 11:20:34AM -0300, Mauro Carvalho Chehab wrote:
> > Em Thu, 20 Jun 2019 14:54:13 +0200
> > Greg Kroah-Hartman <[email protected]> escreveu:
> >
> > > On Thu, Jun 20, 2019 at 02:01:50PM +0200, Johan Hovold wrote:
> > > > > I don't know when "Description" and "RST-Description" would be used.
> > > > > Why not just parse "Description" like rst text and if things are "messy"
> > > > > we fix them up as found, like you did with the ":" here? It doesn't
> > > > > have to be complex, we can always fix them up after-the-fact if new
> > > > > stuff gets added that doesn't quite parse properly.
> > > > >
> > > > > Just like we do for most kernel-doc formatting :)
> > > >
> > > > But kernel-doc has a documented format, which was sort of the point I
> > > > was trying to make. If the new get_abi.pl scripts expects a colon I
> > > > think it should be mentioned somewhere (e.g. Documentation/ABI/README).
> > > >
> > > > Grepping for attribute entries in linux-next still reveals a number
> > > > descriptions that still lack that colon and use varying formatting. More
> > > > are bound to be added later, but perhaps that's ok depending on what
> > > > you're aiming at here.
> > >
> > > I'm aiming for "good enough" to start with, and then we can work through
> > > the exceptions.
> > >
> > > But given that Mauro hasn't resent the script that does the conversion
> > > of the files, I don't know if that will even matter... {hint}
> >
> > It sounds I missed something... are you expecting a new version?
>
> Yes, the last round of patches didn't have a SPDX header on the script,
> so I couldn't add it to the tree :(

I could swear I sent you a version with SPDX on it... anyway, I'm
re-sending the hole thing. The SPDX header addition is on a separate
patch.

Thanks,
Mauro

2019-06-20 17:55:36

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

On Thu, Jun 20, 2019 at 02:16:28PM -0300, Mauro Carvalho Chehab wrote:
> Em Thu, 20 Jun 2019 18:29:45 +0200
> Greg Kroah-Hartman <[email protected]> escreveu:
>
> > On Thu, Jun 20, 2019 at 11:20:34AM -0300, Mauro Carvalho Chehab wrote:
> > > Em Thu, 20 Jun 2019 14:54:13 +0200
> > > Greg Kroah-Hartman <[email protected]> escreveu:
> > >
> > > > On Thu, Jun 20, 2019 at 02:01:50PM +0200, Johan Hovold wrote:
> > > > > > I don't know when "Description" and "RST-Description" would be used.
> > > > > > Why not just parse "Description" like rst text and if things are "messy"
> > > > > > we fix them up as found, like you did with the ":" here? It doesn't
> > > > > > have to be complex, we can always fix them up after-the-fact if new
> > > > > > stuff gets added that doesn't quite parse properly.
> > > > > >
> > > > > > Just like we do for most kernel-doc formatting :)
> > > > >
> > > > > But kernel-doc has a documented format, which was sort of the point I
> > > > > was trying to make. If the new get_abi.pl scripts expects a colon I
> > > > > think it should be mentioned somewhere (e.g. Documentation/ABI/README).
> > > > >
> > > > > Grepping for attribute entries in linux-next still reveals a number
> > > > > descriptions that still lack that colon and use varying formatting. More
> > > > > are bound to be added later, but perhaps that's ok depending on what
> > > > > you're aiming at here.
> > > >
> > > > I'm aiming for "good enough" to start with, and then we can work through
> > > > the exceptions.
> > > >
> > > > But given that Mauro hasn't resent the script that does the conversion
> > > > of the files, I don't know if that will even matter... {hint}
> > >
> > > It sounds I missed something... are you expecting a new version?
> >
> > Yes, the last round of patches didn't have a SPDX header on the script,
> > so I couldn't add it to the tree :(
>
> I could swear I sent you a version with SPDX on it... anyway, I'm
> re-sending the hole thing. The SPDX header addition is on a separate
> patch.

Ah, you didn't cc: me on the whole thread this time, that's why I missed
it last time. I sucked it off lkml now, will look at it in the morning.

thanks,

greg k-h

2019-06-21 07:22:26

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

On Thu, Jun 20, 2019 at 11:27:49AM -0300, Mauro Carvalho Chehab wrote:
> Em Wed, 19 Jun 2019 13:14:08 -0300
> Mauro Carvalho Chehab <[email protected]> escreveu:
>
> > Em Wed, 19 Jun 2019 17:02:07 +0200
> > Greg Kroah-Hartman <[email protected]> escreveu:
> >
> > > On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> > > > Hi Johan,
> > > >
> > > > Em Wed, 19 Jun 2019 14:51:35 +0200
> > > > Johan Hovold <[email protected]> escreveu:
> > > >
> > > > > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > > > > From: Mauro Carvalho Chehab <[email protected]>
> > > > > >
> > > > > > When parsing via script, it is important to know if the script
> > > > > > should consider a description as a literal block that should
> > > > > > be displayed as-is, or if the description can be considered
> > > > > > as a normal text.
> > > > > >
> > > > > > Change descriptions to ensure that the preceding line of a table
> > > > > > ends with a colon. That makes easy to identify the need of a
> > > > > > literal block.
> > > > >
> > > > > In the cover letter you say that the first four patches of this series,
> > > > > including this one, "fix some ABI descriptions that are violating the
> > > > > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > > > > given that it's you that is now *introducing* a new syntax requirement
> > > > > to assist your script.
> > > >
> > > > Yeah, what's there at the cover letter doesn't apply to this specific
> > > > patch. The thing is that I wrote this series a lot of time ago (2016/17).
> > > >
> > > > I revived those per a request at KS ML, as we still need to expose the
> > > > ABI content on some book that will be used by userspace people.
> > > >
> > > > So, I just rebased it on the top of curent Kernel, add a cover letter
> > > > with the things I remembered and re-sent.
> > > >
> > > > In the specific case of this patch, the ":" there actually makes sense
> > > > for someone that it is reading it as a text file, and it is an easy
> > > > hack to make it parse better.
> > > >
> > > > > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > > > > how will anyone adding new ABI descriptions learn about it?
> > > >
> > > > Yeah, either that or provide an alternative to "Description" tag, to be
> > > > used with more complex ABI descriptions.
> > > >
> > > > One of the things that occurred to me, back on 2017, is that we should
> > > > have a way to to specify that an specific ABI description would have
> > > > a rich format. Something like:
> > > >
> > > > What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
> > > > Date: August 2010
> > > > Contact: Stefan Achatz <[email protected]>
> > > > RST-Description:
> > > > It is possible to switch the cpi setting of the mouse with the
> > > > press of a button.
> > > > When read, this file returns the raw number of the actual cpi
> > > > setting reported by the mouse. This number has to be further
> > > > processed to receive the real dpi value:
> > > >
> > > > ===== =====
> > > > VALUE DPI
> > > > ===== =====
> > > > 1 400
> > > > 2 800
> > > > 4 1600
> > > > ===== =====
> > > >
> > > > With that, the script will know that the description contents will be using
> > > > the ReST markup, and parse it accordingly. Right now, what it does, instead,
> > > > is to place the description on a code-block, e. g. it will produce this
> > > > output for the description:
> > > >
> > > > ::
> > > >
> > > > It is possible to switch the cpi setting of the mouse with the
> > > > press of a button.
> > > > When read, this file returns the raw number of the actual cpi
> > > > setting reported by the mouse. This number has to be further
> > > > processed to receive the real dpi value:
> > > >
> > > > VALUE DPI
> > > > 1 400
> > > > 2 800
> > > > 4 1600
> > > >
> > > >
> > > > Greg,
> > > >
> > > > what do you think?
> > >
> > > I don't know when "Description" and "RST-Description" would be used.
> > > Why not just parse "Description" like rst text and if things are "messy"
> > > we fix them up as found, like you did with the ":" here? It doesn't
> > > have to be complex, we can always fix them up after-the-fact if new
> > > stuff gets added that doesn't quite parse properly.
> > >
> > > Just like we do for most kernel-doc formatting :)
> >
> > Works for me. Yet, I guess I tried that, back on 2017.
> >
> > If I'm not mistaken, the initial patchset to solve the broken things
> > won't be small, and will be require a lot of attention in order to
> > identify what's broken and where.
> >
> > Btw, one thing is to pass at ReST validation. Another thing is to
> > produce something that people can read.
> >
> > Right now, the pertinent logic at the script I wrote (scripts/get_abi.pl)
> > is here:
> >
> > if (!($desc =~ /^\s*$/)) {
> > if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
> > # put everything inside a code block
> > $desc =~ s/\n/\n /g;
> >
> > print "::\n\n";
> > print " $desc\n\n";
> > } else {
> > # Escape any special chars from description
> > $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
> >
> > print "$desc\n\n";
> > }
> > }
> >
> > If it discovers something weird enough, it just places everything
> > into a comment block. Otherwise, it assumes that it is a plain
> > text and that any special characters should be escaped.
> >
> > If the above block is replaced by a simple:
> >
> > print "$desc\n\n";
> >
> > The description content will be handled as a ReST file.
> >
> > I don't have any time right now to do this change and to handle the
> > warnings that will start to popup.
> >
> > Btw, a single replace there is enough to show the amount of problems that
> > it will rise, as it will basically break Sphinx build with:
> >
> > reading sources... [ 1%] admin-guide/abi-testing
> > reST markup error:
> > get_abi.pl rest --dir $srctree/Documentation/ABI/testing:45261: (SEVERE/4) Missing matching underline for section title overline.
> >
> > ==========================
> > PCIe Device AER statistics
> > These attributes show up under all the devices that are AER capable. These
>
> To be clear here: the problem with the above is that ReST has zero
> tolerance and actually behaves like a crash, if it receives something like:
>
> ==========================
> PCIe Device AER statistics
> ==========================
>
> For it to work, it has to have zero spaces before ===..=== line, e. g.:
>
> ==========================
> PCIe Device AER statistics
> ==========================
>
> Ok, maybe we could try to teach the parser a way to identify the initial
> spacing of the first description line and remove that amount of
> spaces/tabs for the following lines, but it may require some heuristics.

Or we can clean this type of thing up by hand :)

Let's see how bad this gets, the documentation in these files should not
be very complex as they _should_ only be one-value-per-file, but as you
have shown in this very example, that rule is violated :(

thanks,

greg k-h

2019-06-21 09:49:49

[permalink] [raw]

Subject: Re: [PATCH 04/14] ABI: better identificate tables

Em Fri, 21 Jun 2019 09:21:52 +0200
Greg Kroah-Hartman <[email protected]> escreveu:

> On Thu, Jun 20, 2019 at 11:27:49AM -0300, Mauro Carvalho Chehab wrote:
> > Em Wed, 19 Jun 2019 13:14:08 -0300
> > Mauro Carvalho Chehab <[email protected]> escreveu:
> >
> > > Em Wed, 19 Jun 2019 17:02:07 +0200
> > > Greg Kroah-Hartman <[email protected]> escreveu:
> > >
> > > > On Wed, Jun 19, 2019 at 10:56:33AM -0300, Mauro Carvalho Chehab wrote:
> > > > > Hi Johan,
> > > > >
> > > > > Em Wed, 19 Jun 2019 14:51:35 +0200
> > > > > Johan Hovold <[email protected]> escreveu:
> > > > >
> > > > > > On Thu, Jun 13, 2019 at 11:04:10PM -0300, Mauro Carvalho Chehab wrote:
> > > > > > > From: Mauro Carvalho Chehab <[email protected]>
> > > > > > >
> > > > > > > When parsing via script, it is important to know if the script
> > > > > > > should consider a description as a literal block that should
> > > > > > > be displayed as-is, or if the description can be considered
> > > > > > > as a normal text.
> > > > > > >
> > > > > > > Change descriptions to ensure that the preceding line of a table
> > > > > > > ends with a colon. That makes easy to identify the need of a
> > > > > > > literal block.
> > > > > >
> > > > > > In the cover letter you say that the first four patches of this series,
> > > > > > including this one, "fix some ABI descriptions that are violating the
> > > > > > syntax described at Documentation/ABI/README". This seems a bit harsh,
> > > > > > given that it's you that is now *introducing* a new syntax requirement
> > > > > > to assist your script.
> > > > >
> > > > > Yeah, what's there at the cover letter doesn't apply to this specific
> > > > > patch. The thing is that I wrote this series a lot of time ago (2016/17).
> > > > >
> > > > > I revived those per a request at KS ML, as we still need to expose the
> > > > > ABI content on some book that will be used by userspace people.
> > > > >
> > > > > So, I just rebased it on the top of curent Kernel, add a cover letter
> > > > > with the things I remembered and re-sent.
> > > > >
> > > > > In the specific case of this patch, the ":" there actually makes sense
> > > > > for someone that it is reading it as a text file, and it is an easy
> > > > > hack to make it parse better.
> > > > >
> > > > > > Specifically, this new requirement isn't documented anywhere AFAICT, so
> > > > > > how will anyone adding new ABI descriptions learn about it?
> > > > >
> > > > > Yeah, either that or provide an alternative to "Description" tag, to be
> > > > > used with more complex ABI descriptions.
> > > > >
> > > > > One of the things that occurred to me, back on 2017, is that we should
> > > > > have a way to to specify that an specific ABI description would have
> > > > > a rich format. Something like:
> > > > >
> > > > > What: /sys/bus/usb/devices/<busnum>-<devnum>:<config num>.<interface num>/<hid-bus>:<vendor-id>:<product-id>.<num>/pyra/roccatpyra<minor>/actual_cpi
> > > > > Date: August 2010
> > > > > Contact: Stefan Achatz <[email protected]>
> > > > > RST-Description:
> > > > > It is possible to switch the cpi setting of the mouse with the
> > > > > press of a button.
> > > > > When read, this file returns the raw number of the actual cpi
> > > > > setting reported by the mouse. This number has to be further
> > > > > processed to receive the real dpi value:
> > > > >
> > > > > ===== =====
> > > > > VALUE DPI
> > > > > ===== =====
> > > > > 1 400
> > > > > 2 800
> > > > > 4 1600
> > > > > ===== =====
> > > > >
> > > > > With that, the script will know that the description contents will be using
> > > > > the ReST markup, and parse it accordingly. Right now, what it does, instead,
> > > > > is to place the description on a code-block, e. g. it will produce this
> > > > > output for the description:
> > > > >
> > > > > ::
> > > > >
> > > > > It is possible to switch the cpi setting of the mouse with the
> > > > > press of a button.
> > > > > When read, this file returns the raw number of the actual cpi
> > > > > setting reported by the mouse. This number has to be further
> > > > > processed to receive the real dpi value:
> > > > >
> > > > > VALUE DPI
> > > > > 1 400
> > > > > 2 800
> > > > > 4 1600
> > > > >
> > > > >
> > > > > Greg,
> > > > >
> > > > > what do you think?
> > > >
> > > > I don't know when "Description" and "RST-Description" would be used.
> > > > Why not just parse "Description" like rst text and if things are "messy"
> > > > we fix them up as found, like you did with the ":" here? It doesn't
> > > > have to be complex, we can always fix them up after-the-fact if new
> > > > stuff gets added that doesn't quite parse properly.
> > > >
> > > > Just like we do for most kernel-doc formatting :)
> > >
> > > Works for me. Yet, I guess I tried that, back on 2017.
> > >
> > > If I'm not mistaken, the initial patchset to solve the broken things
> > > won't be small, and will be require a lot of attention in order to
> > > identify what's broken and where.
> > >
> > > Btw, one thing is to pass at ReST validation. Another thing is to
> > > produce something that people can read.
> > >
> > > Right now, the pertinent logic at the script I wrote (scripts/get_abi.pl)
> > > is here:
> > >
> > > if (!($desc =~ /^\s*$/)) {
> > > if ($desc =~ m/\:\n/ || $desc =~ m/\n[\t ]+/ || $desc =~ m/[\x00-\x08\x0b-\x1f\x7b-\xff]/) {
> > > # put everything inside a code block
> > > $desc =~ s/\n/\n /g;
> > >
> > > print "::\n\n";
> > > print " $desc\n\n";
> > > } else {
> > > # Escape any special chars from description
> > > $desc =~s/([\x00-\x08\x0b-\x1f\x21-\x2a\x2d\x2f\x3c-\x40\x5c\x5e-\x60\x7b-\xff])/\\$1/g;
> > >
> > > print "$desc\n\n";
> > > }
> > > }
> > >
> > > If it discovers something weird enough, it just places everything
> > > into a comment block. Otherwise, it assumes that it is a plain
> > > text and that any special characters should be escaped.
> > >
> > > If the above block is replaced by a simple:
> > >
> > > print "$desc\n\n";
> > >
> > > The description content will be handled as a ReST file.
> > >
> > > I don't have any time right now to do this change and to handle the
> > > warnings that will start to popup.
> > >
> > > Btw, a single replace there is enough to show the amount of problems that
> > > it will rise, as it will basically break Sphinx build with:
> > >
> > > reading sources... [ 1%] admin-guide/abi-testing
> > > reST markup error:
> > > get_abi.pl rest --dir $srctree/Documentation/ABI/testing:45261: (SEVERE/4) Missing matching underline for section title overline.
> > >
> > > ==========================
> > > PCIe Device AER statistics
> > > These attributes show up under all the devices that are AER capable. These
> >
> > To be clear here: the problem with the above is that ReST has zero
> > tolerance and actually behaves like a crash, if it receives something like:
> >
> > ==========================
> > PCIe Device AER statistics
> > ==========================
> >
> > For it to work, it has to have zero spaces before ===..=== line, e. g.:
> >
> > ==========================
> > PCIe Device AER statistics
> > ==========================
> >
> > Ok, maybe we could try to teach the parser a way to identify the initial
> > spacing of the first description line and remove that amount of
> > spaces/tabs for the following lines, but it may require some heuristics.
>
> Or we can clean this type of thing up by hand :)
>
> Let's see how bad this gets, the documentation in these files should not
> be very complex as they _should_ only be one-value-per-file, but as you
> have shown in this very example, that rule is violated :(

Hi Greg,

Doing a manual review - still incomplete (just stable/ dir) - and I didn't
try to run the script over ABI file yet - this is what I found so
far[1]: 17 files over 37 needed adjustments (45%).

That trick of detecting a complex file when it finds a ':' and parse the
hole description as a literal block solves most of them.

From the changes below, this file:
Documentation/ABI/stable/sysfs-driver-mlxreg-io

Found a "creative way" to place the "What:" field avoiding long lines...

At least this one should be fixed, as the parser won't handle its
content right (it won't complain, but the result output will be wrong).

[1] I also added a note at README that files should be ReST-compatible

diff --git a/Documentation/ABI/README b/Documentation/ABI/README
index 3121029dce21..8bac9cb09a6d 100644
--- a/Documentation/ABI/README
+++ b/Documentation/ABI/README
@@ -32,7 +32,7 @@ The different levels of stability are:
layout of the files below for details on how to do this.)

obsolete/
- This directory documents interfaces that are still remaining in
+ This directory documents interfaces that are still remaining in
the kernel, but are marked to be removed at some later point in
time. The description of the interface will document the reason
why it is obsolete and when it can be expected to be removed.
@@ -58,6 +58,14 @@ Users: All users of this interface who wish to be notified when
be changed further.

+Note:
+ The fields should be use a simple notation, compatible with ReST markup.
+ Also, the file **should not** have a top-level index, like::
+
+ ===
+ foo
+ ===
+
How things move between levels:

Interfaces in stable may move to obsolete, as long as the proper
diff --git a/Documentation/ABI/stable/firewire-cdev b/Documentation/ABI/stable/firewire-cdev
index f72ed653878a..d8df3004c318 100644
--- a/Documentation/ABI/stable/firewire-cdev
+++ b/Documentation/ABI/stable/firewire-cdev
@@ -20,6 +20,7 @@ Description:
- Query node ID
- Query maximum speed of the path between this node
and local node
+
- The 1394 bus (i.e. "card") to which the node is attached to:
- Isochronous stream transmission and reception
- Asynchronous stream transmission and reception
@@ -31,6 +32,7 @@ Description:
manager
- Query cycle time
- Bus reset initiation, bus reset event reception
+
- All 1394 buses:
- Allocation of IEEE 1212 address ranges on the local
link layers, reception of inbound requests to such
diff --git a/Documentation/ABI/stable/sysfs-acpi-pmprofile b/Documentation/ABI/stable/sysfs-acpi-pmprofile
index 964c7a8afb26..3b0e34eef83c 100644
--- a/Documentation/ABI/stable/sysfs-acpi-pmprofile
+++ b/Documentation/ABI/stable/sysfs-acpi-pmprofile
@@ -11,12 +11,15 @@ Values: For possible values see ACPI specification:
Field: Preferred_PM_Profile

Currently these values are defined by spec:
- 0 Unspecified
- 1 Desktop
- 2 Mobile
- 3 Workstation
- 4 Enterprise Server
- 5 SOHO Server
- 6 Appliance PC
- 7 Performance Server
+
+ == =================
+ 0 Unspecified
+ 1 Desktop
+ 2 Mobile
+ 3 Workstation
+ 4 Enterprise Server
+ 5 SOHO Server
+ 6 Appliance PC
+ 7 Performance Server
>7 Reserved
+ == =================
diff --git a/Documentation/ABI/stable/sysfs-bus-firewire b/Documentation/ABI/stable/sysfs-bus-firewire
index 41e5a0cd1e3e..0c8c023855b6 100644
--- a/Documentation/ABI/stable/sysfs-bus-firewire
+++ b/Documentation/ABI/stable/sysfs-bus-firewire
@@ -125,7 +125,9 @@ Description:
Read-only attribute, immutable during the target's lifetime.
Format, as exposed by firewire-sbp2 since 2.6.22, May 2007:
Colon-separated hexadecimal string representations of
+
u64 EUI-64 : u24 directory_ID : u16 LUN
+
without 0x prefixes, without whitespace. The former sbp2 driver
(removed in 2.6.37 after being superseded by firewire-sbp2) used
a somewhat shorter format which was not as close to SAM.
diff --git a/Documentation/ABI/stable/sysfs-bus-nvmem b/Documentation/ABI/stable/sysfs-bus-nvmem
index 9ffba8576f7b..c399323f37de 100644
--- a/Documentation/ABI/stable/sysfs-bus-nvmem
+++ b/Documentation/ABI/stable/sysfs-bus-nvmem
@@ -9,13 +9,14 @@ Description:
Note: This file is only present if CONFIG_NVMEM_SYSFS
is enabled

- ex:
- hexdump /sys/bus/nvmem/devices/qfprom0/nvmem
+ ex::

- 0000000 0000 0000 0000 0000 0000 0000 0000 0000
- *
- 00000a0 db10 2240 0000 e000 0c00 0c00 0000 0c00
- 0000000 0000 0000 0000 0000 0000 0000 0000 0000
- ...
- *
- 0001000
+ hexdump /sys/bus/nvmem/devices/qfprom0/nvmem
+
+ 0000000 0000 0000 0000 0000 0000 0000 0000 0000
+ *
+ 00000a0 db10 2240 0000 e000 0c00 0c00 0000 0c00
+ 0000000 0000 0000 0000 0000 0000 0000 0000 0000
+ ...
+ *
+ 0001000
diff --git a/Documentation/ABI/stable/sysfs-bus-usb b/Documentation/ABI/stable/sysfs-bus-usb
index b832eeff9999..cad4bc232520 100644
--- a/Documentation/ABI/stable/sysfs-bus-usb
+++ b/Documentation/ABI/stable/sysfs-bus-usb
@@ -50,8 +50,10 @@ Description:

Tools can use this file and the connected_duration file to
compute the percentage of time that a device has been active.
- For example,
- echo $((100 * `cat active_duration` / `cat connected_duration`))
+ For example::
+
+ echo $((100 * `cat active_duration` / `cat connected_duration`))
+
will give an integer percentage. Note that this does not
account for counter wrap.
Users:
diff --git a/Documentation/ABI/stable/sysfs-class-backlight b/Documentation/ABI/stable/sysfs-class-backlight
index 70302f370e7e..023fb52645f8 100644
--- a/Documentation/ABI/stable/sysfs-class-backlight
+++ b/Documentation/ABI/stable/sysfs-class-backlight
@@ -4,6 +4,7 @@ KernelVersion: 2.6.12
Contact: Richard Purdie <[email protected]>
Description:
Control BACKLIGHT power, values are FB_BLANK_* from fb.h
+
- FB_BLANK_UNBLANK (0) : power on.
- FB_BLANK_POWERDOWN (4) : power off
Users: HAL
diff --git a/Documentation/ABI/stable/sysfs-class-infiniband b/Documentation/ABI/stable/sysfs-class-infiniband
index 17211ceb9bf4..3f58952a8ccb 100644
--- a/Documentation/ABI/stable/sysfs-class-infiniband
+++ b/Documentation/ABI/stable/sysfs-class-infiniband
@@ -8,12 +8,14 @@ Date: Apr, 2005
KernelVersion: v2.6.12
Contact: [email protected]
Description:
+ =============== ===========================================
node_type: (RO) Node type (CA, RNIC, usNIC, usNIC UDP,
switch or router)

node_guid: (RO) Node GUID

sys_image_guid: (RO) System image GUID
+ =============== ===========================================

What: /sys/class/infiniband/<device>/node_desc
@@ -47,6 +49,7 @@ KernelVersion: v2.6.12
Contact: [email protected]
Description:

+ =============== ===============================================
lid: (RO) Port LID

rate: (RO) Port data rate (active width * active
@@ -66,8 +69,9 @@ Description:

cap_mask: (RO) Port capability mask. 2 bits here are
settable- IsCommunicationManagementSupported
- (set when CM module is loaded) and IsSM (set via
- open of issmN file).
+ (set when CM module is loaded) and IsSM (set
+ via open of issmN file).
+ =============== ===============================================

What: /sys/class/infiniband/<device>/ports/<port-num>/link_layer
@@ -143,7 +147,7 @@ Description:
poor link signal integrity

Data info:
- ---------
+ ----------

port_xmit_data: (RO) Total number of data octets, divided by 4
(lanes), transmitted on all VLs. This is 64 bit counter
@@ -177,7 +181,7 @@ Description:
packets with errors.

Misc info:
- ---------
+ ----------

port_xmit_discards: (RO) Total number of outbound packets
discarded by the port because the port is down or congested.
@@ -244,9 +248,11 @@ Description:
two umad devices and two issm devices, while a switch will have
one device of each type (for switch port 0).

+ ======= =====================================
ibdev: (RO) Show Infiniband (IB) device name

port: (RO) Display port number
+ ======= =====================================

What: /sys/class/infiniband_mad/abi_version
@@ -281,10 +287,12 @@ Date: Sept, 2005
KernelVersion: v2.6.14
Contact: [email protected]
Description:
+ =============== ===========================================
ibdev: (RO) Display Infiniband (IB) device name

abi_version: (RO) Show ABI version of IB device specific
interfaces.
+ =============== ===========================================

What: /sys/class/infiniband_verbs/abi_version
@@ -306,12 +314,14 @@ Date: Apr, 2005
KernelVersion: v2.6.12
Contact: [email protected]
Description:
+ =============== ================================================
hw_rev: (RO) Hardware revision number

hca_type: (RO) Host Channel Adapter type: MT23108, MT25208
(MT23108 compat mode), MT25208 or MT25204

board_id: (RO) Manufacturing board ID
+ =============== ================================================

sysfs interface for Chelsio T3 RDMA Driver (cxgb3)
@@ -324,6 +334,7 @@ Date: Feb, 2007
KernelVersion: v2.6.21
Contact: [email protected]
Description:
+ =============== ==============================================
hw_rev: (RO) Hardware revision number

hca_type: (RO) HCA type. Here it is a driver short name.
@@ -331,6 +342,7 @@ Description:
driver structure (e.g. pci_driver::name).

board_id: (RO) Manufacturing board id
+ =============== ==============================================

sysfs interface for Mellanox ConnectX HCA IB driver (mlx4)
@@ -343,11 +355,13 @@ Date: Sep, 2007
KernelVersion: v2.6.24
Contact: [email protected]
Description:
+ =============== ===============================
hw_rev: (RO) Hardware revision number

hca_type: (RO) Host channel adapter type

board_id: (RO) Manufacturing board ID
+ =============== ===============================

What: /sys/class/infiniband/mlx4_X/iov/ports/<port-num>/gids/<n>
@@ -373,6 +387,7 @@ Description:
example, ports/1/pkeys/10 contains the value at index 10 in port
1's P_Key table.

+ ======================= ==========================================
gids/<n>: (RO) The physical port gids n = 0..127

admin_guids/<n>: (RW) Allows examining or changing the
@@ -401,6 +416,7 @@ Description:
guest, whenever it uses its pkey index
1, will actually be using the real pkey
index 10.
+ ======================= ==========================================

What: /sys/class/infiniband/mlx4_X/iov/<pci-slot-num>/ports/<m>/smi_enabled
@@ -412,12 +428,14 @@ Description:
Enabling QP0 on VFs for selected VF/port. By default, no VFs are
enabled for QP0 operation.

- smi_enabled: (RO) Indicates whether smi is currently enabled
- for the indicated VF/port
+ ================= ==== ===========================================
+ smi_enabled: (RO) Indicates whether smi is currently enabled
+ for the indicated VF/port

- enable_smi_admin:(RW) Used by the admin to request that smi
- capability be enabled or disabled for the
- indicated VF/port. 0 = disable, 1 = enable.
+ enable_smi_admin: (RW) Used by the admin to request that smi
+ capability be enabled or disabled for the
+ indicated VF/port. 0 = disable, 1 = enable.
+ ================= ==== ===========================================

The requested enablement will occur at the next reset of the VF
(e.g. driver restart on the VM which owns the VF).
@@ -433,11 +451,13 @@ Date: Feb, 2008
KernelVersion: v2.6.25
Contact: [email protected]
Description:
+ =============== ==== ==================================
hw_rev: (RO) Hardware revision number

hca_type: (RO) Host Channel Adapter type (NEX020)

board_id: (RO) Manufacturing board id
+ =============== ==== ==================================

sysfs interface for Chelsio T4/T5 RDMA driver (cxgb4)
@@ -451,6 +471,7 @@ KernelVersion: v2.6.35
Contact: [email protected]
Description:

+ =============== =============================================
hw_rev: (RO) Hardware revision number

hca_type: (RO) Driver short name. Should normally match
@@ -459,6 +480,7 @@ Description:

board_id: (RO) Manufacturing board id. (Vendor + device
information)
+ =============== =============================================

sysfs interface for Intel IB driver qib
@@ -479,6 +501,7 @@ Date: May, 2010
KernelVersion: v2.6.35
Contact: [email protected]
Description:
+ =============== ======================================================
version: (RO) Display version information of installed software
and drivers.

@@ -505,6 +528,7 @@ Description:
chip_reset: (WO) Reset the chip if possible by writing
"reset" to this file. Only allowed if no user
contexts are open that use chip resources.
+ =============== ======================================================

What: /sys/class/infiniband/qibX/ports/N/sl2vl/[0-15]
@@ -524,6 +548,7 @@ Contact: [email protected]
Description:
Per-port congestion control. Both are binary attributes.

+ =============== ================================================
cc_table_bin: (RO) Congestion control table size followed by
table entries.

@@ -532,6 +557,7 @@ Description:
congestion entries - increase, timer, event log
trigger threshold and the minimum injection rate
delay.
+ =============== ================================================

What: /sys/class/infiniband/qibX/ports/N/linkstate/loopback
What: /sys/class/infiniband/qibX/ports/N/linkstate/led_override
@@ -544,6 +570,7 @@ Contact: [email protected]
Description:
[to be documented]

+ =============== ===============================================
loopback: (WO)
led_override: (WO)
hrtbt_enable: (RW)
@@ -554,6 +581,7 @@ Description:
errors. Possible states are- "Initted",
"Present", "IB_link_up", "IB_configured" or
"Fatal_Hardware_Error".
+ =============== ===============================================

What: /sys/class/infiniband/qibX/ports/N/diag_counters/rc_resends
What: /sys/class/infiniband/qibX/ports/N/diag_counters/seq_naks
@@ -602,6 +630,7 @@ Contact: Christian Benvenuti <[email protected]>,
[email protected]
Description:

+ =============== ===============================================
board_id: (RO) Manufacturing board id

config: (RO) Report the configuration for this PF
@@ -614,6 +643,7 @@ Description:

iface: (RO) Shows which network interface this usNIC
entry is associated to (visible with ifconfig).
+ =============== ===============================================

What: /sys/class/infiniband/usnic_X/qpn/summary
What: /sys/class/infiniband/usnic_X/qpn/context
@@ -658,6 +688,7 @@ Date: May, 2016
KernelVersion: v4.6
Contact: [email protected]
Description:
+ =============== ==== ========================================
hw_rev: (RO) Hardware revision number

board_id: (RO) Manufacturing board id
@@ -676,6 +707,7 @@ Description:
available.

tempsense: (RO) Thermal sense information
+ =============== ==== ========================================

What: /sys/class/infiniband/hfi1_X/ports/N/CCMgtA/cc_settings_bin
@@ -687,19 +719,21 @@ Contact: [email protected]
Description:
Per-port congestion control.

- cc_table_bin: (RO) CCA tables used by PSM2 Congestion control
+ =============== ================================================
+ cc_table_bin (RO) CCA tables used by PSM2 Congestion control
table size followed by table entries. Binary
attribute.

- cc_settings_bin:(RO) Congestion settings: port control, control
+ cc_settings_bin (RO) Congestion settings: port control, control
map and an array of 16 entries for the
congestion entries - increase, timer, event log
trigger threshold and the minimum injection rate
delay. Binary attribute.

- cc_prescan: (RW) enable prescanning for faster BECN
+ cc_prescan (RW) enable prescanning for faster BECN
response. Write "on" to enable and "off" to
disable.
+ =============== ================================================

What: /sys/class/infiniband/hfi1_X/ports/N/sc2vl/[0-31]
What: /sys/class/infiniband/hfi1_X/ports/N/sl2sc/[0-31]
@@ -708,11 +742,13 @@ Date: May, 2016
KernelVersion: v4.6
Contact: [email protected]
Description:
+ =============== ===================================================
sc2vl/: (RO) 32 files (0 - 31) used to translate sl->vl

sl2sc/: (RO) 32 files (0 - 31) used to translate sl->sc

vl2mtu/: (RO) 16 files (0 - 15) used to determine MTU for vl
+ =============== ===================================================

What: /sys/class/infiniband/hfi1_X/sdma_N/cpu_list
@@ -723,26 +759,28 @@ Contact: [email protected]
Description:
sdma<N>/ contains one directory per sdma engine (0 - 15)

+ =============== ==============================================
cpu_list: (RW) List of cpus for user-process to sdma
engine assignment.

vl: (RO) Displays the virtual lane (vl) the sdma
engine maps to.
+ =============== ==============================================

This interface gives the user control on the affinity settings
for the device. As an example, to set an sdma engine irq
affinity and thread affinity of a user processes to use the
sdma engine, which is "near" in terms of NUMA configuration, or
- physical cpu location, the user will do:
+ physical cpu location, the user will do::

- echo "3" > /proc/irq/<N>/smp_affinity_list
- echo "4-7" > /sys/devices/.../sdma3/cpu_list
- cat /sys/devices/.../sdma3/vl
- 0
- echo "8" > /proc/irq/<M>/smp_affinity_list
- echo "9-12" > /sys/devices/.../sdma4/cpu_list
- cat /sys/devices/.../sdma4/vl
- 1
+ echo "3" > /proc/irq/<N>/smp_affinity_list
+ echo "4-7" > /sys/devices/.../sdma3/cpu_list
+ cat /sys/devices/.../sdma3/vl
+ 0
+ echo "8" > /proc/irq/<M>/smp_affinity_list
+ echo "9-12" > /sys/devices/.../sdma4/cpu_list
+ cat /sys/devices/.../sdma4/vl
+ 1

to make sure that when a process runs on cpus 4,5,6, or 7, and
uses vl=0, then sdma engine 3 is selected by the driver, and
@@ -764,11 +802,13 @@ Date: Jan, 2016
KernelVersion: v4.10
Contact: [email protected]
Description:
+ =============== ==== ========================
hw_rev: (RO) Hardware revision number

hca_type: (RO) Show HCA type (I40IW)

board_id: (RO) I40IW board ID
+ =============== ==== ========================

sysfs interface for QLogic qedr NIC Driver
@@ -781,9 +821,11 @@ KernelVersion: v4.10
Contact: [email protected]
Description:

+ =============== ==== ========================
hw_rev: (RO) Hardware revision number

hca_type: (RO) Display HCA type
+ =============== ==== ========================

sysfs interface for VMware Paravirtual RDMA driver
@@ -797,11 +839,13 @@ KernelVersion: v4.10
Contact: [email protected]
Description:

+ =============== ==== =====================================
hw_rev: (RO) Hardware revision number

hca_type: (RO) Host channel adapter type

board_id: (RO) Display PVRDMA manufacturing board ID
+ =============== ==== =====================================

sysfs interface for Broadcom NetXtreme-E RoCE driver
@@ -813,6 +857,8 @@ Date: Feb, 2017
KernelVersion: v4.11
Contact: [email protected]
Description:
+ =============== ==== =========================
hw_rev: (RO) Hardware revision number

hca_type: (RO) Host channel adapter type
+ =============== ==== =========================
diff --git a/Documentation/ABI/stable/sysfs-class-rfkill b/Documentation/ABI/stable/sysfs-class-rfkill
index 80151a409d67..370254b6296a 100644
--- a/Documentation/ABI/stable/sysfs-class-rfkill
+++ b/Documentation/ABI/stable/sysfs-class-rfkill
@@ -54,6 +54,7 @@ Description: Current state of the transmitter.
through this interface. There will likely be another attempt to
remove it in the future.
Values: A numeric value.
+
0: RFKILL_STATE_SOFT_BLOCKED
transmitter is turned off by software
1: RFKILL_STATE_UNBLOCKED
@@ -69,6 +70,7 @@ KernelVersion v2.6.34
Contact: [email protected]
Description: Current hardblock state. This file is read only.
Values: A numeric value.
+
0: inactive
The transmitter is (potentially) active.
1: active
@@ -82,6 +84,7 @@ KernelVersion v2.6.34
Contact: [email protected]
Description: Current softblock state. This file is read and write.
Values: A numeric value.
+
0: inactive
The transmitter is (potentially) active.
1: active
diff --git a/Documentation/ABI/stable/sysfs-class-tpm b/Documentation/ABI/stable/sysfs-class-tpm
index c0e23830f56a..bbee8899a90e 100644
--- a/Documentation/ABI/stable/sysfs-class-tpm
+++ b/Documentation/ABI/stable/sysfs-class-tpm
@@ -32,11 +32,11 @@ KernelVersion: 2.6.12
Contact: [email protected]
Description: The "caps" property contains TPM manufacturer and version info.

- Example output:
+ Example output::

- Manufacturer: 0x53544d20
- TCG version: 1.2
- Firmware version: 8.16
+ Manufacturer: 0x53544d20
+ TCG version: 1.2
+ Firmware version: 8.16

Manufacturer is a hex dump of the 4 byte manufacturer info
space in a TPM. TCG version shows the TCG TPM spec level that
@@ -54,9 +54,9 @@ Description: The "durations" property shows the 3 vendor-specific values
any longer than necessary before starting to poll for a
result.

- Example output:
+ Example output::

- 3015000 4508000 180995000 [original]
+ 3015000 4508000 180995000 [original]

Here the short, medium and long durations are displayed in
usecs. "[original]" indicates that the values are displayed
@@ -92,14 +92,14 @@ Description: The "pcrs" property will dump the current value of all Platform
values may be constantly changing, the output is only valid
for a snapshot in time.

- Example output:
+ Example output::

- PCR-00: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
- PCR-01: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
- PCR-02: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
- PCR-03: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
- PCR-04: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
- ...
+ PCR-00: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
+ PCR-01: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
+ PCR-02: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
+ PCR-03: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
+ PCR-04: 3A 3F 78 0F 11 A4 B4 99 69 FC AA 80 CD 6E 39 57 C3 3B 22 75
+ ...

The number of PCRs and hex bytes needed to represent a PCR
value will vary depending on TPM chip version. For TPM 1.1 and
@@ -119,44 +119,44 @@ Description: The "pubek" property will return the TPM's public endorsement
ated at TPM manufacture time and exists for the life of the
chip.

- Example output:
-
- Algorithm: 00 00 00 01
- Encscheme: 00 03
- Sigscheme: 00 01
- Parameters: 00 00 08 00 00 00 00 02 00 00 00 00
- Modulus length: 256
- Modulus:
- B4 76 41 82 C9 20 2C 10 18 40 BC 8B E5 44 4C 6C
- 3A B2 92 0C A4 9B 2A 83 EB 5C 12 85 04 48 A0 B6
- 1E E4 81 84 CE B2 F2 45 1C F0 85 99 61 02 4D EB
- 86 C4 F7 F3 29 60 52 93 6B B2 E5 AB 8B A9 09 E3
- D7 0E 7D CA 41 BF 43 07 65 86 3C 8C 13 7A D0 8B
- 82 5E 96 0B F8 1F 5F 34 06 DA A2 52 C1 A9 D5 26
- 0F F4 04 4B D9 3F 2D F2 AC 2F 74 64 1F 8B CD 3E
- 1E 30 38 6C 70 63 69 AB E2 50 DF 49 05 2E E1 8D
- 6F 78 44 DA 57 43 69 EE 76 6C 38 8A E9 8E A3 F0
- A7 1F 3C A8 D0 12 15 3E CA 0E BD FA 24 CD 33 C6
- 47 AE A4 18 83 8E 22 39 75 93 86 E6 FD 66 48 B6
- 10 AD 94 14 65 F9 6A 17 78 BD 16 53 84 30 BF 70
- E0 DC 65 FD 3C C6 B0 1E BF B9 C1 B5 6C EF B1 3A
- F8 28 05 83 62 26 11 DC B4 6B 5A 97 FF 32 26 B6
- F7 02 71 CF 15 AE 16 DD D1 C1 8E A8 CF 9B 50 7B
- C3 91 FF 44 1E CF 7C 39 FE 17 77 21 20 BD CE 9B
-
- Possible values:
-
- Algorithm: TPM_ALG_RSA (1)
- Encscheme: TPM_ES_RSAESPKCSv15 (2)
+ Example output::
+
+ Algorithm: 00 00 00 01
+ Encscheme: 00 03
+ Sigscheme: 00 01
+ Parameters: 00 00 08 00 00 00 00 02 00 00 00 00
+ Modulus length: 256
+ Modulus:
+ B4 76 41 82 C9 20 2C 10 18 40 BC 8B E5 44 4C 6C
+ 3A B2 92 0C A4 9B 2A 83 EB 5C 12 85 04 48 A0 B6
+ 1E E4 81 84 CE B2 F2 45 1C F0 85 99 61 02 4D EB
+ 86 C4 F7 F3 29 60 52 93 6B B2 E5 AB 8B A9 09 E3
+ D7 0E 7D CA 41 BF 43 07 65 86 3C 8C 13 7A D0 8B
+ 82 5E 96 0B F8 1F 5F 34 06 DA A2 52 C1 A9 D5 26
+ 0F F4 04 4B D9 3F 2D F2 AC 2F 74 64 1F 8B CD 3E
+ 1E 30 38 6C 70 63 69 AB E2 50 DF 49 05 2E E1 8D
+ 6F 78 44 DA 57 43 69 EE 76 6C 38 8A E9 8E A3 F0
+ A7 1F 3C A8 D0 12 15 3E CA 0E BD FA 24 CD 33 C6
+ 47 AE A4 18 83 8E 22 39 75 93 86 E6 FD 66 48 B6
+ 10 AD 94 14 65 F9 6A 17 78 BD 16 53 84 30 BF 70
+ E0 DC 65 FD 3C C6 B0 1E BF B9 C1 B5 6C EF B1 3A
+ F8 28 05 83 62 26 11 DC B4 6B 5A 97 FF 32 26 B6
+ F7 02 71 CF 15 AE 16 DD D1 C1 8E A8 CF 9B 50 7B
+ C3 91 FF 44 1E CF 7C 39 FE 17 77 21 20 BD CE 9B
+
+ Possible values::
+
+ Algorithm: TPM_ALG_RSA (1)
+ Encscheme: TPM_ES_RSAESPKCSv15 (2)
TPM_ES_RSAESOAEP_SHA1_MGF1 (3)
- Sigscheme: TPM_SS_NONE (1)
- Parameters, a byte string of 3 u32 values:
+ Sigscheme: TPM_SS_NONE (1)
+ Parameters, a byte string of 3 u32 values:
Key Length (bits): 00 00 08 00 (2048)
Num primes: 00 00 00 02 (2)
Exponent Size: 00 00 00 00 (0 means the
default exp)
- Modulus Length: 256 (bytes)
- Modulus: The 256 byte Endorsement Key modulus
+ Modulus Length: 256 (bytes)
+ Modulus: The 256 byte Endorsement Key modulus

What: /sys/class/tpm/tpmX/device/temp_deactivated
Date: April 2006
@@ -176,9 +176,9 @@ Description: The "timeouts" property shows the 4 vendor-specific values
timeouts is defined by the TPM interface spec that the chip
conforms to.

- Example output:
+ Example output::

- 750000 750000 750000 750000 [original]
+ 750000 750000 750000 750000 [original]

The four timeout values are shown in usecs, with a trailing
"[original]" or "[adjusted]" depending on whether the values
diff --git a/Documentation/ABI/stable/sysfs-devices b/Documentation/ABI/stable/sysfs-devices
index 4404bd9b96c1..42bf1eab5677 100644
--- a/Documentation/ABI/stable/sysfs-devices
+++ b/Documentation/ABI/stable/sysfs-devices
@@ -1,5 +1,6 @@
-# Note: This documents additional properties of any device beyond what
-# is documented in Documentation/admin-guide/sysfs-rules.rst
+Note:
+ This documents additional properties of any device beyond what
+ is documented in Documentation/admin-guide/sysfs-rules.rst

What: /sys/devices/*/of_node
Date: February 2015
diff --git a/Documentation/ABI/stable/sysfs-driver-ib_srp b/Documentation/ABI/stable/sysfs-driver-ib_srp
index 7049a2b50359..2d706b4900ce 100644
--- a/Documentation/ABI/stable/sysfs-driver-ib_srp
+++ b/Documentation/ABI/stable/sysfs-driver-ib_srp
@@ -6,6 +6,7 @@ Description: Interface for making ib_srp connect to a new target.
One can request ib_srp to connect to a new target by writing
a comma-separated list of login parameters to this sysfs
attribute. The supported parameters are:
+
* id_ext, a 16-digit hexadecimal number specifying the eight
byte identifier extension in the 16-byte SRP target port
identifier. The target port identifier is sent by ib_srp
diff --git a/Documentation/ABI/stable/sysfs-driver-mlxreg-io b/Documentation/ABI/stable/sysfs-driver-mlxreg-io
index 156319fc5b80..3544968f43cc 100644
--- a/Documentation/ABI/stable/sysfs-driver-mlxreg-io
+++ b/Documentation/ABI/stable/sysfs-driver-mlxreg-io
@@ -1,5 +1,4 @@
-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- asic_health
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/asic_health

Date: June 2018
KernelVersion: 4.19
@@ -9,9 +8,8 @@ Description: This file shows ASIC health status. The possible values are:

The files are read only.

-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- cpld1_version
- cpld2_version
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/cpld1_version
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/cpld2_version
Date: June 2018
KernelVersion: 4.19
Contact: Vadim Pasternak <vadimpmellanox.com>
@@ -20,8 +18,7 @@ Description: These files show with which CPLD versions have been burned

The files are read only.

-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- fan_dir
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/fan_dir

Date: December 2018
KernelVersion: 5.0
@@ -32,8 +29,7 @@ Description: This file shows the system fans direction:

The files are read only.

-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- jtag_enable
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/jtag_enable

Date: November 2018
KernelVersion: 5.0
@@ -43,8 +39,7 @@ Description: These files show with which CPLD versions have been burned

The files are read only.

-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- jtag_enable
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/jtag_enable

Date: November 2018
KernelVersion: 5.0
@@ -87,16 +82,15 @@ Description: These files allow asserting system power cycling, switching

The files are write only.

-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- reset_aux_pwr_or_ref
- reset_asic_thermal
- reset_hotswap_or_halt
- reset_hotswap_or_wd
- reset_fw_reset
- reset_long_pb
- reset_main_pwr_fail
- reset_short_pb
- reset_sw_reset
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_aux_pwr_or_ref
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_asic_thermal
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_hotswap_or_halt
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_hotswap_or_wd
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_fw_reset
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_long_pb
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_main_pwr_fail
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_short_pb
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_sw_reset
Date: June 2018
KernelVersion: 4.19
Contact: Vadim Pasternak <vadimpmellanox.com>
@@ -110,11 +104,10 @@ Description: These files show the system reset cause, as following: power

The files are read only.

-What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/
- reset_comex_pwr_fail
- reset_from_comex
- reset_system
- reset_voltmon_upgrade_fail
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_comex_pwr_fail
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_from_comex
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_system
+What: /sys/devices/platform/mlxplat/mlxreg-io/hwmon/hwmon*/reset_voltmon_upgrade_fail

Date: November 2018
KernelVersion: 5.0
diff --git a/Documentation/ABI/stable/sysfs-firmware-efi-vars b/Documentation/ABI/stable/sysfs-firmware-efi-vars
index 5def20b9019e..46ccd233e359 100644
--- a/Documentation/ABI/stable/sysfs-firmware-efi-vars
+++ b/Documentation/ABI/stable/sysfs-firmware-efi-vars
@@ -17,6 +17,7 @@ Description:
directory has a name of the form "<key>-<vendor guid>"
and contains the following files:

+ =============== ========================================
attributes: A read-only text file enumerating the
EFI variable flags. Potential values
include:
@@ -59,12 +60,14 @@ Description:

size: As ASCII representation of the size of
the variable's value.
+ =============== ========================================

In addition, two other magic binary files are provided
in the top-level directory and are used for adding and
removing variables:

+ =============== ========================================
new_var: Takes a "struct efi_variable" and
instructs the EFI firmware to create a
new variable.
@@ -73,3 +76,4 @@ Description:
instructs the EFI firmware to remove any
variable that has a matching vendor GUID
and variable key name.
+ =============== ========================================
diff --git a/Documentation/ABI/stable/sysfs-firmware-opal-dump b/Documentation/ABI/stable/sysfs-firmware-opal-dump
index 32fe7f5c4880..1f74f45327ba 100644
--- a/Documentation/ABI/stable/sysfs-firmware-opal-dump
+++ b/Documentation/ABI/stable/sysfs-firmware-opal-dump
@@ -7,6 +7,7 @@ Description:

This is only for the powerpc/powernv platform.

+ =============== ===============================================
initiate_dump: When '1' is written to it,
we will initiate a dump.
Read this file for supported commands.
@@ -19,8 +20,11 @@ Description:
and ID of the dump, use the id and type files.
Do not rely on any particular size of dump
type or dump id.
+ =============== ===============================================

Each dump has the following files:
+
+ =============== ===============================================
id: An ASCII representation of the dump ID
in hex (e.g. '0x01')
type: An ASCII representation of the type of
@@ -39,3 +43,4 @@ Description:
inaccessible.
Reading this file will get a list of
supported actions.
+ =============== ===============================================
diff --git a/Documentation/ABI/stable/sysfs-firmware-opal-elog b/Documentation/ABI/stable/sysfs-firmware-opal-elog
index 2536434d49d0..7c8a61a2d005 100644
--- a/Documentation/ABI/stable/sysfs-firmware-opal-elog
+++ b/Documentation/ABI/stable/sysfs-firmware-opal-elog
@@ -38,6 +38,7 @@ Description:
For each log entry (directory), there are the following
files:

+ ============== ================================================
id: An ASCII representation of the ID of the
error log, in hex - e.g. "0x01".

@@ -58,3 +59,4 @@ Description:
entry will be removed from sysfs.
Reading this file will list the supported
operations (currently just acknowledge).
+ ============== ================================================
diff --git a/Documentation/ABI/stable/sysfs-hypervisor-xen b/Documentation/ABI/stable/sysfs-hypervisor-xen
index 3cf5cdfcd9a8..01bd46be261e 100644
--- a/Documentation/ABI/stable/sysfs-hypervisor-xen
+++ b/Documentation/ABI/stable/sysfs-hypervisor-xen
@@ -33,6 +33,7 @@ Description: If running under Xen:
Space separated list of supported guest system types. Each type
is in the format: <class>-<major>.<minor>-<arch>
With:
+ ======== ============================================
<class>: "xen" -- x86: paravirtualized, arm: standard
"hvm" -- x86 only: fully virtualized
<major>: major guest interface version
@@ -43,6 +44,7 @@ Description: If running under Xen:
"x86_64": 64 bit x86 guest
"armv7l": 32 bit arm guest
"aarch64": 64 bit arm guest
+ ======== ============================================

What: /sys/hypervisor/properties/changeset
Date: March 2009
diff --git a/Documentation/ABI/stable/vdso b/Documentation/ABI/stable/vdso
index 55406ec8a35a..73ed1240a5c0 100644
--- a/Documentation/ABI/stable/vdso
+++ b/Documentation/ABI/stable/vdso
@@ -23,6 +23,7 @@ Unless otherwise noted, the set of symbols with any given version and the
ABI of those symbols is considered stable. It may vary across architectures,
though.

-(As of this writing, this ABI documentation as been confirmed for x86_64.
+Note:
+ As of this writing, this ABI documentation as been confirmed for x86_64.
The maintainers of the other vDSO-using architectures should confirm
- that it is correct for their architecture.)
+ that it is correct for their architecture.

Thanks,
Mauro

2019-06-21 14:27:48

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

Em Wed, 19 Jun 2019 13:37:39 -0300
Mauro Carvalho Chehab <[email protected]> escreveu:

> Em Tue, 18 Jun 2019 11:47:32 +0300
> Jani Nikula <[email protected]> escreveu:
>
> > On Mon, 17 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> > > Yeah, I guess it should be possible to do that. How a python script
> > > can identify if it was called by Sphinx, or if it was called directly?
> >
> > if __name__ == '__main__':
> > # run on the command-line, not imported
>
> Ok, when I have some spare time, I may try to convert one script
> to python and see how it behaves.

Did a quick test here...

Probably I'm doing something wrong (as I'm a rookie with Python), but,
in order to be able to use the same script as command line and as an Sphinx
extension, everything that it is currently there should be "escaped"
by an:

if __name__ != '__main__':

As event the class definition:

class KernelCmd(Directive):

depends on:

from docutils.parsers.rst import directives, Directive

With is only required when one needs to parse ReST - e. g. only
when the script runs as a Sphinx extension.

If this is right, as we want a script that can run by command line
to parse and search inside ABI files, at the end of the day, it will
be a lot easier to maintain if the parser script is different from the
Sphinx extension.

If so, as the Sphinx extension script will need to call a parsing script
anyway, it doesn't matter the language of the script with will be
doing the ABI file parsing.

See the enclosing file. I didn't add anything from the ABI parsing
script yet. It was just changed in order to not generate an error
when trying to run it from command line.

Thanks,
Mauro
#!/usr/bin/env python3
# coding=utf-8
# SPDX-License-Identifier: GPL-2.0
#
u"""
kernel-abi
~~~~~~~~~~

Implementation of the ``kernel-abi`` reST-directive.

:copyright: Copyright (C) 2016 Markus Heiser
:copyright: Copyright (C) 2016-2019 Mauro Carvalho Chehab
:maintained-by: Mauro Carvalho Chehab <[email protected]>
:license: GPL Version 2, June 1991 see Linux/COPYING for details.

The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
scripts/get_abi.pl script to parse the Kernel ABI files.

Overview of directive's argument and options.

.. code-block:: rst

.. kernel-abi:: <ABI directory location>
:debug:

The argument ``<ABI directory location>`` is required. It contains the
location of the ABI files to be parsed.

``debug``
Inserts a code-block with the *raw* reST. Sometimes it is helpful to see
what reST is generated.

"""

import codecs
import os
import subprocess
import sys

from os import path

if __name__ != '__main__':
from docutils import nodes, statemachine
from docutils.statemachine import ViewList
from docutils.parsers.rst import directives, Directive
from docutils.utils.error_reporting import ErrorString

#
# AutodocReporter is only good up to Sphinx 1.7
#
import sphinx

Use_SSI = sphinx.__version__[:3] >= '1.7'
if Use_SSI:
from sphinx.util.docutils import switch_source_input
else:
from sphinx.ext.autodoc import AutodocReporter

__version__ = '1.0'

if __name__ != '__main__':
def setup(app):

app.add_directive("kernel-abi", KernelCmd)
return dict(
version = __version__
, parallel_read_safe = True
, parallel_write_safe = True
)

class KernelCmd(Directive):

u"""KernelABI (``kernel-abi``) directive"""

required_arguments = 1
optional_arguments = 2
has_content = False
final_argument_whitespace = True

option_spec = {
"debug" : directives.flag,
"rst" : directives.unchanged
}

def warn(self, message, **replace):
replace["fname"] = self.state.document.current_source
replace["line_no"] = replace.get("line_no", self.lineno)
message = ("%(fname)s:%(line_no)s: [kernel-abi WARN] : " + message) % replace
self.state.document.settings.env.app.warn(message, prefix="")

def run(self):

doc = self.state.document
if not doc.settings.file_insertion_enabled:
raise self.warning("docutils: file insertion disabled")

env = doc.settings.env
cwd = path.dirname(doc.current_source)
fname = self.arguments[0]

cmd = "get_abi.pl rest --dir "
cmd += fname

if 'rst' in self.options:
cmd += " --rst-source"

srctree = path.abspath(os.environ["srctree"])

# extend PATH with $(srctree)/scripts
path_env = os.pathsep.join([
srctree + os.sep + "scripts",
os.environ["PATH"]
])
shell_env = os.environ.copy()
shell_env["PATH"] = path_env
shell_env["srctree"] = srctree

lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
nodeList = self.nestedParse(lines, self.arguments[0])
return nodeList

def runCmd(self, cmd, **kwargs):
u"""Run command ``cmd`` and return it's stdout as unicode."""

try:
proc = subprocess.Popen(
cmd
, stdout = subprocess.PIPE
, stderr = subprocess.PIPE
, **kwargs
)
out, err = proc.communicate()

out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')

if proc.returncode != 0:
raise self.severe(
u"command '%s' failed with return code %d"
% (cmd, proc.returncode)
)
except OSError as exc:
raise self.severe(u"problems with '%s' directive: %s."
% (self.name, ErrorString(exc)))
return out

def nestedParse(self, lines, fname):
content = ViewList()
node = nodes.section()

if "debug" in self.options:
code_block = "\n\n.. code-block:: rst\n :linenos:\n"
for l in lines.split("\n"):
code_block += "\n " + l
lines = code_block + "\n\n"

for c, l in enumerate(lines.split("\n")):
content.append(l, fname, c)

buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter

if Use_SSI:
with switch_source_input(self.state, content):
self.state.nested_parse(content, 0, node, match_titles=1)
else:
self.state.memo.title_styles = []
self.state.memo.section_level = 0
self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
try:
self.state.nested_parse(content, 0, node, match_titles=1)
finally:
self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf

return node.children

2019-06-27 09:48:00

[permalink] [raw]

Subject: Re: [PATCH 12/14] doc-rst: add ABI documentation to the admin-guide book

On Fri, 21 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
> Em Wed, 19 Jun 2019 13:37:39 -0300
> Mauro Carvalho Chehab <[email protected]> escreveu:
>
>> Em Tue, 18 Jun 2019 11:47:32 +0300
>> Jani Nikula <[email protected]> escreveu:
>>
>> > On Mon, 17 Jun 2019, Mauro Carvalho Chehab <[email protected]> wrote:
>> > > Yeah, I guess it should be possible to do that. How a python script
>> > > can identify if it was called by Sphinx, or if it was called directly?
>> >
>> > if __name__ == '__main__':
>> > # run on the command-line, not imported
>>
>> Ok, when I have some spare time, I may try to convert one script
>> to python and see how it behaves.
>
> Did a quick test here...
>
> Probably I'm doing something wrong (as I'm a rookie with Python), but,
> in order to be able to use the same script as command line and as an Sphinx
> extension, everything that it is currently there should be "escaped"
> by an:
>
> if __name__ != '__main__':
>
> As event the class definition:
>
> class KernelCmd(Directive):
>
> depends on:
>
> from docutils.parsers.rst import directives, Directive
>
> With is only required when one needs to parse ReST - e. g. only
> when the script runs as a Sphinx extension.
>
> If this is right, as we want a script that can run by command line
> to parse and search inside ABI files, at the end of the day, it will
> be a lot easier to maintain if the parser script is different from the
> Sphinx extension.

Split it into two files, one has the nuts and bolts of parsing and has
the "if __name__ == '__main__':" bit to run on the command line, and the
other interfaces with Sphinx and imports the parser.

> If so, as the Sphinx extension script will need to call a parsing script
> anyway, it doesn't matter the language of the script with will be
> doing the ABI file parsing.

Calling the parser using an API will be easier to use, maintain and
extend than using pipes, with all the interleaved sideband information
to adjust line numbers and whatnot.

BR,
Jani.

>
> See the enclosing file. I didn't add anything from the ABI parsing
> script yet. It was just changed in order to not generate an error
> when trying to run it from command line.
>
>
> Thanks,
> Mauro
> #!/usr/bin/env python3
> # coding=utf-8
> # SPDX-License-Identifier: GPL-2.0
> #
> u"""
> kernel-abi
> ~~~~~~~~~~
>
> Implementation of the ``kernel-abi`` reST-directive.
>
> :copyright: Copyright (C) 2016 Markus Heiser
> :copyright: Copyright (C) 2016-2019 Mauro Carvalho Chehab
> :maintained-by: Mauro Carvalho Chehab <[email protected]>
> :license: GPL Version 2, June 1991 see Linux/COPYING for details.
>
> The ``kernel-abi`` (:py:class:`KernelCmd`) directive calls the
> scripts/get_abi.pl script to parse the Kernel ABI files.
>
> Overview of directive's argument and options.
>
> .. code-block:: rst
>
> .. kernel-abi:: <ABI directory location>
> :debug:
>
> The argument ``<ABI directory location>`` is required. It contains the
> location of the ABI files to be parsed.
>
> ``debug``
> Inserts a code-block with the *raw* reST. Sometimes it is helpful to see
> what reST is generated.
>
> """
>
> import codecs
> import os
> import subprocess
> import sys
>
> from os import path
>
> if __name__ != '__main__':
> from docutils import nodes, statemachine
> from docutils.statemachine import ViewList
> from docutils.parsers.rst import directives, Directive
> from docutils.utils.error_reporting import ErrorString
>
> #
> # AutodocReporter is only good up to Sphinx 1.7
> #
> import sphinx
>
> Use_SSI = sphinx.__version__[:3] >= '1.7'
> if Use_SSI:
> from sphinx.util.docutils import switch_source_input
> else:
> from sphinx.ext.autodoc import AutodocReporter
>
> __version__ = '1.0'
>
> if __name__ != '__main__':
> def setup(app):
>
> app.add_directive("kernel-abi", KernelCmd)
> return dict(
> version = __version__
> , parallel_read_safe = True
> , parallel_write_safe = True
> )
>
> class KernelCmd(Directive):
>
> u"""KernelABI (``kernel-abi``) directive"""
>
> required_arguments = 1
> optional_arguments = 2
> has_content = False
> final_argument_whitespace = True
>
> option_spec = {
> "debug" : directives.flag,
> "rst" : directives.unchanged
> }
>
> def warn(self, message, **replace):
> replace["fname"] = self.state.document.current_source
> replace["line_no"] = replace.get("line_no", self.lineno)
> message = ("%(fname)s:%(line_no)s: [kernel-abi WARN] : " + message) % replace
> self.state.document.settings.env.app.warn(message, prefix="")
>
> def run(self):
>
> doc = self.state.document
> if not doc.settings.file_insertion_enabled:
> raise self.warning("docutils: file insertion disabled")
>
> env = doc.settings.env
> cwd = path.dirname(doc.current_source)
> fname = self.arguments[0]
>
> cmd = "get_abi.pl rest --dir "
> cmd += fname
>
> if 'rst' in self.options:
> cmd += " --rst-source"
>
> srctree = path.abspath(os.environ["srctree"])
>
> # extend PATH with $(srctree)/scripts
> path_env = os.pathsep.join([
> srctree + os.sep + "scripts",
> os.environ["PATH"]
> ])
> shell_env = os.environ.copy()
> shell_env["PATH"] = path_env
> shell_env["srctree"] = srctree
>
> lines = self.runCmd(cmd, shell=True, cwd=cwd, env=shell_env)
> nodeList = self.nestedParse(lines, self.arguments[0])
> return nodeList
>
> def runCmd(self, cmd, **kwargs):
> u"""Run command ``cmd`` and return it's stdout as unicode."""
>
> try:
> proc = subprocess.Popen(
> cmd
> , stdout = subprocess.PIPE
> , stderr = subprocess.PIPE
> , **kwargs
> )
> out, err = proc.communicate()
>
> out, err = codecs.decode(out, 'utf-8'), codecs.decode(err, 'utf-8')
>
> if proc.returncode != 0:
> raise self.severe(
> u"command '%s' failed with return code %d"
> % (cmd, proc.returncode)
> )
> except OSError as exc:
> raise self.severe(u"problems with '%s' directive: %s."
> % (self.name, ErrorString(exc)))
> return out
>
> def nestedParse(self, lines, fname):
> content = ViewList()
> node = nodes.section()
>
> if "debug" in self.options:
> code_block = "\n\n.. code-block:: rst\n :linenos:\n"
> for l in lines.split("\n"):
> code_block += "\n " + l
> lines = code_block + "\n\n"
>
> for c, l in enumerate(lines.split("\n")):
> content.append(l, fname, c)
>
> buf = self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter
>
> if Use_SSI:
> with switch_source_input(self.state, content):
> self.state.nested_parse(content, 0, node, match_titles=1)
> else:
> self.state.memo.title_styles = []
> self.state.memo.section_level = 0
> self.state.memo.reporter = AutodocReporter(content, self.state.memo.reporter)
> try:
> self.state.nested_parse(content, 0, node, match_titles=1)
> finally:
> self.state.memo.title_styles, self.state.memo.section_level, self.state.memo.reporter = buf
>
> return node.children

--
Jani Nikula, Intel Open Source Graphics Center

2019-06-27 10:52:50