Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1754000AbbGWSSq (ORCPT <rfc822;w@1wt.eu>);
	Thu, 23 Jul 2015 14:18:46 -0400
Received: from bhuna.collabora.co.uk ([93.93.135.160]:53968 "EHLO
	bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1753904AbbGWSSn (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Thu, 23 Jul 2015 14:18:43 -0400
From: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
To: linux-doc@vger.kernel.org
Cc: Randy Dunlap <rdunlap@infradead.org>,
        Daniel Vetter <daniel.vetter@ffwll.ch>,
        Laurent Pinchart <laurent.pinchart@ideasonboard.com>,
        Jonathan Corbet <corbet@lwn.net>,
        Herbert Xu <herbert@gondor.apana.org.au>,
        Stephan Mueller <smueller@chronox.de>, Michal Marek <mmarek@suse.cz>,
        linux-kernel@vger.kernel.org,
        intel-gfx <intel-gfx@lists.freedesktop.org>,
        dri-devel <dri-devel@lists.freedesktop.org>,
        Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Subject: [PATCH 1/4] scripts/kernel-doc: Adding cross-reference links to html documentation.
Date: Thu, 23 Jul 2015 15:16:24 -0300
Message-Id: <1437675387-25719-2-git-send-email-danilo.cesar@collabora.co.uk>
X-Mailer: git-send-email 2.1.4
In-Reply-To: <1437675387-25719-1-git-send-email-danilo.cesar@collabora.co.uk>
References: <1437675387-25719-1-git-send-email-danilo.cesar@collabora.co.uk>
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
Content-Length: 9024
Lines: 319

Functions, Structs and Parameters definitions on kernel documentation
are pure cosmetic, it only highlights the element.

To ease the navigation in the documentation we should use <links> inside
those tags so readers can easily jump between methods directly.

This was discussed in 2014[1] and is implemented by getting a list
of <refentries> from the DocBook XML to generate a database. Then it looks
for <function>,<structnames> and <paramdef> tags that matches the ones in
the database. As it only links existent references, no broken links are
added.

[1] - lists.freedesktop.org/archives/dri-devel/2014-August/065404.html

Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@collabora.co.uk>
Cc: Randy Dunlap <rdunlap@infradead.org>
Cc: Daniel Vetter <daniel.vetter@ffwll.ch>
Cc: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: Herbert Xu <herbert@gondor.apana.org.au>
Cc: Stephan Mueller <smueller@chronox.de>
Cc: Michal Marek <mmarek@suse.cz>
Cc: linux-kernel@vger.kernel.org
Cc: linux-doc@vger.kernel.org
Cc: intel-gfx <intel-gfx@lists.freedesktop.org>
Cc: dri-devel <dri-devel@lists.freedesktop.org>
---
 Documentation/DocBook/Makefile |  43 ++++++---
 scripts/kernel-doc-xml-ref     | 198 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 228 insertions(+), 13 deletions(-)
 create mode 100755 scripts/kernel-doc-xml-ref

diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
index b6a6a2e..322255b 100644
--- a/Documentation/DocBook/Makefile
+++ b/Documentation/DocBook/Makefile
@@ -64,8 +64,9 @@ installmandocs: mandocs
 
 ###
 #External programs used
-KERNELDOC = $(srctree)/scripts/kernel-doc
-DOCPROC   = $(objtree)/scripts/docproc
+KERNELDOCXMLREF = $(srctree)/scripts/kernel-doc-xml-ref
+KERNELDOC       = $(srctree)/scripts/kernel-doc
+DOCPROC         = $(objtree)/scripts/docproc
 
 XMLTOFLAGS = -m $(srctree)/$(src)/stylesheet.xsl
 XMLTOFLAGS += --skip-validation
@@ -89,7 +90,7 @@ define rule_docproc
         ) > $(dir $@).$(notdir $@).cmd
 endef
 
-%.xml: %.tmpl $(KERNELDOC) $(DOCPROC) FORCE
+%.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE
 	$(call if_changed_rule,docproc)
 
 # Tell kbuild to always build the programs
@@ -140,7 +141,20 @@ quiet_cmd_db2html = HTML    $@
 		echo '<a HREF="$(patsubst %.html,%,$(notdir $@))/index.html"> \
 		$(patsubst %.html,%,$(notdir $@))</a><p>' > $@
 
-%.html:	%.xml
+###
+# Rules to create an aux XML and .db, and use them to re-process the DocBook XML
+# to fill internal hyperlinks
+       gen_aux_xml = :
+ quiet_gen_aux_xml = echo '  XMLREF  $@'
+silent_gen_aux_xml = :
+%.aux.xml: %.xml
+	@$($(quiet)gen_aux_xml)
+	@rm -rf $@
+	@(cat $< | egrep "^<refentry id" | egrep -o "\".*\"" | cut -f 2 -d \" > $<.db)
+	@$(KERNELDOCXMLREF) -db $<.db $< > $@
+.PRECIOUS: %.aux.xml
+
+%.html:	%.aux.xml
 	@(which xmlto > /dev/null 2>&1) || \
 	 (echo "*** You need to install xmlto ***"; \
 	  exit 1)
@@ -209,15 +223,18 @@ dochelp:
 ###
 # Temporary files left by various tools
 clean-files := $(DOCBOOKS) \
-	$(patsubst %.xml, %.dvi,  $(DOCBOOKS)) \
-	$(patsubst %.xml, %.aux,  $(DOCBOOKS)) \
-	$(patsubst %.xml, %.tex,  $(DOCBOOKS)) \
-	$(patsubst %.xml, %.log,  $(DOCBOOKS)) \
-	$(patsubst %.xml, %.out,  $(DOCBOOKS)) \
-	$(patsubst %.xml, %.ps,   $(DOCBOOKS)) \
-	$(patsubst %.xml, %.pdf,  $(DOCBOOKS)) \
-	$(patsubst %.xml, %.html, $(DOCBOOKS)) \
-	$(patsubst %.xml, %.9,    $(DOCBOOKS)) \
+	$(patsubst %.xml, %.dvi,     $(DOCBOOKS)) \
+	$(patsubst %.xml, %.aux,     $(DOCBOOKS)) \
+	$(patsubst %.xml, %.tex,     $(DOCBOOKS)) \
+	$(patsubst %.xml, %.log,     $(DOCBOOKS)) \
+	$(patsubst %.xml, %.out,     $(DOCBOOKS)) \
+	$(patsubst %.xml, %.ps,      $(DOCBOOKS)) \
+	$(patsubst %.xml, %.pdf,     $(DOCBOOKS)) \
+	$(patsubst %.xml, %.html,    $(DOCBOOKS)) \
+	$(patsubst %.xml, %.9,       $(DOCBOOKS)) \
+	$(patsubst %.xml, %.aux.xml, $(DOCBOOKS)) \
+	$(patsubst %.xml, %.xml.db,  $(DOCBOOKS)) \
+	$(patsubst %.xml, %.xml,     $(DOCBOOKS)) \
 	$(index)
 
 clean-dirs := $(patsubst %.xml,%,$(DOCBOOKS)) man
diff --git a/scripts/kernel-doc-xml-ref b/scripts/kernel-doc-xml-ref
new file mode 100755
index 0000000..104a5a5
--- /dev/null
+++ b/scripts/kernel-doc-xml-ref
@@ -0,0 +1,198 @@
+#!/usr/bin/perl -w
+
+use strict;
+
+## Copyright (C) 2015  Intel Corporation                         ##
+#                                                                ##
+## This software falls under the GNU General Public License.     ##
+## Please read the COPYING file for more information             ##
+#
+#
+# This software reads a XML file and a list of valid interal
+# references to replace Docbook tags with links.
+#
+# The list of "valid internal references" must be one-per-line in the following format:
+#      API-struct-foo
+#      API-enum-bar
+#      API-my-function
+#
+# The software walks over the XML file looking for xml tags representing possible references
+# to the Document. Each reference will be cross checked against the "Valid Internal Reference" list. If
+# the referece is found it replaces its content by a <link> tag.
+#
+# usage:
+# kernel-doc-xml-ref -db filename
+#		     xml filename > outputfile
+
+# read arguments
+if ($#ARGV != 2) {
+	usage();
+}
+
+#Holds the database filename
+my $databasefile;
+my @database;
+
+#holds the inputfile
+my $inputfile;
+my $errors = 0;
+
+my %highlights = (
+	"<function>(.*?)</function>",
+	    "\"<function>\" . convert_function(\$1, \$line) . \"</function>\"",
+	"<structname>(.*?)</structname>",
+	    "\"<structname>\" . convert_struct(\$1) . \"</structname>\"",
+	"<funcdef>(.*?)<function>(.*?)</function></funcdef>",
+	    "\"<funcdef>\" . convert_param(\$1) . \"<function>\$2</function></funcdef>\"",
+	"<paramdef>(.*?)<parameter>(.*?)</parameter></paramdef>",
+	    "\"<paramdef>\" . convert_param(\$1) . \"<parameter>\$2</parameter></paramdef>\"");
+
+while($ARGV[0] =~ m/^-(.*)/) {
+	my $cmd = shift @ARGV;
+	if ($cmd eq "-db") {
+		$databasefile = shift @ARGV
+	} else {
+		usage();
+	}
+}
+$inputfile = shift @ARGV;
+
+sub open_database {
+	open (my $handle, '<', $databasefile) or die "Cannot open $databasefile";
+	chomp(my @lines = <$handle>);
+	close $handle;
+
+	@database = @lines;
+}
+
+sub process_file {
+	open_database();
+
+	my $dohighlight;
+	foreach my $pattern (keys %highlights) {
+		$dohighlight .=  "\$line =~ s:$pattern:$highlights{$pattern}:eg;\n";
+	}
+
+	open(FILE, $inputfile) or die("Could not open $inputfile") or die ("Cannot open $inputfile");
+	foreach my $line (<FILE>)  {
+		eval $dohighlight;
+		print $line;
+	}
+}
+
+sub trim($_)
+{
+	my $str = $_[0];
+	$str =~ s/^\s+|\s+$//g;
+	return $str
+}
+
+sub has_key_defined($_)
+{
+	if ( grep( /^$_[0]$/, @database)) {
+		return 1;
+	}
+	return 0;
+}
+
+# Gets a <function> content and add it a hyperlink if possible.
+sub convert_function($_)
+{
+	my $arg = $_[0];
+	my $key = $_[0];
+
+	my $line = $_[1];
+
+	$key = trim($key);
+
+	$key =~ s/[^A-Za-z0-9]/-/g;
+	$key = "API-" . $key;
+
+	# We shouldn't add links to <funcdef> prototype
+	if (!has_key_defined($key) || $line =~ m/\s+<funcdef/i) {
+		return $arg;
+	}
+
+	my $head = $arg;
+	my $tail = "";
+	if ($arg =~ /(.*?)( ?)$/) {
+		$head = $1;
+		$tail = $2;
+	}
+	return "<link linkend=\"$key\">$head</link>$tail";
+}
+
+# Converting a struct text to link
+sub convert_struct($_)
+{
+	my $arg = $_[0];
+	my $key = $_[0];
+	$key =~ s/(struct )?(\w)/$2/g;
+	$key =~ s/[^A-Za-z0-9]/-/g;
+	$key = "API-struct-" . $key;
+
+	if (!has_key_defined($key)) {
+		return $arg;
+	}
+
+	my ($head, $tail) = split_pointer($arg);
+	return "<link linkend=\"$key\">$head</link>$tail";
+}
+
+# Identify "object *" elements
+sub split_pointer($_)
+{
+	my $arg = $_[0];
+	if ($arg =~ /(.*?)( ?\* ?)/) {
+		return ($1, $2);
+	}
+	return ($arg, "");
+}
+
+sub convert_param($_)
+{
+	my $type = $_[0];
+	my $keyname = convert_key_name($type);
+
+	if (!has_key_defined($keyname)) {
+		return $type;
+	}
+
+	my ($head, $tail) = split_pointer($type);
+	return "<link linkend=\"$keyname\">$head</link>$tail";
+
+}
+
+# DocBook links are in the API-<TYPE>-<STRUCT-NAME> format
+# This method gets an element and returns a valid DocBook reference for it.
+sub convert_key_name($_)
+{
+	#Pattern $2 is optional and might be uninitialized
+	no warnings 'uninitialized';
+
+	my $str = $_[0];
+	$str =~ s/(const|static)? ?(struct)? ?([a-zA-Z0-9_]+) ?(\*|&)?/$2 $3/g ;
+
+	# trim
+	$str =~ s/^\s+|\s+$//g;
+
+	# spaces and _ to -
+	$str =~ s/[^A-Za-z0-9]/-/g;
+
+	return "API-" . $str;
+}
+
+sub usage {
+	print "Usage: $0 -db database filename\n";
+	print "         xml source file(s) > outputfile\n";
+	exit 1;
+}
+
+# starting point
+process_file();
+
+if ($errors) {
+	print STDERR "$errors errors\n";
+}
+
+exit($errors);
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/