Received: by 2002:a25:c593:0:0:0:0:0 with SMTP id v141csp2111288ybe; Tue, 3 Sep 2019 08:09:09 -0700 (PDT) X-Google-Smtp-Source: APXvYqwJfZru/BB57xRz2F2+y7lzlJ7vusExNguQrMICtcLzdzmUvqrx3HBwc3q9uS58QDhIvfby X-Received: by 2002:a63:481c:: with SMTP id v28mr30399116pga.50.1567523349307; Tue, 03 Sep 2019 08:09:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1567523349; cv=none; d=google.com; s=arc-20160816; b=JjhLNOI+tGU2u2jkzakUqQugdOtfzE98Yru+EvvCUvr7fiYuWMqmRxutl9yEWuJU2u zoNWg2zAwBCPgG4xs0UKYS7idlCpZzOsQ0Scy20WQLsd9nZfr+Ep/+h9BGPcRrzNCg52 f9HvSS38hdwwGlDNHAEkaKZvB3hwpLXin57fFdm/zPtlPNBsagrBD1cWJH4sCF7PH2x5 tKlXTjSjfSlRmkG8kdbu4jZVMd9cuTdFomFg358NPtbFWQu8YErwEZlnA+XlkSKSDxG4 aFVeT79M38p8VAwcKM2YPZDHoF/zG2J5JHaDnteWOXOmVzY6HLmPjm2b1yfbbInJQX3x A5+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:from:subject:references :mime-version:message-id:in-reply-to:date:dkim-signature; bh=b+cqXt6YX0mnoCtMoAi03cOiwWu215/p06/ubbgFzR4=; b=TeWbRF2d5ElgOCtMAkSAX6UnWL64qayTO+2gU3FasnZczAkZCb2fPcWP3f25epjbUx 7Aj7N2nQAO7MBV/LMz57I4c9bYAlo6ygTqoyDlyNwRhFCEQd2Fg+2zjF/93mpIaFJjA7 l1Zo2+qT8/iyh1FMxasFg0r9YXwXcSt0BKPnXBsIrSGPL+OSXUfY43FR+h25mo6jVcSd FkW0d9uS0UGmJXZZP8KIGvrZD6OMmmRMwFk2sz0JNLG4ohYYgJGpDcbAlvnv7ft52s9T +Xrt/xi4Sk2ND5BmsZrOaATgaNa1T4kya2lt2GOTrDEb8UsXfAKQc6KUaxNGtdSo8xeg XQjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=pghRn16I; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id h31si14719160pgm.129.2019.09.03.08.08.53; Tue, 03 Sep 2019 08:09:09 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=pghRn16I; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729897AbfICPHt (ORCPT + 99 others); Tue, 3 Sep 2019 11:07:49 -0400 Received: from mail-yb1-f202.google.com ([209.85.219.202]:47972 "EHLO mail-yb1-f202.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729879AbfICPHs (ORCPT ); Tue, 3 Sep 2019 11:07:48 -0400 Received: by mail-yb1-f202.google.com with SMTP id y12so5050269ybg.14 for ; Tue, 03 Sep 2019 08:07:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=b+cqXt6YX0mnoCtMoAi03cOiwWu215/p06/ubbgFzR4=; b=pghRn16IfODUbRoRofiFw9344SQdsDElwKyv5Z95AmcH9uCOS/7iVJBCyGHvc/XCEy gkY5kuzQ6cLmq8hF+aYXS8YpZRO1LElUudHSnNOlAg92jvEAVfMIX0sXYM+j80ne2XN5 ew4kT/L8zajaJGRIn38fVNp6Bga8rn18WL17+ND4o5SoAMvp4PhgdP+vUMjEOyttyKeo 8I4AKstWoIagp2ogkgcbLX8OT1rZ164y6do+U7Ih2Vm0fqPQXkeDQscyE/S/Fqfu824T 5jcb0MlM+415vcNXYgIcezGLpu/FmCO736XMgZ58KY59Ay0KHzA9qZHW8kUBknr4+GiS qE3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=b+cqXt6YX0mnoCtMoAi03cOiwWu215/p06/ubbgFzR4=; b=Mk5il7Pxtw5QJoraWFSfHCJMwXlAC0vHB9+a4b6lGpW4oArrpVHEabIUiZ9v1LP64+ tPNtWamSvgicagSw6C8wnyb2OJGSo8qlkNjgEtTsTcXlmi5hi/SVPjIClbceGKI+FTBG 2jiHejomR+GXqy+hNhZzPKcqx+9BovldncXnL4oQ60uWmPHVzCqfClaZWOVD+LuhHlQr dbiwjnkA8fNjtyMFoUuvbTLLXCPLCj5FMUpCHSDVdQPrkoVOdZQIoQ9OY2EX4SHKrlzB FzqznhtD35rTDrnnG++YNnvkG5cA0gDDkzIlN1vJkIq5674/dRgdug5mltNOgvMD7+kT 9+IQ== X-Gm-Message-State: APjAAAWBD9mM16Y61jJhjYMm6MzPtQVzNMakJn4lTFOdYwPyMUeMh8Ah mHBpz9ZKsp7wsMwM8zMNmhdY925nDe/69VzeY+9keKEQdA7UbFH+22RKH1oDgc7yxp2bKDMsP8h ICbuBnuCuS3/Bxalfqdley/86ypNpwwHmV5pDnntlW6NFMIWIZSdgfUEHZcyIKy0z5Z8dFbbViv 4= X-Received: by 2002:a81:84d0:: with SMTP id u199mr17120277ywf.284.1567523267075; Tue, 03 Sep 2019 08:07:47 -0700 (PDT) Date: Tue, 3 Sep 2019 16:06:34 +0100 In-Reply-To: <20190903150638.242049-1-maennich@google.com> Message-Id: <20190903150638.242049-9-maennich@google.com> Mime-Version: 1.0 References: <20180716122125.175792-1-maco@android.com> <20190903150638.242049-1-maennich@google.com> X-Mailer: git-send-email 2.23.0.187.g17f5b7556c-goog Subject: [PATCH v4 08/12] scripts: Coccinelle script for namespace dependencies. From: Matthias Maennich To: linux-kernel@vger.kernel.org Cc: kernel-team@android.com, maennich@google.com, arnd@arndb.de, gregkh@linuxfoundation.org, jeyu@kernel.org, joel@joelfernandes.org, lucas.de.marchi@gmail.com, maco@android.com, sspatil@google.com, will@kernel.org, yamada.masahiro@socionext.com, linux-kbuild@vger.kernel.org, linux-modules@vger.kernel.org, linux-usb@vger.kernel.org, usb-storage@lists.one-eyed-alien.net, linux-watchdog@vger.kernel.org, Julia Lawall Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org A script that uses the '.ns_deps' files generated by modpost to automatically add the required symbol namespace dependencies to each module. Usage: 1) Move some symbols to a namespace with EXPORT_SYMBOL_NS() or define DEFAULT_SYMBOL_NAMESPACE 2) Run 'make' (or 'make modules') and get warnings about modules not importing that namespace. 3) Run 'make nsdeps' to automatically add required import statements to said modules. This makes it easer for subsystem maintainers to introduce and maintain symbol namespaces into their codebase. Co-developed-by: Martijn Coenen Signed-off-by: Martijn Coenen Acked-by: Julia Lawall Reviewed-by: Greg Kroah-Hartman Signed-off-by: Matthias Maennich --- MAINTAINERS | 5 ++ Makefile | 12 +++++ scripts/Makefile.modpost | 4 +- scripts/coccinelle/misc/add_namespace.cocci | 23 ++++++++ scripts/nsdeps | 60 +++++++++++++++++++++ 5 files changed, 103 insertions(+), 1 deletion(-) create mode 100644 scripts/coccinelle/misc/add_namespace.cocci create mode 100644 scripts/nsdeps diff --git a/MAINTAINERS b/MAINTAINERS index e7a47b5210fd..3b170c11a0e6 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -11436,6 +11436,11 @@ S: Maintained T: git git://git.kernel.org/pub/scm/linux/kernel/git/wtarreau/nolibc.git F: tools/include/nolibc/ +NSDEPS +M: Matthias Maennich +S: Maintained +F: scripts/nsdeps + NTB AMD DRIVER M: Shyam Sundar S K L: linux-ntb@googlegroups.com diff --git a/Makefile b/Makefile index 0cdb957ae2c3..573e90e738ad 100644 --- a/Makefile +++ b/Makefile @@ -1500,6 +1500,9 @@ help: @echo ' headerdep - Detect inclusion cycles in headers' @echo ' coccicheck - Check with Coccinelle' @echo '' + @echo 'Tools:' + @echo ' nsdeps - Generate missing symbol namespace dependencies' + @echo '' @echo 'Kernel selftest:' @echo ' kselftest - Build and run kernel selftest (run as root)' @echo ' Build, install, and boot kernel before' @@ -1687,6 +1690,15 @@ quiet_cmd_tags = GEN $@ tags TAGS cscope gtags: FORCE $(call cmd,tags) +# Script to generate missing namespace dependencies +# --------------------------------------------------------------------------- + +PHONY += nsdeps + +nsdeps: + $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost nsdeps + $(Q)$(CONFIG_SHELL) $(srctree)/scripts/$@ + # Scripts to check various things for consistency # --------------------------------------------------------------------------- diff --git a/scripts/Makefile.modpost b/scripts/Makefile.modpost index 26e6574ecd08..743fe3a2e885 100644 --- a/scripts/Makefile.modpost +++ b/scripts/Makefile.modpost @@ -56,7 +56,8 @@ MODPOST = scripts/mod/modpost \ $(if $(KBUILD_EXTMOD),$(addprefix -e ,$(KBUILD_EXTRA_SYMBOLS))) \ $(if $(KBUILD_EXTMOD),-o $(modulesymfile)) \ $(if $(CONFIG_SECTION_MISMATCH_WARN_ONLY),,-E) \ - $(if $(KBUILD_MODPOST_WARN),-w) + $(if $(KBUILD_MODPOST_WARN),-w) \ + $(if $(filter nsdeps,$(MAKECMDGOALS)),-d) ifdef MODPOST_VMLINUX @@ -134,6 +135,7 @@ $(modules): %.ko :%.o %.mod.o FORCE targets += $(modules) +nsdeps: __modpost # Add FORCE to the prequisites of a target to force it to be always rebuilt. # --------------------------------------------------------------------------- diff --git a/scripts/coccinelle/misc/add_namespace.cocci b/scripts/coccinelle/misc/add_namespace.cocci new file mode 100644 index 000000000000..c832bb6445a8 --- /dev/null +++ b/scripts/coccinelle/misc/add_namespace.cocci @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0-only +// +/// Adds missing MODULE_IMPORT_NS statements to source files +/// +/// This script is usually called from scripts/nsdeps with -D ns= to +/// add a missing namespace tag to a module source file. +/// + +@has_ns_import@ +declarer name MODULE_IMPORT_NS; +identifier virtual.ns; +@@ +MODULE_IMPORT_NS(ns); + +// Add missing imports, but only adjacent to a MODULE_LICENSE statement. +// That ensures we are adding it only to the main module source file. +@do_import depends on !has_ns_import@ +declarer name MODULE_LICENSE; +expression license; +identifier virtual.ns; +@@ +MODULE_LICENSE(license); ++ MODULE_IMPORT_NS(ns); diff --git a/scripts/nsdeps b/scripts/nsdeps new file mode 100644 index 000000000000..75dcf79a11ed --- /dev/null +++ b/scripts/nsdeps @@ -0,0 +1,60 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Linux kernel symbol namespace import generator +# +# This script requires a minimum spatch version. +SPATCH_REQ_VERSION="1.0.4" + +DIR="$(dirname $(readlink -f $0))/.." +SPATCH="`which ${SPATCH:=spatch}`" +if [ ! -x "$SPATCH" ]; then + echo 'spatch is part of the Coccinelle project and is available at http://coccinelle.lip6.fr/' + exit 1 +fi + +SPATCH_REQ_VERSION_NUM=$(echo $SPATCH_REQ_VERSION | ${DIR}/scripts/ld-version.sh) +SPATCH_VERSION=$($SPATCH --version | head -1 | awk '{print $3}') +SPATCH_VERSION_NUM=$(echo $SPATCH_VERSION | ${DIR}/scripts/ld-version.sh) + +if [ "$SPATCH_VERSION_NUM" -lt "$SPATCH_REQ_VERSION_NUM" ] ; then + echo "spatch needs to be version $SPATCH_REQ_VERSION or higher" + exit 1 +fi + +set -e + +generate_deps_for_ns() { + $SPATCH --very-quiet --in-place --sp-file \ + $srctree/scripts/coccinelle/misc/add_namespace.cocci -D ns=$1 $2 +} + +generate_deps() { + local mod_name=`basename $@ .ko` + local mod_file=`echo $@ | sed -e 's/\.ko/\.mod/'` + local ns_deps_file=`echo $@ | sed -e 's/\.ko/\.ns_deps/'` + if [ ! -f "$ns_deps_file" ]; then return; fi + local mod_source_files=`cat $mod_file | sed -n 1p \ + | sed -e 's/\.o/\.c/g' \ + | sed "s/[^ ]* */${srctree}\/&/g"` + for ns in `cat $ns_deps_file`; do + echo "Adding namespace $ns to module $mod_name (if needed)." + generate_deps_for_ns $ns $mod_source_files + # sort the imports + for source_file in $mod_source_files; do + sed '/MODULE_IMPORT_NS/Q' $source_file > ${source_file}.tmp + offset=$(wc -l ${source_file}.tmp | awk '{print $1;}') + cat $source_file | grep MODULE_IMPORT_NS | sort -u >> ${source_file}.tmp + tail -n +$((offset +1)) ${source_file} | grep -v MODULE_IMPORT_NS >> ${source_file}.tmp + if ! diff -q ${source_file} ${source_file}.tmp; then + mv ${source_file}.tmp ${source_file} + else + rm ${source_file}.tmp + fi + done + done +} + +for f in `cat $objtree/modules.order`; do + generate_deps $f +done + -- 2.23.0.187.g17f5b7556c-goog