From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org,
        kernel-team@meta.com, mingo@kernel.org
Cc:     stern@rowland.harvard.edu, parri.andrea@gmail.com, will@kernel.org,
        peterz@infradead.org, boqun.feng@gmail.com, npiggin@gmail.com,
        dhowells@redhat.com, j.alglave@ucl.ac.uk, luc.maranget@inria.fr,
        akiyks@gmail.com, "Paul E. McKenney" <paulmck@kernel.org>
Subject: [PATCH memory-model scripts 31/31] tools/memory-model: Document LKMM test procedure
Date:   Mon, 20 Mar 2023 18:05:49 -0700
Message-Id: <20230321010549.51296-31-paulmck@kernel.org>
In-Reply-To: <4e5839bb-e980-4931-a550-3548d025a32a@paulmck-laptop>
References: <4e5839bb-e980-4931-a550-3548d025a32a@paulmck-laptop>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

This commit documents how to run the various scripts in order to test
a potentially pervasive change to the memory model.

Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
---
 tools/memory-model/scripts/README | 32 +++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/tools/memory-model/scripts/README b/tools/memory-model/scripts/README
index cc2c4e5be9ec..fb39bd0fd1b9 100644
--- a/tools/memory-model/scripts/README
+++ b/tools/memory-model/scripts/README
@@ -76,3 +76,35 @@ runlitmushist.sh
 README
 
 	This file
+
+Testing a change to LKMM might go as follows:
+
+	# Populate expected results without that change, and
+	# runs for about an hour on an 8-CPU x86 system:
+	scripts/initlitmushist.sh --timeout 10m --procs 10
+	# Incorporate the change:
+	git am -s -3 /path/to/patch # Or whatever it takes.
+
+	# Test the new version of LKMM as follows...
+
+	# Runs in seconds, good smoke test:
+	scripts/checkalllitmus.sh
+
+	# Compares results to those produced by initlitmushist.sh,
+	# and runs for about an hour on an 8-CPU x86 system:
+	scripts/checklitmushist.sh --timeout 10m --procs 10
+
+	# Checks results against Result tags, runs in minutes:
+	scripts/checkghlitmus.sh --timeout 10m --procs 10
+
+The checkghlitmus.sh should not report errors in cases where the
+checklitmushist.sh script did not also report a change.  However,
+this check is nevertheless valuable because it can find errors in the
+original version of LKMM.  Note however, that given the above procedure,
+an error in the original LKMM version that is fixed by the patch will
+be reported both as a mismatch by checklitmushist.sh and as an error
+by checkghlitmus.sh.  One exception to this rule of thumb is when the
+test fails completely on the original version of LKMM and passes on the
+new version.  In this case, checklitmushist.sh will report a mismatch
+and checkghlitmus.sh will report success.  This happens when the change
+to LKMM introduces a new primitive for which litmus tests already existed.
-- 
2.40.0.rc2