Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Clark Williams <williams@redhat.com>
To:     John Kacur <jkacur@redhat.com>
Cc:     linux-kernel@vger.kernel.org
Subject: [PATCH] hwlatdetect: disable/enable c-state transitions during detection
Date:   Mon,  6 May 2019 14:40:46 -0500
Message-Id: <20190506194046.13144-1-williams@redhat.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Recent performance tuning problems led me to realize that just running
at fifo:99 and turning off interrupts isn't enough while looking for
BIOS induced latencies. Power savings logic is built into most modern
cpus and so must be disabled while looking for BIOS induced (SMI/NMI)
latencies.

Use the /dev/cpu_dma_latency mechanism to disable c-state transitions
while running the hardware latency detector. Open the file
/dev/cpu_dma_latency and write a 32-bit zero to it, which will prevent
c-state transitions while the file is open.

Signed-off-by: Clark Williams <williams@redhat.com>
---
 src/hwlatdetect/hwlatdetect.py | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/src/hwlatdetect/hwlatdetect.py b/src/hwlatdetect/hwlatdetect.py
index 2c8f9f160419..368079a158b1 100755
--- a/src/hwlatdetect/hwlatdetect.py
+++ b/src/hwlatdetect/hwlatdetect.py
@@ -1,6 +1,6 @@
 #!/usr/bin/python3
 
-# (C) 2018 Clark Williams <williams@redhat.com>
+# (C) 2018,2019 Clark Williams <williams@redhat.com>
 # (C) 2015,2016 Clark Williams <williams@redhat.com>
 # (C) 2009 Clark Williams <williams@redhat.com>
 #
@@ -213,6 +213,22 @@ watch = False
             counts = [ int(x.strip()) for x in p.stdout.readlines()]
         return counts
 
+    # methods for preventing/enabling c-state transitions
+    # openinging /dev/cpu_dma_latency and writeing a 32-bit zero to that file will prevent
+    # c-state transitions while the file descriptor is open.
+    # use c_states_off() to disable c-state transitions
+    # use c_states_on() to close the file descriptor and re-enable c-states
+    #
+    def c_states_off(self):
+        self.dma_latency_handle = os.open("/dev/cpu_dma_latency", os.O_WRONLY)
+        os.write(self.dma_latency_handle, b'\x00\x00\x00\x00')
+        debug("c-states disabled")
+
+    def c_states_on(self):
+        if self.dma_latency_handle:
+            os.close(self.dma_latency_handle)
+        debug("c-states enabled")
+
     def cleanup(self):
         raise RuntimeError("must override base method 'cleanup'!")
 
@@ -235,6 +251,7 @@ watch = False
     def start(self):
         count = 0
         threshold = int(self.get("threshold"))
+        self.c_states_off()
         debug("enabling detector module (threshold: %d)" % threshold)
         self.set("enable", 1)
         while self.get("enable") == 0:
@@ -258,6 +275,7 @@ watch = False
             time.sleep(0.1)
             debug("retrying disable of detector module(%d)" % count)
             self.set("enable", 0)
+        self.c_states_on()
         debug("detector module disabled")
 
     def detect(self):
-- 
2.21.0