Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752129AbbHPWaD (ORCPT ); Sun, 16 Aug 2015 18:30:03 -0400 Received: from mx1.redhat.com ([209.132.183.28]:47979 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751312AbbHPWaB (ORCPT ); Sun, 16 Aug 2015 18:30:01 -0400 Date: Mon, 17 Aug 2015 00:29:56 +0200 From: Jiri Olsa To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, Ingo Molnar , Borislav Petkov , Robert Richter , "H. Peter Anvin" , Thomas Gleixner , Arnaldo Carvalho de Melo , Namhyung Kim , Jan Stancek Subject: [BUG/RFC] perf test fails on AMD CPUs Message-ID: <20150816222956.GA14290@krava.brq.redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6683 Lines: 109 hi, 'perf test 18' is failing on systems with AMD processor. The only reason I could find is that AMD does not set 'resume flag' in RFLAGS register the way the Intel CPU does. (simplified) test scenario: - create breakpoint (on test_function) perf event with SIGIO signal to be delivered any time the breakpoint is hit - run test_function expected course of actions is: 1) CPU hits 'test_function' 2) DB exception is triggered, with RFLAGS.RF=0 3) DB exception handler sets regs->RFLAGS.RF=1 and perf handler triggers irq_work pending work 4) DB exception executes iretd 5) irq_work interrupt is triggered, with RFLAGS.RF=1 6) irq_work interrupt calls kill_fasync with SIGIO signal 7) irq_work interrupt on return to userspace calls prepare_exit_to_usermode which actually delivers the SIGIO signal 8) sigreturn syscall prepare registers to return to the instruction from step 1) and sets RFLAGS.RF to the its original value from step 5) (RFLAGS.RF=1) 9) CPU hits 'test_function' and DB exception is NOT triggered due to RFLAGS.RF=1 this is how I see it works on Intel But AMD gives me RFLAGS.RF=0 on step 5, which makes the step 9 to trigger the DB exception once again and makes the test fail. I'm not sure this test ever worked on AMD CPUs, anyway is there anything I'm missing or is this some AMD/Intel quirk? thanks, jirka AMD description of RF flag (SDM 3.1.6): ======================================= Resume Flag (RF) Bit. Bit 16. The RF bit allows an instruction to be restarted following an instruction breakpoint resulting in a debug exception (#DB). This bit prevents multiple debug exceptions from occurring on the same instruction. The processor clears the RF bit after every instruction is successfully executed, except when the instruction is: • • An IRET that sets the RF bit. JMP, CALL, or INTn through a task gate. In both of the above cases, RF is not cleared to 0 until the next instruction successfully executes. When an exception occurs (or when a string instruction is interrupted), the processor normally sets RF=1 in the RFLAGS image saved on the interrupt stack. However, when a #DB exception occurs as a result of an instruction breakpoint, the processor clears the RF bit to 0 in the interrupt-stack RFLAGS image. For instruction restart to work properly following an instruction breakpoint, the #DB exception handler must set RF to 1 in the interrupt-stack RFLAGS image. When an IRET is later executed to return to the instruction that caused the instruction-breakpoint #DB exception, the set RF bit (RF=1) is loaded from the interrupt-stack RFLAGS image. RF is not cleared by the processor until the instruction causing the #DB exception successfully executes. Intel description of RF flag (SDM 17.3.1.1): ============================================ Because the debug exception for an instruction breakpoint is generated before the instruction is executed, if the instruction breakpoint is not removed by the exception handler; the processor will detect the instruction breakpoint again when the instruction is restarted and generate another debug exception. To prevent looping on an instruction breakpoint, the Intel 64 and IA-32 architectures provide the RF flag (resume flag) in the EFLAGS register (see Section 2.3, “System Flags and Fields in the EFLAGS Register,” in the Intel® 64 and IA-32 Architectures Software Developer’s Manual, Volume 3A). When the RF flag is set, the processor ignores instruction breakpoints. All Intel 64 and IA-32 processors manage the RF flag as follows. The RF Flag is cleared at the start of the instruction after the check for code breakpoint, CS limit violation and FP exceptions. Task Switches and IRETD/IRETQ instruc- tions transfer the RF image from the TSS/stack to the EFLAGS register. When calling an event handler, Intel 64 and IA-32 processors establish the value of the RF flag in the EFLAGS image pushed on the stack: • For any fault-class exception except a debug exception generated in response to an instruction breakpoint, the value pushed for RF is 1. • For any interrupt arriving after any iteration of a repeated string instruction but the last iteration, the value pushed for RF is 1. • For any trap-class exception generated by any iteration of a repeated string instruction but the last iteration, the value pushed for RF is 1. • For other cases, the value pushed for RF is the value that was in EFLAG.RF at the time the event handler was called. This includes: — Debug exceptions generated in response to instruction breakpoints — Hardware-generated interrupts arriving between instructions (including those arriving after the last iteration of a repeated string instruction) — Trap-class exceptions generated after an instruction completes (including those generated after the last iteration of a repeated string instruction) — Software-generated interrupts (RF is pushed as 0, since it was cleared at the start of the software interrupt) As noted above, the processor does not set the RF flag prior to calling the debug exception handler for debug exceptions resulting from instruction breakpoints. The debug exception handler can prevent recurrence of the instruction breakpoint by setting the RF flag in the EFLAGS image on the stack. If the RF flag in the EFLAGS image 17-8 Vol. 3BDEBUG, BRANCH PROFILE, TSC, AND RESOURCE MONITORING FEATURES is set when the processor returns from the exception handler, it is copied into the RF flag in the EFLAGS register by IRETD/IRETQ or a task switch that causes the return. The processor then ignores instruction breakpoints for the duration of the next instruction. (Note that the POPF, POPFD, and IRET instructions do not transfer the RF image into the EFLAGS register.) Setting the RF flag does not prevent other types of debug-exception conditions (such as, I/O or data breakpoints) from being detected, nor does it prevent non-debug exceptions from being generated. For the Pentium processor, when an instruction breakpoint coincides with another fault-type exception (such as a page fault), the processor may generate one spurious debug exception after the second exception has been handled, even though the debug exception handler set the RF flag in the EFLAGS image. To prevent a spurious exception with Pentium processors, all fault-class exception handlers should set the RF flag in the EFLAGS image. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/