Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753252AbbHQHdh (ORCPT ); Mon, 17 Aug 2015 03:33:37 -0400 Received: from mx1.redhat.com ([209.132.183.28]:36522 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751481AbbHQHdg (ORCPT ); Mon, 17 Aug 2015 03:33:36 -0400 Date: Mon, 17 Aug 2015 09:33:31 +0200 From: Jiri Olsa To: Borislav Petkov Cc: linux-kernel@vger.kernel.org, x86@kernel.org, Peter Zijlstra , Ingo Molnar , Robert Richter , "H. Peter Anvin" , Thomas Gleixner , Arnaldo Carvalho de Melo , Namhyung Kim , Jan Stancek , Andy Lutomirski Subject: Re: [BUG/RFC] perf test fails on AMD CPUs Message-ID: <20150817073331.GA22373@krava.brq.redhat.com> References: <20150816222956.GA14290@krava.brq.redhat.com> <20150817043603.GB9387@nazgul.tnic> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20150817043603.GB9387@nazgul.tnic> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 7899 Lines: 140 On Mon, Aug 17, 2015 at 06:36:03AM +0200, Borislav Petkov wrote: > On Mon, Aug 17, 2015 at 12:29:56AM +0200, Jiri Olsa wrote: > > hi, > > 'perf test 18' is failing on systems with AMD processor. > > Hmm, still using that b0rked test box? :-) heh, nope.. have seen this on at least 2 boxes so far > > Also, which kernel? oops, it's latest tip master 4fa1239db610 Merge branch 'ras/core' jirka > > There have been substantial changes to the entry code recently. Although > I don't see anything being done differently on AMD there except > X86_BUG_SYSRET_SS_ATTRS but that should be unrelated. > > > The only reason I could find is that AMD does not set 'resume flag' > > in RFLAGS register the way the Intel CPU does. > > > > (simplified) test scenario: > > > > - create breakpoint (on test_function) perf event with SIGIO signal > > to be delivered any time the breakpoint is hit > > - run test_function > > > > > > expected course of actions is: > > 1) CPU hits 'test_function' > > 2) DB exception is triggered, with RFLAGS.RF=0 > > 3) DB exception handler sets regs->RFLAGS.RF=1 and perf handler > > triggers irq_work pending work > > 4) DB exception executes iretd > > 5) irq_work interrupt is triggered, with RFLAGS.RF=1 > > 6) irq_work interrupt calls kill_fasync with SIGIO signal > > 7) irq_work interrupt on return to userspace calls prepare_exit_to_usermode > > which actually delivers the SIGIO signal > > 8) sigreturn syscall prepare registers to return to the > > instruction from step 1) and sets RFLAGS.RF to the its original > > value from step 5) (RFLAGS.RF=1) > > 9) CPU hits 'test_function' and DB exception is NOT triggered > > due to RFLAGS.RF=1 > > > > this is how I see it works on Intel > > > > But AMD gives me RFLAGS.RF=0 on step 5, which makes the step 9 to > > trigger the DB exception once again and makes the test fail. > > Adding Andy, he might have an idea. Leaving in the rest for reference. > > > I'm not sure this test ever worked on AMD CPUs, anyway is there > > anything I'm missing or is this some AMD/Intel quirk? > > > > thanks, > > jirka > > > > > > > > AMD description of RF flag (SDM 3.1.6): > > ======================================= > > Resume Flag (RF) Bit. Bit 16. The RF bit allows an instruction to be restarted following an > > instruction breakpoint resulting in a debug exception (#DB). This bit prevents multiple debug > > exceptions from occurring on the same instruction. > > The processor clears the RF bit after every instruction is successfully executed, except when the > > instruction is: > > • > > • > > An IRET that sets the RF bit. > > JMP, CALL, or INTn through a task gate. > > In both of the above cases, RF is not cleared to 0 until the next instruction successfully executes. > > When an exception occurs (or when a string instruction is interrupted), the processor normally sets > > RF=1 in the RFLAGS image saved on the interrupt stack. However, when a #DB exception occurs as a > > result of an instruction breakpoint, the processor clears the RF bit to 0 in the interrupt-stack RFLAGS > > image. > > For instruction restart to work properly following an instruction breakpoint, the #DB exception > > handler must set RF to 1 in the interrupt-stack RFLAGS image. When an IRET is later executed to > > return to the instruction that caused the instruction-breakpoint #DB exception, the set RF bit (RF=1) is > > loaded from the interrupt-stack RFLAGS image. RF is not cleared by the processor until the > > instruction causing the #DB exception successfully executes. > > > > Intel description of RF flag (SDM 17.3.1.1): > > ============================================ > > Because the debug exception for an instruction breakpoint is generated before the instruction is executed, if the > > instruction breakpoint is not removed by the exception handler; the processor will detect the instruction breakpoint > > again when the instruction is restarted and generate another debug exception. To prevent looping on an instruction > > breakpoint, the Intel 64 and IA-32 architectures provide the RF flag (resume flag) in the EFLAGS register (see > > Section 2.3, “System Flags and Fields in the EFLAGS Register,” in the Intel® 64 and IA-32 Architectures Software > > Developer’s Manual, Volume 3A). When the RF flag is set, the processor ignores instruction breakpoints. > > All Intel 64 and IA-32 processors manage the RF flag as follows. The RF Flag is cleared at the start of the instruction > > after the check for code breakpoint, CS limit violation and FP exceptions. Task Switches and IRETD/IRETQ instruc- > > tions transfer the RF image from the TSS/stack to the EFLAGS register. > > When calling an event handler, Intel 64 and IA-32 processors establish the value of the RF flag in the EFLAGS image > > pushed on the stack: > > • For any fault-class exception except a debug exception generated in response to an instruction breakpoint, the > > value pushed for RF is 1. > > • For any interrupt arriving after any iteration of a repeated string instruction but the last iteration, the value > > pushed for RF is 1. > > • For any trap-class exception generated by any iteration of a repeated string instruction but the last iteration, > > the value pushed for RF is 1. > > • For other cases, the value pushed for RF is the value that was in EFLAG.RF at the time the event handler was > > called. This includes: > > — Debug exceptions generated in response to instruction breakpoints > > — Hardware-generated interrupts arriving between instructions (including those arriving after the last > > iteration of a repeated string instruction) > > — Trap-class exceptions generated after an instruction completes (including those generated after the last > > iteration of a repeated string instruction) > > — Software-generated interrupts (RF is pushed as 0, since it was cleared at the start of the software interrupt) > > As noted above, the processor does not set the RF flag prior to calling the debug exception handler for debug > > exceptions resulting from instruction breakpoints. The debug exception handler can prevent recurrence of the > > instruction breakpoint by setting the RF flag in the EFLAGS image on the stack. If the RF flag in the EFLAGS image > > 17-8 Vol. 3BDEBUG, BRANCH PROFILE, TSC, AND RESOURCE MONITORING FEATURES > > is set when the processor returns from the exception handler, it is copied into the RF flag in the EFLAGS register by > > IRETD/IRETQ or a task switch that causes the return. The processor then ignores instruction breakpoints for the > > duration of the next instruction. (Note that the POPF, POPFD, and IRET instructions do not transfer the RF image > > into the EFLAGS register.) Setting the RF flag does not prevent other types of debug-exception conditions (such as, > > I/O or data breakpoints) from being detected, nor does it prevent non-debug exceptions from being generated. > > For the Pentium processor, when an instruction breakpoint coincides with another fault-type exception (such as a > > page fault), the processor may generate one spurious debug exception after the second exception has been > > handled, even though the debug exception handler set the RF flag in the EFLAGS image. To prevent a spurious > > exception with Pentium processors, all fault-class exception handlers should set the RF flag in the EFLAGS image. > > -- > Regards/Gruss, > Boris. > > ECO tip #101: Trim your mails when you reply. > > SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton, HRB 21284 (AG Nürnberg) > -- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/