Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S965394AbbDVOMB (ORCPT ); Wed, 22 Apr 2015 10:12:01 -0400 Received: from terminus.zytor.com ([198.137.202.10]:37054 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932349AbbDVOL4 (ORCPT ); Wed, 22 Apr 2015 10:11:56 -0400 Date: Wed, 22 Apr 2015 07:11:12 -0700 From: tip-bot for Denys Vlasenko Message-ID: Cc: mingo@kernel.org, luto@amacapital.net, wad@chromium.org, oleg@redhat.com, torvalds@linux-foundation.org, brgerst@gmail.com, hpa@zytor.com, dvlasenk@redhat.com, ast@plumgrid.com, tglx@linutronix.de, keescook@chromium.org, rostedt@goodmis.org, bp@alien8.de, linux-kernel@vger.kernel.org, fweisbec@gmail.com, luto@kernel.org Reply-To: dvlasenk@redhat.com, ast@plumgrid.com, tglx@linutronix.de, mingo@kernel.org, luto@amacapital.net, wad@chromium.org, oleg@redhat.com, brgerst@gmail.com, torvalds@linux-foundation.org, hpa@zytor.com, bp@alien8.de, fweisbec@gmail.com, linux-kernel@vger.kernel.org, luto@kernel.org, keescook@chromium.org, rostedt@goodmis.org In-Reply-To: <1429633649-20169-1-git-send-email-dvlasenk@redhat.com> References: <1429633649-20169-1-git-send-email-dvlasenk@redhat.com> To: linux-tip-commits@vger.kernel.org Subject: [tip:x86/asm] x86/asm/entry/64: Implement better check for canonical addresses Git-Commit-ID: 17be0aec74fb036eb4eb32c2268f3420a034762b X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4435 Lines: 115 Commit-ID: 17be0aec74fb036eb4eb32c2268f3420a034762b Gitweb: http://git.kernel.org/tip/17be0aec74fb036eb4eb32c2268f3420a034762b Author: Denys Vlasenko AuthorDate: Tue, 21 Apr 2015 18:27:29 +0200 Committer: Ingo Molnar CommitDate: Wed, 22 Apr 2015 08:37:56 +0200 x86/asm/entry/64: Implement better check for canonical addresses This change makes the check exact (no more false positives on "negative" addresses). Andy explains: "Canonical addresses either start with 17 zeros or 17 ones. In the old code, we checked that the top (64-47) = 17 bits were all zero. We did this by shifting right by 47 bits and making sure that nothing was left. In the new code, we're shifting left by (64 - 48) = 16 bits and then signed shifting right by the same amount, this propagating the 17th highest bit to all positions to its left. If we get the same value we started with, then we're good to go." While it isn't really important to be fully correct here - almost all addresses we'll ever see will be userspace ones, but OTOH it looks to be cheap enough: the new code uses two more ALU ops but preserves %rcx, allowing to not reload it from pt_regs->cx again. On disassembly level, the changes are: cmp %rcx,0x80(%rsp) -> mov 0x80(%rsp),%r11; cmp %rcx,%r11 shr $0x2f,%rcx -> shl $0x10,%rcx; sar $0x10,%rcx; cmp %rcx,%r11 mov 0x58(%rsp),%rcx -> (eliminated) Signed-off-by: Denys Vlasenko Acked-by: Andy Lutomirski Cc: Alexei Starovoitov Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Brian Gerst Cc: Frederic Weisbecker Cc: H. Peter Anvin Cc: Kees Cook Cc: Linus Torvalds Cc: Oleg Nesterov Cc: Steven Rostedt Cc: Thomas Gleixner Cc: Will Drewry Link: http://lkml.kernel.org/r/1429633649-20169-1-git-send-email-dvlasenk@redhat.com [ Changelog massage. ] Signed-off-by: Ingo Molnar --- arch/x86/kernel/entry_64.S | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/arch/x86/kernel/entry_64.S b/arch/x86/kernel/entry_64.S index c7b2384..3c78a15 100644 --- a/arch/x86/kernel/entry_64.S +++ b/arch/x86/kernel/entry_64.S @@ -410,26 +410,27 @@ syscall_return: * a completely clean 64-bit userspace context. */ movq RCX(%rsp),%rcx - cmpq %rcx,RIP(%rsp) /* RCX == RIP */ + movq RIP(%rsp),%r11 + cmpq %rcx,%r11 /* RCX == RIP */ jne opportunistic_sysret_failed /* * On Intel CPUs, SYSRET with non-canonical RCX/RIP will #GP * in kernel space. This essentially lets the user take over - * the kernel, since userspace controls RSP. It's not worth - * testing for canonicalness exactly -- this check detects any - * of the 17 high bits set, which is true for non-canonical - * or kernel addresses. (This will pessimize vsyscall=native. - * Big deal.) + * the kernel, since userspace controls RSP. * - * If virtual addresses ever become wider, this will need + * If width of "canonical tail" ever becomes variable, this will need * to be updated to remain correct on both old and new CPUs. */ .ifne __VIRTUAL_MASK_SHIFT - 47 .error "virtual address width changed -- SYSRET checks need update" .endif - shr $__VIRTUAL_MASK_SHIFT, %rcx - jnz opportunistic_sysret_failed + /* Change top 16 bits to be the sign-extension of 47th bit */ + shl $(64 - (__VIRTUAL_MASK_SHIFT+1)), %rcx + sar $(64 - (__VIRTUAL_MASK_SHIFT+1)), %rcx + /* If this changed %rcx, it was not canonical */ + cmpq %rcx, %r11 + jne opportunistic_sysret_failed cmpq $__USER_CS,CS(%rsp) /* CS must match SYSRET */ jne opportunistic_sysret_failed @@ -466,8 +467,8 @@ syscall_return: */ syscall_return_via_sysret: CFI_REMEMBER_STATE - /* r11 is already restored (see code above) */ - RESTORE_C_REGS_EXCEPT_R11 + /* rcx and r11 are already restored (see code above) */ + RESTORE_C_REGS_EXCEPT_RCX_R11 movq RSP(%rsp),%rsp USERGS_SYSRET64 CFI_RESTORE_STATE -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/