Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754112AbbEAPQw (ORCPT ); Fri, 1 May 2015 11:16:52 -0400 Received: from bombadil.infradead.org ([198.137.202.9]:52292 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753543AbbEAPQt (ORCPT ); Fri, 1 May 2015 11:16:49 -0400 Date: Fri, 1 May 2015 17:16:30 +0200 From: Peter Zijlstra To: Ingo Molnar , "H. Peter Anvin" , Thomas Gleixner , Linus Torvalds Cc: linux-kernel@vger.kernel.org, Borislav Petkov , Jakub Jelinek Subject: [PATCH] x86: Optimize variable_test_bit() Message-ID: <20150501151630.GH5029@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.21 (2012-12-30) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2133 Lines: 59 While looking at some asm I noticed we produce the most horrid code for test_bit(): 1a5e: 49 0f a3 30 bt %rsi,(%r8) 1a62: 45 19 c0 sbb %r8d,%r8d 1a65: 45 85 c0 test %r8d,%r8d 1a68: 75 a6 jne 1a10 Since test_bit() doesn't actually have any output variables, we can use asm goto without having to add a memory clobber. This reduces the code to something sensible: 1a12: 49 0f a3 30 bt %rsi,(%r8) 1a16: 72 68 jb 1a80 Signed-off-by: Peter Zijlstra (Intel) --- PS. should we kill the memory clobber for __test_and_change_bit()? It seems inconsistent and out of place. PPS. Jakub, I see gcc5.1 still hasn't got output operands for asm goto; is this something we can get 'fixed' ? arch/x86/include/asm/bitops.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/x86/include/asm/bitops.h b/arch/x86/include/asm/bitops.h index cfe3b954d5e4..bcf4fa77c04f 100644 --- a/arch/x86/include/asm/bitops.h +++ b/arch/x86/include/asm/bitops.h @@ -313,6 +313,15 @@ static __always_inline int constant_test_bit(long nr, const volatile unsigned lo static inline int variable_test_bit(long nr, volatile const unsigned long *addr) { +#ifdef CC_HAVE_ASM_GOTO + asm_volatile_goto ("bt %1, %0\n\t" + "jc %l[cc_label]" + : : "m" (*(unsigned long *)addr), "Ir" (nr) + : : cc_label); + return 0; +cc_label: + return 1; +#else int oldbit; asm volatile("bt %2,%1\n\t" @@ -321,6 +330,7 @@ static inline int variable_test_bit(long nr, volatile const unsigned long *addr) : "m" (*(unsigned long *)addr), "Ir" (nr)); return oldbit; +#endif } #if 0 /* Fool kernel-doc since it doesn't do macros yet */ -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/