Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754705AbbEOJlH (ORCPT ); Fri, 15 May 2015 05:41:07 -0400 Received: from terminus.zytor.com ([198.137.202.10]:60066 "EHLO terminus.zytor.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754293AbbEOJlC (ORCPT ); Fri, 15 May 2015 05:41:02 -0400 Date: Fri, 15 May 2015 02:39:34 -0700 From: tip-bot for Ingo Molnar Message-ID: Cc: mingo@kernel.org, tglx@linutronix.de, linux-kernel@vger.kernel.org, dave@stgolabs.net, a.p.zijlstra@chello.nl, hpa@zytor.com, jason.low2@hp.com, aswin@hp.com, luto@amacapital.net, tim.c.chen@linux.intel.com, torvalds@linux-foundation.org, paulmck@linux.vnet.ibm.com, peterz@infradead.org, bp@alien8.de, dvlasenk@redhat.com, brgerst@gmail.com Reply-To: dave@stgolabs.net, linux-kernel@vger.kernel.org, tglx@linutronix.de, mingo@kernel.org, brgerst@gmail.com, peterz@infradead.org, dvlasenk@redhat.com, bp@alien8.de, paulmck@linux.vnet.ibm.com, torvalds@linux-foundation.org, tim.c.chen@linux.intel.com, luto@amacapital.net, aswin@hp.com, jason.low2@hp.com, hpa@zytor.com, a.p.zijlstra@chello.nl In-Reply-To: <20150410120846.GA17101@gmail.com> References: <20150410120846.GA17101@gmail.com> To: linux-tip-commits@vger.kernel.org Subject: [tip:x86/asm] x86: Align jump targets to 1-byte boundaries Git-Commit-ID: be6cb02779ca74d83481f017db21578cfe92891c X-Mailer: tip-git-log-daemon Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8 Content-Disposition: inline Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4970 Lines: 124 Commit-ID: be6cb02779ca74d83481f017db21578cfe92891c Gitweb: http://git.kernel.org/tip/be6cb02779ca74d83481f017db21578cfe92891c Author: Ingo Molnar AuthorDate: Fri, 10 Apr 2015 14:08:46 +0200 Committer: Ingo Molnar CommitDate: Fri, 15 May 2015 11:04:28 +0200 x86: Align jump targets to 1-byte boundaries The following NOP in a hot function caught my attention: > 5a: 66 0f 1f 44 00 00 nopw 0x0(%rax,%rax,1) That's a dead NOP that bloats the function a bit, added for the default 16-byte alignment that GCC applies for jump targets. I realize that x86 CPU manufacturers recommend 16-byte jump target alignments (it's in the Intel optimization manual), to help their relatively narrow decoder prefetch alignment and uop cache constraints, but the cost of that is very significant: text data bss dec filename 12566391 1617840 1089536 15273767 vmlinux.align.16-byte 12224951 1617840 1089536 14932327 vmlinux.align.1-byte By using 1-byte jump target alignment (i.e. no alignment at all) we get an almost 3% reduction in kernel size (!) - and a probably similar reduction in I$ footprint. Now, the usual justification for jump target alignment is the following: - modern decoders tend to have 16-byte (effective) decoder prefetch windows. (AMD documents it higher but measurements suggest the effective prefetch window on curretn uarchs is still around 16 bytes) - on Intel there's also the uop-cache with cachelines that have 16-byte granularity and limited associativity. - older x86 uarchs had a penalty for decoder fetches that crossed 16-byte boundaries. These limits are mostly gone from recent uarchs. So if a forward jump target is aligned to cacheline boundary then prefetches will start from a new prefetch-cacheline and there's higher chance for decoding in fewer steps and packing tightly. But I think that argument is flawed for typical optimized kernel code flows: forward jumps often go to 'cold' (uncommon) pieces of code, and aligning cold code to cache lines does not bring a lot of advantages (they are uncommon), while it causes collateral damage: - their alignment 'spreads out' the cache footprint, it shifts followup hot code further out - plus it slows down even 'cold' code that immediately follows 'hot' code (like in the above case), which could have benefited from the partial cacheline that comes off the end of hot code. But even in the cache-hot case the 16 byte alignment brings disadvantages: - it spreads out the cache footprint, possibly making the code fall out of the L1 I$. - On Intel CPUs, recent microarchitectures have plenty of uop cache (typically doubling every 3 years) - while the size of the L1 cache grows much less aggressively. So workloads are rarely uop cache limited. The only situation where alignment might matter are tight loops that could fit into a single 16 byte chunk - but those are pretty rare in the kernel: if they exist they tend to be pointer chasing or generic memory ops, which both tend to be cache miss (or cache allocation) intensive and are not decoder bandwidth limited. So the balance of arguments strongly favors packing kernel instructions tightly versus maximizing for decoder bandwidth: this patch changes the jump target alignment from 16 bytes to 1 byte (tightly packed, unaligned). Acked-by: Denys Vlasenko Cc: Andy Lutomirski Cc: Aswin Chandramouleeswaran Cc: Borislav Petkov Cc: Brian Gerst Cc: Davidlohr Bueso Cc: H. Peter Anvin Cc: Jason Low Cc: Linus Torvalds Cc: Paul E. McKenney Cc: Peter Zijlstra Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: Tim Chen Link: http://lkml.kernel.org/r/20150410120846.GA17101@gmail.com Signed-off-by: Ingo Molnar --- arch/x86/Makefile | 3 +++ 1 file changed, 3 insertions(+) diff --git a/arch/x86/Makefile b/arch/x86/Makefile index c7c3187..ca17e5f 100644 --- a/arch/x86/Makefile +++ b/arch/x86/Makefile @@ -77,6 +77,9 @@ else KBUILD_AFLAGS += -m64 KBUILD_CFLAGS += -m64 + # Align jump targets to 1 byte, not the default 16 bytes: + KBUILD_CFLAGS += -falign-jumps=1 + # Don't autogenerate traditional x87 instructions KBUILD_CFLAGS += $(call cc-option,-mno-80387) KBUILD_CFLAGS += $(call cc-option,-mno-fp-ret-in-387) -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/