Received: by 2002:ac0:a581:0:0:0:0:0 with SMTP id m1-v6csp235668imm; Tue, 19 Jun 2018 20:06:21 -0700 (PDT) X-Google-Smtp-Source: ADUXVKLrxaEM69tHk2JCH7Mu+r6HA7+hZy6rl5xvQG+4qIbfleydDR204t4g0gm0cRdmBnLrtMEH X-Received: by 2002:a17:902:9a95:: with SMTP id w21-v6mr21600713plp.168.1529463980973; Tue, 19 Jun 2018 20:06:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529463980; cv=none; d=google.com; s=arc-20160816; b=eKx9s7eABfBrp+DUCVQGdUwubO05gJktBeNvK1QHV+DhkAI5oM5neI1ixptloq46JC OUrhWxiVBopYJQ75zyHavn9JVqlZtCEN8cXtNQs7w8YvvhBPOXdKWJJCejZGuXYeGQdp QgNBqsnalee9s1wBCl1+xji0jRNqLzsFWtfff017dND1jBihrkkw2YtXetlk4JyNiw3X mJvqWtd5Q+p5XcDoIUrsQonqBHXmwE/bQstRdWAimWLf8ltTB19KlOrlaQJMXMMm6PdL GIZYm7Hq8iPwQrNoy9NU/cOYr68Z4ArLyDtAMBXdy6lUKrfu6QVrsLmXoK0yjolOYNzF v+Jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:arc-authentication-results; bh=GVJbaTc0oQW5Cv0+i4rLI4+vKNzGehbxm/M2XEHit1M=; b=K5G454hHvksg/ntVj0Yg5ES5BIY/nmD3LHu8yPTDhKbHWf7nklAft3y6XBTwu4Moho bmyIq1De5r41hJBjg51rMK+OssLHq34eaNxlxRcJu/IHEyJo5806y0dvZc5lEjgVi1OO tqu83SN1mNQLFP2DRfqr+WPCAAUQgqdligQLCW/ZXPyosr7/4LfFVl9o3DhlXk7UzxXA NqdbwWATWq2uVMsx84ZoFdgLwebl8hSOLI1OYLh5MQdTjyuQNTceN1khQ+FS5HCFGPHf DaQysUJIvV7sS9ZOFeNJ2GTdQ/nXOqlp3JMM+90mDx7LTgWvfi6ub60B4HHLohYj+rVz PrPA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=vmware.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a10-v6si1485458pln.349.2018.06.19.20.06.07; Tue, 19 Jun 2018 20:06:20 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=vmware.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754547AbeFTDFF (ORCPT + 99 others); Tue, 19 Jun 2018 23:05:05 -0400 Received: from ex13-edg-ou-001.vmware.com ([208.91.0.189]:58289 "EHLO EX13-EDG-OU-001.vmware.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754240AbeFTDDn (ORCPT ); Tue, 19 Jun 2018 23:03:43 -0400 Received: from sc9-mailhost3.vmware.com (10.113.161.73) by EX13-EDG-OU-001.vmware.com (10.113.208.155) with Microsoft SMTP Server id 15.0.1156.6; Tue, 19 Jun 2018 20:03:06 -0700 Received: from sc2-haas01-esx0118.eng.vmware.com (sc2-haas01-esx0118.eng.vmware.com [10.172.44.118]) by sc9-mailhost3.vmware.com (Postfix) with ESMTP id 43DEB40DF4; Tue, 19 Jun 2018 20:03:40 -0700 (PDT) From: Nadav Amit To: , CC: Nadav Amit , Alok Kataria , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Subject: [PATCH v5 6/9] x86: prevent inline distortion by paravirt ops Date: Tue, 19 Jun 2018 12:48:51 -0700 Message-ID: <20180619194854.69486-7-namit@vmware.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180619194854.69486-1-namit@vmware.com> References: <20180619194854.69486-1-namit@vmware.com> MIME-Version: 1.0 Content-Type: text/plain Received-SPF: None (EX13-EDG-OU-001.vmware.com: namit@vmware.com does not designate permitted sender hosts) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org GCC considers the number of statements in inlined assembly blocks, according to new-lines and semicolons, as an indication to the cost of the block in time and space. This data is distorted by the kernel code, which puts information in alternative sections. As a result, the compiler may perform incorrect inlining and branch optimizations. The solution is to set an assembly macro and call it from the inlined assembly block. As a result GCC considers the inline assembly block as a single instruction. The effect of the patch is a more aggressive inlining, which also causes a size increase of kernel. text data bss dec hex filename 18147336 10226688 2957312 31331336 1de1408 ./vmlinux before 18162555 10226288 2957312 31346155 1de4deb ./vmlinux after (+14819) Static text symbols: Before: 40053 After: 39942 (-111) Cc: Alok Kataria Cc: Thomas Gleixner Cc: Ingo Molnar Cc: "H. Peter Anvin" Cc: x86@kernel.org Cc: virtualization@lists.linux-foundation.org Reviewed-by: Juergen Gross Signed-off-by: Nadav Amit --- arch/x86/include/asm/paravirt_types.h | 56 +++++++++++++-------------- arch/x86/kernel/macros.S | 1 + 2 files changed, 28 insertions(+), 29 deletions(-) diff --git a/arch/x86/include/asm/paravirt_types.h b/arch/x86/include/asm/paravirt_types.h index 180bc0bff0fb..a3d886238b58 100644 --- a/arch/x86/include/asm/paravirt_types.h +++ b/arch/x86/include/asm/paravirt_types.h @@ -343,23 +343,11 @@ extern struct pv_lock_ops pv_lock_ops; #define paravirt_clobber(clobber) \ [paravirt_clobber] "i" (clobber) -/* - * Generate some code, and mark it as patchable by the - * apply_paravirt() alternate instruction patcher. - */ -#define _paravirt_alt(insn_string, type, clobber) \ - "771:\n\t" insn_string "\n" "772:\n" \ - ".pushsection .parainstructions,\"a\"\n" \ - _ASM_ALIGN "\n" \ - _ASM_PTR " 771b\n" \ - " .byte " type "\n" \ - " .byte 772b-771b\n" \ - " .short " clobber "\n" \ - ".popsection\n" - /* Generate patchable code, with the default asm parameters. */ -#define paravirt_alt(insn_string) \ - _paravirt_alt(insn_string, "%c[paravirt_typenum]", "%c[paravirt_clobber]") +#define paravirt_alt \ + "PARAVIRT_CALL type=\"%c[paravirt_typenum]\"" \ + " clobber=\"%c[paravirt_clobber]\"" \ + " pv_opptr=\"%c[paravirt_opptr]\"" /* Simple instruction patching code. */ #define NATIVE_LABEL(a,x,b) "\n\t.globl " a #x "_" #b "\n" a #x "_" #b ":\n\t" @@ -387,16 +375,6 @@ unsigned native_patch(u8 type, u16 clobbers, void *ibuf, int paravirt_disable_iospace(void); -/* - * This generates an indirect call based on the operation type number. - * The type number, computed in PARAVIRT_PATCH, is derived from the - * offset into the paravirt_patch_template structure, and can therefore be - * freely converted back into a structure offset. - */ -#define PARAVIRT_CALL \ - ANNOTATE_RETPOLINE_SAFE \ - "call *%c[paravirt_opptr];" - /* * These macros are intended to wrap calls through one of the paravirt * ops structs, so that they can be later identified and patched at @@ -534,7 +512,7 @@ int paravirt_disable_iospace(void); /* since this condition will never hold */ \ if (sizeof(rettype) > sizeof(unsigned long)) { \ asm volatile(pre \ - paravirt_alt(PARAVIRT_CALL) \ + paravirt_alt \ post \ : call_clbr, ASM_CALL_CONSTRAINT \ : paravirt_type(op), \ @@ -544,7 +522,7 @@ int paravirt_disable_iospace(void); __ret = (rettype)((((u64)__edx) << 32) | __eax); \ } else { \ asm volatile(pre \ - paravirt_alt(PARAVIRT_CALL) \ + paravirt_alt \ post \ : call_clbr, ASM_CALL_CONSTRAINT \ : paravirt_type(op), \ @@ -571,7 +549,7 @@ int paravirt_disable_iospace(void); PVOP_VCALL_ARGS; \ PVOP_TEST_NULL(op); \ asm volatile(pre \ - paravirt_alt(PARAVIRT_CALL) \ + paravirt_alt \ post \ : call_clbr, ASM_CALL_CONSTRAINT \ : paravirt_type(op), \ @@ -691,6 +669,26 @@ struct paravirt_patch_site { extern struct paravirt_patch_site __parainstructions[], __parainstructions_end[]; +#else /* __ASSEMBLY__ */ + +/* + * This generates an indirect call based on the operation type number. + * The type number, computed in PARAVIRT_PATCH, is derived from the + * offset into the paravirt_patch_template structure, and can therefore be + * freely converted back into a structure offset. + */ +.macro PARAVIRT_CALL type:req clobber:req pv_opptr:req +771: ANNOTATE_RETPOLINE_SAFE + call *\pv_opptr +772: .pushsection .parainstructions,"a" + _ASM_ALIGN + _ASM_PTR 771b + .byte \type + .byte 772b-771b + .short \clobber + .popsection +.endm + #endif /* __ASSEMBLY__ */ #endif /* _ASM_X86_PARAVIRT_TYPES_H */ diff --git a/arch/x86/kernel/macros.S b/arch/x86/kernel/macros.S index 66ccb8e823b1..71d8b716b111 100644 --- a/arch/x86/kernel/macros.S +++ b/arch/x86/kernel/macros.S @@ -10,3 +10,4 @@ #include #include #include +#include -- 2.17.0