Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp10455386imu; Sun, 30 Dec 2018 23:22:07 -0800 (PST) X-Google-Smtp-Source: ALg8bN4SS2M1UXAYno0eNW6Jw840gJzBRS87GJ4/HuJGmOd3TXsyf1Iu+cnqWiUta5oQs+Nv2Hgh X-Received: by 2002:a17:902:14b:: with SMTP id 69mr36796429plb.52.1546240927514; Sun, 30 Dec 2018 23:22:07 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1546240927; cv=none; d=google.com; s=arc-20160816; b=ZN9k9RRdgKSZd08FEIfS9YLhYCIhm3kiMWCokkYBa8RyL0o+ZLcCc3oBSCc0hcdP/L YolD6+s6pqFIzPqFSikOOXy5+5vH1oS5jCPq45OFyPRWVUgaPhhSVPunEqT9K9VPm+nX EdsX31chWg2f2uZwtPre3lAe7pWPKqUG0gF7ItPaXBx6hYJAZ6F2DMRoCGudKdMyRBCr Hy13jxzMUXMv+S+FpHCsKblDvzqT1G3Xzp3jbob3XYmCqPlQQ0kzOErHFLREg5rpRwLu YZ4SUuqkcU47LSz83RjYRfJPpMe5EE8omb8qdmtQn9CWsGE2pVYYiFhTKRtKOy8+Ln3U 5BzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:message-id:date:subject:cc :to:from; bh=ENMpg4mL/JDl3BZhCmnYNt+z3wkun0qCeOCmjiPCfxc=; b=jc5EA0azVUs4/BIBDZ+P6P6uJoK3G6xvEY7uMuLdagA3MiX4ZhtFQWeCwVbgcMyoUp ADpNZ9FjzuRXGjUDZFd3lvVAMlPS53XBZa5uFwzmY7xIYuIjXnuux7onIogTBuCW22jx 1f/3O4pYbfLj+WD9arCuvDZNpn1Ug1qPcJiOtzmTOLX+OzCrGBYnHjytV6LeSyktwWde G2xJw+ve+R+GWXV/F/13ri22d3/DRVXxCeHUndpX2msYgf02R6aro4/J+c0Lo1AjB8Z1 0ZUjCc0NhWcFDq46A8NWF/+muhP5CEbXgaa/lQhvNiu/MtAMNgNwYMScgW5Dxt6PGpIW Rd7A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=vmware.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 92si1971823pli.220.2018.12.30.23.21.52; Sun, 30 Dec 2018 23:22:07 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=vmware.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727041AbeLaHUh (ORCPT + 99 others); Mon, 31 Dec 2018 02:20:37 -0500 Received: from ex13-edg-ou-002.vmware.com ([208.91.0.190]:25217 "EHLO EX13-EDG-OU-002.vmware.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726381AbeLaHUh (ORCPT ); Mon, 31 Dec 2018 02:20:37 -0500 Received: from sc9-mailhost3.vmware.com (10.113.161.73) by EX13-EDG-OU-002.vmware.com (10.113.208.156) with Microsoft SMTP Server id 15.0.1156.6; Sun, 30 Dec 2018 23:20:33 -0800 Received: from sc2-haas01-esx0118.eng.vmware.com (sc2-haas01-esx0118.eng.vmware.com [10.172.44.118]) by sc9-mailhost3.vmware.com (Postfix) with ESMTP id 75C4940687; Sun, 30 Dec 2018 23:20:35 -0800 (PST) From: Nadav Amit To: Ingo Molnar , Andy Lutomirski , Peter Zijlstra , Josh Poimboeuf , Edward Cree CC: "H . Peter Anvin" , Thomas Gleixner , LKML , Nadav Amit , X86 ML , Paolo Abeni , Borislav Petkov , David Woodhouse , Nadav Amit Subject: [RFC v2 0/6] x86: dynamic indirect branch promotion Date: Sun, 30 Dec 2018 23:21:06 -0800 Message-ID: <20181231072112.21051-1-namit@vmware.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Content-Type: text/plain Received-SPF: None (EX13-EDG-OU-002.vmware.com: namit@vmware.com does not designate permitted sender hosts) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This is a revised version of optpolines (formerly named retpolines) for dynamic indirect branch promotion in order to reduce retpoline overheads [1]. This version address some of the concerns that were raised before. Accordingly, the code was slightly simplified and patching is now done using the regular int3/breakpoint mechanism. Outline optpolines for multiple targets was added. I do not think the way I implemented it is the correct one. In my original (private) version, if there are more targets than the outline block can hold, the outline block is completely removed. However, I think this is more-or-less how Josh wanted it to be. The code modifications are now done using a gcc-plugin. This allows to easily ignore code from init and other code sections. I think it should also allow us to add opt-in/opt-out support for each branch, for example by marking function pointers using address-space attributes. All of these changes required some optimizations to go away to keep the code simple. I have still did not run the benchmarks again. So I might have not addressed all the open issues, but it is rather hard to finish the implementation since some still open high-level decisions affect the way in which optimizations should be done. Specifically: - Is it going to be the only indirect branch promotion mechanism? If so, it probably should also provide interface similar to Josh's "static-calls" with annotations. - Should it also be used when retpolines are disabled (in the config)? This does complicate the implementation a bit (RFC v1 supported it). - Is it going to be opt-in or opt-out? If it is an opt-out mechanism, memory and performance optimizations need to be more aggressive. - Do we use periodic learning or not? Josh suggested to reconfigure the branches whenever a new target is found. However, I do not know at this time how to do learning efficiently, without making learning much more expensive. [1] https://lore.kernel.org/patchwork/cover/1001332/ Nadav Amit (6): x86: introduce kernel restartable sequence objtool: ignore instructions x86: patch indirect branch promotion x86: interface for accessing indirect branch locations x86: learning and patching indirect branch targets x86: outline optpoline arch/x86/Kconfig | 4 + arch/x86/entry/entry_64.S | 16 +- arch/x86/include/asm/nospec-branch.h | 83 ++ arch/x86/include/asm/sections.h | 2 + arch/x86/kernel/Makefile | 1 + arch/x86/kernel/asm-offsets.c | 9 + arch/x86/kernel/nospec-branch.c | 1293 ++++++++++++++++++ arch/x86/kernel/traps.c | 7 + arch/x86/kernel/vmlinux.lds.S | 7 + arch/x86/lib/retpoline.S | 83 ++ include/linux/cpuhotplug.h | 1 + include/linux/module.h | 9 + kernel/module.c | 8 + scripts/Makefile.gcc-plugins | 3 + scripts/gcc-plugins/x86_call_markup_plugin.c | 329 +++++ tools/objtool/check.c | 21 +- 16 files changed, 1872 insertions(+), 4 deletions(-) create mode 100644 arch/x86/kernel/nospec-branch.c create mode 100644 scripts/gcc-plugins/x86_call_markup_plugin.c -- 2.17.1