Received: by 2002:a05:6a10:16a7:0:0:0:0 with SMTP id gp39csp1720212pxb; Fri, 20 Nov 2020 17:50:43 -0800 (PST) X-Google-Smtp-Source: ABdhPJy271tNF2lM5XCYTiAuiylP986MKJew1lv9jsgJudWhtQ+4u9xFLDsiR1Q9Mlh4umm0n/BR X-Received: by 2002:a17:906:d8c9:: with SMTP id re9mr10212900ejb.266.1605923443245; Fri, 20 Nov 2020 17:50:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1605923443; cv=none; d=google.com; s=arc-20160816; b=V6jlMz80OPLA3jVnoviXWGQNunBGfpVroM8ZaumGy2yigJMZxByaYNDWsX6RNvr8no yrQVnqWMDZ/0ennknskF24jhxkUHj6KcWrdDUE8/DSVUxPm/EjgmkeDUo1i7BzbI6rma e7jltF8hstysumyIzer+K3BBjrpH+8R1NxJWqewJxbrvuIaZ06K+4AJFek1JaajxbJec 8Q4lNt1rPNKf7lh1ggLIsCMZAT2rUKyrP0Mo8Y8zJdVUuytcFvIm3oCIPtNcKQ4o741K aEqFtVRUMypJbZj8jZeh2lCoeuEyVga7abbEO52mBaMzZWGe9nCfT7PRLFN5W1aEZqVP SpoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=v8J7ye6KqEOqzvSxgHSVDOvhErTqxQnjIzQcu9TjNgY=; b=Kfb8IAqX7qL8IRB9xbqHftsESBcSfViKmml2FO5QyDjcyrBDmq4PeAjD0ax+F3QnOx dFLVWujLMzuA9kEbd9hvESLBrdG/vHxxM96h26qIl7MHt3jI7cs0wy25CHShlQ2ZX5Uv 2VH7HxE4Dr87a3hD3qGerFV6deZryGwbNjmlmYma8oPlfw3YfR289r2Y0sS7YwKJ/nm0 4CnWmXlf/byLsc+JlhZxcf0Hq3Pz997Tdmh7h53K4kiNlifu09jSIbdmHtPvJE7oxNP6 gzHm0a8cy4wafUEkDgy6my9JRSiH/ii5c0Y0YV7eyoE9B7VI2iOHkbCOidZ1poI+1M/0 SEhA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=gxEj26Ay; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id b6si2664687ejk.518.2020.11.20.17.50.20; Fri, 20 Nov 2020 17:50:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=gxEj26Ay; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726117AbgKUBq6 (ORCPT + 99 others); Fri, 20 Nov 2020 20:46:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52716 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725944AbgKUBq6 (ORCPT ); Fri, 20 Nov 2020 20:46:58 -0500 Received: from mail-vs1-xe41.google.com (mail-vs1-xe41.google.com [IPv6:2607:f8b0:4864:20::e41]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BAF5C061A48 for ; Fri, 20 Nov 2020 17:46:57 -0800 (PST) Received: by mail-vs1-xe41.google.com with SMTP id v6so6076244vsd.1 for ; Fri, 20 Nov 2020 17:46:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=v8J7ye6KqEOqzvSxgHSVDOvhErTqxQnjIzQcu9TjNgY=; b=gxEj26AyhbRPRwqI4nRtVKE+oYSAVh0zKiN7CirbrkbRNHePoBQ5EYK/6EpmHgONJ6 rDzEI9xmd2qIIpAUwbpr8Q0yverWthQEp5Gqcylo5sKUIhwtfUGoz4tRuCQbcZaqkKAW oOidkmqwHF6T/d09/hXg1sRnsEGdmWMfeAH42QOzSzvfH/8ZXoiPcEUHHvnMMpdidxdk SBLmbP4NBAr0IyhZ6Aqnke827HaJix7UCMkkb3BI4YYVtQZ/NJlRBUzgqsBFXQMQhe2Z Gl9SL1sbqejJ1I1r/vXI0aht45bXxB+2OmXIeoLxJEhKfDetrsMJexh+NAMprZAZTUbp CUEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=v8J7ye6KqEOqzvSxgHSVDOvhErTqxQnjIzQcu9TjNgY=; b=RBHoKhiwd2FNHt1qcToBdIid/E+7mnFMQeWlMyr5f6g4+cj2l0MsQdLJmrFKcRxEy3 asaybfp1RIH1h740jrhLQPbx6MpPaJH4EvqtuNHlKfMnQahYo1drrmyg9iV+5s6tkEgs T/fsSiOpk5Nmusq6ZbeOg8F3lFtq6wZ06luOTKQtukNIXFDoZhieNi599L+67y5lG408 lwiN0cADW9BjGQhPVXXWfJcHu11Q4r1ubMm7Du5592FcC8BtUUk4g8t4bvSR6l4ASSG6 DUUGcYgfRQz5zlOEGRFfiAs0dFYbf2JpyCqENYKLGqOhk8bQGDq9ifndh0ksYem71g5/ IWNw== X-Gm-Message-State: AOAM530Qr714CYPddUGIA8wo2fon71YTFXjr8CdnEy4BYnhY5gW4UvO1 l4SaNDQgbDBkttuOVRy4m1OKScB0AJmww6rFrGc7Xw== X-Received: by 2002:a05:6102:22da:: with SMTP id a26mr15670580vsh.13.1605923216073; Fri, 20 Nov 2020 17:46:56 -0800 (PST) MIME-Version: 1.0 References: <20201118220731.925424-1-samitolvanen@google.com> <20201118220731.925424-3-samitolvanen@google.com> <202011201144.3F2BB70C@keescook> <20201120202935.GA1220359@ubuntu-m3-large-x86> <202011201241.B159562D7@keescook> <202011201556.3B910EF@keescook> In-Reply-To: <202011201556.3B910EF@keescook> From: Sami Tolvanen Date: Fri, 20 Nov 2020 17:46:44 -0800 Message-ID: Subject: Re: [PATCH v7 02/17] kbuild: add support for Clang LTO To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Steven Rostedt , Will Deacon , Josh Poimboeuf , Peter Zijlstra , Greg Kroah-Hartman , "Paul E. McKenney" , clang-built-linux , Kernel Hardening , linux-arch , Linux ARM , Linux Kbuild mailing list , LKML , linux-pci@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Nov 20, 2020 at 3:59 PM Kees Cook wrote: > > On Fri, Nov 20, 2020 at 12:58:41PM -0800, Sami Tolvanen wrote: > > On Fri, Nov 20, 2020 at 12:43 PM Kees Cook wrote: > > > > > > On Fri, Nov 20, 2020 at 01:29:35PM -0700, Nathan Chancellor wrote: > > > > On Fri, Nov 20, 2020 at 11:47:21AM -0800, Kees Cook wrote: > > > > > On Fri, Nov 20, 2020 at 08:23:11AM -0800, Sami Tolvanen wrote: > > > > > > Changing the ThinLTO config to a choice and moving it after the main > > > > > > LTO config sounds like a good idea to me. I'll see if I can change > > > > > > this in v8. Thanks! > > > > > > > > > > Originally, I thought this might be a bit ugly once GCC LTO is added, > > > > > but this could be just a choice like we're done for the stack > > > > > initialization. Something like an "LTO" choice of NONE, CLANG_FULL, > > > > > CLANG_THIN, and in the future GCC, etc. > > > > > > > > Having two separate choices might be a little bit cleaner though? One > > > > for the compiler (LTO_CLANG versus LTO_GCC) and one for the type > > > > (THINLTO versus FULLLTO). The type one could just have a "depends on > > > > CC_IS_CLANG" to ensure it only showed up when needed. > > > > > > Right, that's how the stack init choice works. Kconfigs that aren't > > > supported by the compiler won't be shown. I.e. after Sami's future > > > patch, the only choice for GCC will be CONFIG_LTO_NONE. But building > > > under Clang, it would offer CONFIG_LTO_NONE, CONFIG_LTO_CLANG_FULL, > > > CONFIG_LTO_CLANG_THIN, or something. > > > > > > (and I assume CONFIG_LTO would be def_bool y, depends on !LTO_NONE) > > > > I'm fine with adding ThinLTO as another option to the LTO choice, but > > it would duplicate the dependencies and a lot of the help text. I > > suppose we could add another config for the dependencies and have both > > LTO options depend on that instead. > > How about something like this? This separates the arch support, compiler > support, and user choice into three separate Kconfig areas, which I > think should work. Sure, this looks good to me, I'll use this in v8. The only minor concern I have is that ThinLTO cannot be set as the default LTO mode, but I assume anyone who selects LTO is also capable of deciding which mode is better for them. > diff --git a/Makefile b/Makefile > index e397c4caec1b..af902718e882 100644 > --- a/Makefile > +++ b/Makefile > @@ -897,7 +897,7 @@ export CC_FLAGS_SCS > endif > > ifdef CONFIG_LTO_CLANG > -ifdef CONFIG_THINLTO > +ifdef CONFIG_LTO_CLANG_THIN > CC_FLAGS_LTO += -flto=thin -fsplit-lto-unit > KBUILD_LDFLAGS += --thinlto-cache-dir=$(extmod-prefix).thinlto-cache > else > diff --git a/arch/Kconfig b/arch/Kconfig > index cdd29b5fdb56..5c22e10e4c12 100644 > --- a/arch/Kconfig > +++ b/arch/Kconfig > @@ -600,6 +600,14 @@ config SHADOW_CALL_STACK > > config LTO > bool > + help > + Selected if the kernel will be built using the compiler's LTO feature. > + > +config LTO_CLANG > + bool > + select LTO > + help > + Selected if the kernel will be built using Clang's LTO feature. > > config ARCH_SUPPORTS_LTO_CLANG > bool > @@ -609,28 +617,25 @@ config ARCH_SUPPORTS_LTO_CLANG > - compiling inline assembly with Clang's integrated assembler, > - and linking with LLD. > > -config ARCH_SUPPORTS_THINLTO > +config ARCH_SUPPORTS_LTO_CLANG_THIN > bool > help > - An architecture should select this option if it supports Clang's > - ThinLTO. > + An architecture should select this option if it can supports Clang's > + ThinLTO mode. > > -config THINLTO > - bool "Clang ThinLTO" > - depends on LTO_CLANG && ARCH_SUPPORTS_THINLTO > - default y > +config HAS_LTO_CLANG > + def_bool y > + # Clang >= 11: https://github.com/ClangBuiltLinux/linux/issues/510 > + depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD > + depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm) > + depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm) > + depends on ARCH_SUPPORTS_LTO_CLANG > + depends on !FTRACE_MCOUNT_USE_RECORDMCOUNT > + depends on !KASAN > + depends on !GCOV_KERNEL > help > - This option enables Clang's ThinLTO, which allows for parallel > - optimization and faster incremental compiles. More information > - can be found from Clang's documentation: > - > - https://clang.llvm.org/docs/ThinLTO.html > - > - If you say N here, the compiler will use full LTO, which may > - produce faster code, but building the kernel will be significantly > - slower as the linker won't efficiently utilize multiple threads. > - > - If unsure, say Y. > + The compiler and Kconfig options support building with Clang's > + LTO. > > choice > prompt "Link Time Optimization (LTO)" > @@ -644,20 +649,14 @@ choice > > config LTO_NONE > bool "None" > + help > + Build the kernel normally, without Link Time Optimization (LTO). > > -config LTO_CLANG > - bool "Clang's Link Time Optimization (EXPERIMENTAL)" > - # Clang >= 11: https://github.com/ClangBuiltLinux/linux/issues/510 > - depends on CC_IS_CLANG && CLANG_VERSION >= 110000 && LD_IS_LLD > - depends on $(success,$(NM) --help | head -n 1 | grep -qi llvm) > - depends on $(success,$(AR) --help | head -n 1 | grep -qi llvm) > - depends on ARCH_SUPPORTS_LTO_CLANG > - depends on !FTRACE_MCOUNT_USE_RECORDMCOUNT > - depends on !KASAN > - depends on !GCOV_KERNEL > - select LTO > +config LTO_CLANG_FULL > + bool "Clang Full LTO (EXPERIMENTAL)" > + select LTO_CLANG > help > - This option enables Clang's Link Time Optimization (LTO), which > + This option enables Clang's full Link Time Optimization (LTO), which > allows the compiler to optimize the kernel globally. If you enable > this option, the compiler generates LLVM bitcode instead of ELF > object files, and the actual compilation from bitcode happens at > @@ -667,9 +666,22 @@ config LTO_CLANG > > https://llvm.org/docs/LinkTimeOptimization.html > > - To select this option, you also need to use LLVM tools to handle > - the bitcode by passing LLVM=1 to make. > + During link time, this option can use a large amount of RAM, and > + may take much longer than the ThinLTO option. > > +config LTO_CLANG_THIN > + bool "Clang ThinLTO (EXPERIMENTAL)" > + depends on ARCH_SUPPORTS_LTO_CLANG_THIN > + select LTO_CLANG > + help > + This option enables Clang's ThinLTO, which allows for parallel > + optimization and faster incremental compiles compared to the > + CONFIG_LTO_CLANG_FULL option. More information can be found > + from Clang's documentation: > + > + https://clang.llvm.org/docs/ThinLTO.html > + > + If unsure, say Y. > endchoice The two LTO_CLANG_* options need to depend on HAS_LTO_CLANG, of course. Sami