Received: by 2002:a05:6902:102b:0:0:0:0 with SMTP id x11csp19527ybt; Tue, 30 Jun 2020 13:56:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxwFymhIy+QI1pfF5CY2Yng/BUo9zNADJcZLct/a2CXdVGD8t6XWSwQ9f4CO60RyvdS97tm X-Received: by 2002:a05:6402:1778:: with SMTP id da24mr12689082edb.126.1593550166003; Tue, 30 Jun 2020 13:49:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1593550165; cv=none; d=google.com; s=arc-20160816; b=izWHG84sMqza/n6f22+4hgJFyzm3/Tk1CPoQfJce00nogOLcBRgEQ0G1wS1tfuC+om /P8DGtrwDQRcv54Nnx3tFeJZM+KKhLc+u+1r4nxq26hmIgNqI3zOxbsy0dUCBO6AiHLb C26r7pQkelKYshy0fyV+kBqW+v00SsmG/GkE9FIOjsvlGIcEO1+bda064xGzdU0U/UOp hcup8uRLmb1136N+ye8E+lNiVtpgBkZ6uZEUPCeCOX4EVhFi2re1mAc4weZgfTSDLOCn Ij/YOHKWdZZoF92RQNxGqAtJXG0PE2UFuwdu8xhGewpZI+P97JR8LjWM+2fQZzFX6nP1 AHmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:dkim-signature; bh=+SWuClaaDOYjnOgTPZaQNynsf/0FUgaUl6LbycKFTXY=; b=mVdmcE3+/E9ZENUKV/59+GcGRnqN+h5IOZYUbLULB9rxA1bDBTzWcEA5gKxtxxnLnN zPvMrVJhA+sja2SUhpu1ku8wUzjU+hQEbZ60tudf6590bQkEwQgCunKcimLysGcGWYps UhdPAy41XXRMHzfI1SMffstXtfRV+YDjMRKVcX98oU2PvaHsGt36Ens8XGDGO6xvf3zS HBsjUwofQjdSWGu/3TsDyuj+X0yocrlLswkl4HfXXWuL1JqooQ9s2pICrTsDucUQrHjP 2krRAWY0JH8s/8L8aOZgeHI9MuS3cIwcn4+E7KpGLmpZtnYDQikc4WLNlwSOvuCJ8pKp b7bg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=p1ZTo7sq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id w22si2497056eds.415.2020.06.30.13.49.03; Tue, 30 Jun 2020 13:49:25 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=p1ZTo7sq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727068AbgF3TTl (ORCPT + 99 others); Tue, 30 Jun 2020 15:19:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41908 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727059AbgF3TTk (ORCPT ); Tue, 30 Jun 2020 15:19:40 -0400 Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DFE40C03E97B for ; Tue, 30 Jun 2020 12:19:39 -0700 (PDT) Received: by mail-wm1-x342.google.com with SMTP id q15so19867942wmj.2 for ; Tue, 30 Jun 2020 12:19:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=+SWuClaaDOYjnOgTPZaQNynsf/0FUgaUl6LbycKFTXY=; b=p1ZTo7sqWt0hVOCPEQgh4KonD/ze3MFi/Yo01MsLgVJf5FMsIFlA6JW4xVwwRJLlvb 2LGkFLB4TzM0dvjc8XtPJfVJhY8j+voxfHGY5wyBtTX1C+ZACZgqKm0vEVfpHx7NWev/ lcBMgEAqWixQZY8F67ZRGbDhWdhv/yhtFew6T+y/MorkhkJgTBklhLRQcuC4gHDG6Oma 5i9nqG9qYH/2KTAQvzLCjszZz0sf7Mhc082rZwllvHEn17zE65LAchhrPt5sAy6Dzzit PtTeeRPqNlzXbRW/+6N4LbmHjg3QifubG2x60lzfsOi+sgYvSO6qM++fCjJ07Ct2wEA1 NugQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=+SWuClaaDOYjnOgTPZaQNynsf/0FUgaUl6LbycKFTXY=; b=LPFOufxrln4AAXI3VN2p3OxyPwu7Qna2MJ4GUiqadyv7qM6H6khOJOuPAJCRR3VKgI WhKZDAOCRFbEK7oyXTYnggNv3Klk6eGn3xOlI68C1pTRRsme56a7X4rJd0bw+1+tBYeW q6MQnvNZdjp6fGfi40wW57DE8zFLTh1D9FcJtdPTGoSczCuP0oqfqQ4SlpAN0SxoKEfq OKR7045sCJ4NWLTlWP+Is+5+hcvafQuf0oSd0bgN3zqQhUBfWLwmP0E5ME8DHgpzN/5H BGTvYDSSajYjuT3NlQeKdpgFSdiuhM9xiTsmp99X4HeFwt+AjjOqj3jXCN6mXK2ffDOJ gVjg== X-Gm-Message-State: AOAM53024Bx8cWBq/ylqGjBuPKIy5LkMu06B0RpIvwVBMr4q8UXzyk9/ 4tFQTCmhAkb1s0r+abQL0HA1cQ== X-Received: by 2002:a1c:3286:: with SMTP id y128mr21486460wmy.29.1593544778240; Tue, 30 Jun 2020 12:19:38 -0700 (PDT) Received: from elver.google.com ([100.105.32.75]) by smtp.gmail.com with ESMTPSA id h14sm4799958wrt.36.2020.06.30.12.19.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 30 Jun 2020 12:19:37 -0700 (PDT) Date: Tue, 30 Jun 2020 21:19:31 +0200 From: Marco Elver To: Peter Zijlstra Cc: Nick Desaulniers , Sami Tolvanen , Masahiro Yamada , Will Deacon , Greg Kroah-Hartman , "Paul E. McKenney" , Kees Cook , clang-built-linux , Kernel Hardening , linux-arch , Linux ARM , Linux Kbuild mailing list , LKML , linux-pci@vger.kernel.org, "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" Subject: Re: [PATCH 00/22] add support for Clang LTO Message-ID: <20200630191931.GA884155@elver.google.com> References: <20200624203200.78870-1-samitolvanen@google.com> <20200624211540.GS4817@hirez.programming.kicks-ass.net> <20200625080313.GY4817@hirez.programming.kicks-ass.net> <20200625082433.GC117543@hirez.programming.kicks-ass.net> <20200625085745.GD117543@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200625085745.GD117543@hirez.programming.kicks-ass.net> User-Agent: Mutt/1.13.2 (2019-12-18) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org I was asked for input on this, and after a few days digging through some history, thought I'd comment. Hope you don't mind. On Thu, Jun 25, 2020 at 10:57AM +0200, Peter Zijlstra wrote: > On Thu, Jun 25, 2020 at 10:24:33AM +0200, Peter Zijlstra wrote: > > On Thu, Jun 25, 2020 at 10:03:13AM +0200, Peter Zijlstra wrote: > > > I'm sure Will will respond, but the basic issue is the trainwreck C11 > > > made of dependent loads. > > > > > > Anyway, here's a link to the last time this came up: > > > > > > https://lore.kernel.org/linux-arm-kernel/20171116174830.GX3624@linux.vnet.ibm.com/ > > > > Another good read: > > > > https://lore.kernel.org/lkml/20150520005510.GA23559@linux.vnet.ibm.com/ [...] > Because now the machine can speculate and load now before seq, breaking > the ordering. First of all, I agree with the concerns, but not because of LTO. To set the stage better, and summarize the fundamental problem again: we're in the unfortunate situation that no compiler today has a way to _efficiently_ deal with C11's memory_order_consume [https://lwn.net/Articles/588300/]. If we did, we could just use that and be done with it. But, sadly, that doesn't seem possible right now -- compilers just say consume==acquire. Will suggests doing the same in the kernel: https://lkml.kernel.org/r/20200630173734.14057-19-will@kernel.org What we're most worried about right now is the existence of compiler transformations that could break data dependencies by e.g. turning them into control dependencies. If this is a real worry, I don't think LTO is the magical feature that will uncover those optimizations. If these compiler transformations are real, they also exist in a normal build! And if we are worried about them, we need to stop relying on dependent load ordering across the board; or switch to -O0 for everything. Clearly, we don't want either. Why do we think LTO is special? With LTO, Clang just emits LLVM bitcode instead of ELF objects, and during the linker stage intermodular optimizations across translation unit boundaries are done that might not be possible otherwise [https://llvm.org/docs/LinkTimeOptimization.html]. From the memory model side of things, if we could fully convey our intent to the compiler (the imaginary consume), there would be no problem, because all optimization stages from bitcode generation to the final machine code generation after LTO know about the intended semantics. (Also, keep in mind that LTO is _not_ doing post link optimization of machine code binaries!) But as far as we can tell, there is no evidence of the dreaded "data dependency to control dependency" conversion with LTO that isn't there in non-LTO builds, if it's even there at all. Has the data to control dependency conversion been encountered in the wild? If not, is the resulting reaction an overreaction? If so, we need to be careful blaming LTO for something that it isn't even guilty of. So, we are probably better off untangling LTO from the story: 1. LTO or no LTO does not matter. The LTO series should not get tangled up with memory model issues. 2. The memory model question and problems need to be answered and addressed separately. Thoughts? Thanks, -- Marco