Received: by 2002:a05:6a10:22f:0:0:0:0 with SMTP id 15csp3397263pxk; Mon, 28 Sep 2020 17:03:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyKP21eoP+zu+3QOVtbjNsD7ZsXQjbiHc3EnVmevJXBS6hDiL7DQG/lBSFVGYNfUj8aRlQL X-Received: by 2002:a17:906:b4e:: with SMTP id v14mr1181380ejg.179.1601337792703; Mon, 28 Sep 2020 17:03:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1601337792; cv=none; d=google.com; s=arc-20160816; b=fliXcJOrXwblwhBIJpuVUrVa/CzeEEV2kkT9lzmC7SdoDrVYFvliR24GKlDEq9ZLE7 ZCmax4VH4fLNoNDGHFeOKy2Skc7wGew5gZ/52mTJKAFzch9tMdQgyxdTqFVALqaHoYYZ MrpJ+xmOkpeVeJFZFCh3UhvSFAAWNKyWZ5VkvjFh5V549CWyK8yJkAaho/Mb12S/kvpm HJWXXETvaUpytYkI++t/zmQU45JDFayRVXIYHauDc6YkOZNh2Y0qLrFytcqTmRYqudfh Vsx2pVXq2Fh/jLxEvUDfTVc8Qtsmja/kKfDNvljgHro7Uoi8ED0XTXxHqrNej1A//NRb pc5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:organization:in-reply-to:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :ironport-sdr:ironport-sdr; bh=MCNd9CbTaRNtAVjNfflWJBcDpJ4BKgmxTbeK7KP6FIE=; b=ISAW7LcVotUDygsNZwYNgYqJ/y8xQR0G8inV2IhZl4SM2cESrYehPCdTervhPQZ4GH Jo3s6qOsdr1VmdJ6IDEJBYRvYDIMqWOel7zl+S9Br69AKOdecV5xG2mtrhpbXbRL7xhl /drB1gljuPvsmM9j6Y5MA53KaaS6oHG2E5JrmYt1CCXgBDRp5wuU309gmKKhaXG4kluA cYzJ/xfUa6kiWOQ1ZFXMIM/0+z+oMsnR0I5LKjwADu/1UzmpoILrtHAAGzQmscbFLkfs 2cishrJrPk8gC4pjvWDsSpYsuQLg7yBCSUvtci2ItrJrK5xvlzJHVc3UCgDPZ6rl7rIV cphw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id q6si1631086eji.17.2020.09.28.17.02.44; Mon, 28 Sep 2020 17:03:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727058AbgI1X76 (ORCPT + 99 others); Mon, 28 Sep 2020 19:59:58 -0400 Received: from mga06.intel.com ([134.134.136.31]:12718 "EHLO mga06.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726952AbgI1X76 (ORCPT ); Mon, 28 Sep 2020 19:59:58 -0400 IronPort-SDR: nyRcAbQY9U+v13UtthCqBK0eruFjXoMIUqtkCqxNe1aWwKJ5v39E7pN3QubbyQgvVEiX+eRLox PyQy8gGvTAiQ== X-IronPort-AV: E=McAfee;i="6000,8403,9758"; a="223660966" X-IronPort-AV: E=Sophos;i="5.77,315,1596524400"; d="scan'208";a="223660966" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga004.jf.intel.com ([10.7.209.38]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Sep 2020 14:56:44 -0700 IronPort-SDR: 3TWj+h2LPhdtBpEZ+uEDq8JR3Pkv5DjbrTY5pg8oVvTeweUZJ6ev/zSPMtjbBShluKnNJVlCkM ItrbFVtNgLDQ== X-IronPort-AV: E=Sophos;i="5.77,315,1596524400"; d="scan'208";a="457008323" Received: from jlasecki-mobl2.ger.corp.intel.com (HELO localhost) ([10.252.49.78]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Sep 2020 14:56:37 -0700 Date: Tue, 29 Sep 2020 00:56:35 +0300 From: Jarkko Sakkinen To: Andy Lutomirski Cc: "H.J. Lu" , Andrew Cooper , the arch/x86 maintainers , linux-sgx@vger.kernel.org, LKML , Sean Christopherson , Jethro Beekman , Cedric Xing , Andrew Morton , Andy Shevchenko , asapek@google.com, Borislav Petkov , chenalexchen@google.com, Conrad Parker , cyhanish@google.com, Dave Hansen , "Huang, Haitao" , Josh Triplett , "Huang, Kai" , "Svahn, Kai" , Keith Moyer , Christian Ludloff , Neil Horman , Nathaniel McCallum , Patrick Uiterwijk , David Rientjes , Thomas Gleixner , yaozhangx@google.com, Yu-cheng Yu Subject: Re: [PATCH v38 21/24] x86/vdso: Implement a vDSO for Intel SGX enclave call Message-ID: <20200928215635.GF2705@linux.intel.com> References: <20200915112842.897265-1-jarkko.sakkinen@linux.intel.com> <20200915112842.897265-22-jarkko.sakkinen@linux.intel.com> <721ca14e-21df-3df1-7bef-0b00d0ff90c3@citrix.com> <20200928005842.GC6704@linux.intel.com> <85bc15d5-93cd-e332-ae9a-1e1e66e1181d@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Organization: Intel Finland Oy - BIC 0357606-4 - Westendinkatu 7, 02160 Espoo Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Sep 28, 2020 at 11:12:08AM -0700, Andy Lutomirski wrote: > On Mon, Sep 28, 2020 at 11:08 AM H.J. Lu wrote: > > > > On Mon, Sep 28, 2020 at 9:44 AM Andrew Cooper wrote: > > > > > > On 28/09/2020 01:58, Jarkko Sakkinen wrote: > > > > On Fri, Sep 25, 2020 at 07:23:59PM +0100, Andrew Cooper wrote: > > > >> On 15/09/2020 12:28, Jarkko Sakkinen wrote: > > > >>> diff --git a/arch/x86/entry/vdso/vsgx_enter_enclave.S b/arch/x86/entry/vdso/vsgx_enter_enclave.S > > > >>> new file mode 100644 > > > >>> index 000000000000..adbd59d41517 > > > >>> --- /dev/null > > > >>> +++ b/arch/x86/entry/vdso/vsgx_enter_enclave.S > > > >>> @@ -0,0 +1,157 @@ > > > >>> +SYM_FUNC_START(__vdso_sgx_enter_enclave) > > > >>> > > > >>> +.Lretpoline: > > > >>> + call 2f > > > >>> +1: pause > > > >>> + lfence > > > >>> + jmp 1b > > > >>> +2: mov %rax, (%rsp) > > > >>> + ret > > > >> I hate to throw further spanners in the work, but this is not compatible > > > >> with CET, and the user shadow stack work in progress. > > > > CET goes beyond my expertise. Can you describe, at least rudimentary, > > > > how this code is not compatible? > > > > > > CET Shadow Stacks detect attacks which modify the return address on the > > > stack. > > > > > > Retpoline *is* a ROP gadget. It really does modify the return address > > > on the stack, even if its purpose is defensive (vs Spectre v2) rather > > > than malicious. > > > > > > >> Whichever of these two large series lands first is going to inflict > > > >> fixing this problem on the other. > > > >> > > > >> As the vdso text is global (to a first approximation), it must not be a > > > >> retpoline if any other process is liable to want to use CET-SS. > > > > Why is that? > > > > > > Because when CET-SS is enabled, the ret will suffer a #CP exception > > > (return address on the stack not matching the one recorded in the shadow > > > stack), which I presume/hope is wired into SIGSEGV. > > > > > > > Here is the CET compatible retpoline: > > > > endbr64 > > /* Check if shadow stack is in use. NB: R11 is the only usable > > scratch register for function calls. */ > > xorl %r11d, %r11d > > rdsspq %r11 > > testq %r11, %r11 > > jnz 3f > > call 2f > > 1: > > pause > > lfence > > jmp 1b > > 2: > > mov %rax, (%rsp) > > ret > > 3: > > /* Shadow stack is in use. Make the indirect call. */ > > call *%rax > > ret > > What do we expect user programs to do on CET systems? It would be > nice if we could instead ALTERNATIVE this out if X86_FEATURE_SHSTK. > > --Andy I'm open to do either solution. My thinking was to initially do things vsgx.S local (i.e. consider ALTERNATIVE post upstreaming) and use the above solution but I'm also fine doing ALTERNATIVE. Dave kindly briefed on details how that thing works and it should be perfectly usable for our use case. /Jarkko