Received: by 2002:a25:ef43:0:0:0:0:0 with SMTP id w3csp161534ybm; Thu, 28 May 2020 19:12:31 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzFRVeaHpMXKe9U6dcYvfMVsJ+U7DQdeDEpF3M7SYEwBk0++7XSiMKCFpujE0X5m6hGYeX2 X-Received: by 2002:a17:906:3944:: with SMTP id g4mr5549853eje.55.1590718351613; Thu, 28 May 2020 19:12:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1590718351; cv=none; d=google.com; s=arc-20160816; b=TF2fqzQxhtfxHM2eQUT7mxRhT21ilHVO9++DzPIt5cnaddDpXaqrCQwXDt7jL2a43w sYQgBnkSz29YI+9eBiE/zC6SBDBbuu+sXviNiTSYYkvrTd21RPuznmECX+F3lCbW736N mjdvT4huJtfLmV2jKaCSv86yKRSR7QApQ1BtMUZlMvmJDAoVaaRgVDmwqFpNvIT05E1/ I/Hoy+LS3AfSV6sYI6qZLJMBAlJDQ9c19IrurFVxX+JzvUFePSgdEDfCUnByxIoo7FHB qe0U42iPYhclwNjoQ3A67wkn3cg5FT3MqvaFqbqWSsLeuFI9Wv+ZH89vLl09CxaN9Epu zUtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:date:cc:to:from:subject :message-id:ironport-sdr:ironport-sdr; bh=I6imoXh+vGgZG3k/0lFD28L4QeQBIQw3K38f87OoiE0=; b=y0huoraRXvvg1tsR6qA1utzFu9sxolpxLd896ikC7E3o/73wA9M1XS1tvTAaNNpC3d uNM9TVcUYd6kl5l3fO9uhxROUXyYukMagf6LUffQE5awP8U3zL0fpoo20MXeISMPCQjm xyYePDQg8kSSmPLYrWLi4ADpp/Za2cG+yhOC/IVYdFVhqUNfz2MfK2d7Rw8eAzcTsxwk BwR0Y0A4oa80QJowumhYaYFug+RJIGQBrbtBPZnKUgvOraZAAqm+GhZt2gAS2+Fiuoc9 9DE8ZirbKwqGEm+fKWFBjar8LD1Q4PjCd9MVVz9iD3UKjIdTp1Dkq8GSY8WdxPgpn0kc puOg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id k6si4759901ejx.345.2020.05.28.19.12.08; Thu, 28 May 2020 19:12:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404988AbgE2CJe (ORCPT + 99 others); Thu, 28 May 2020 22:09:34 -0400 Received: from mga09.intel.com ([134.134.136.24]:54479 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404458AbgE2CJc (ORCPT ); Thu, 28 May 2020 22:09:32 -0400 IronPort-SDR: RaIbDDYZx31m5ATKecIMVTVX+yh4/fWUG9bkN+hfOoUlNdZsslRH23Gqjs2jAS9s8BYnGkdQ2x 6bVW3r59tBOg== X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 May 2020 19:09:30 -0700 IronPort-SDR: 7MKLns2Hk4/c5CuRAox9LyQSnnhS904keYMJ/aTD8swi0CnpaHmbsLYkUyt8aCoCA2pvOtIuoT DEEFum8srPDw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.73,447,1583222400"; d="scan'208";a="302678513" Received: from yyu32-desk.sc.intel.com ([143.183.136.146]) by fmsmga002.fm.intel.com with ESMTP; 28 May 2020 19:09:29 -0700 Message-ID: Subject: Re: [PATCH v10 01/26] Documentation/x86: Add CET description From: Yu-cheng Yu To: Andy Lutomirski , Dave Hansen Cc: X86 ML , "H. Peter Anvin" , Thomas Gleixner , Ingo Molnar , LKML , "open list:DOCUMENTATION" , Linux-MM , linux-arch , Linux API , Arnd Bergmann , Balbir Singh , Borislav Petkov , Cyrill Gorcunov , Dave Hansen , Eugene Syromiatnikov , Florian Weimer , "H.J. Lu" , Jann Horn , Jonathan Corbet , Kees Cook , Mike Kravetz , Nadav Amit , Oleg Nesterov , Pavel Machek , Peter Zijlstra , Randy Dunlap , "Ravi V. Shankar" , Vedvyas Shanbhogue , Dave Martin , Weijiang Yang Date: Thu, 28 May 2020 19:08:28 -0700 In-Reply-To: References: <2eb98637-bd2d-dda6-7729-f06ea84256ca@intel.com> <58319765-891D-44B9-AF18-64492B01FF36@amacapital.net> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.32.4 (3.32.4-1.fc30) MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, 2020-05-19 at 18:04 -0700, Andy Lutomirski wrote: > On Mon, May 18, 2020 at 6:35 PM Andy Lutomirski wrote: > > [...] > > > On May 18, 2020, at 5:38 PM, Dave Hansen wrote: > > > [...] > > > The sadistic parts of selftests/x86 come from real bugs. Either bugs > > > where the kernel fell over, or where behavior changed that broke apps. > > > I'd suggest doing some research on where that particular test case came > > > from. Find the author of the test, look at the changelogs. > > > > > > If this is something that a real app does, this is a problem. If it's a > > > sadistic test that Andy L added because it was an attack vector against > > > the entry code, it's a different story. > > > > There are quite a few tests that do these horrible things in there. IN my personal opinion, sigreturn.c is one of the most important tests we have — it does every horrible thing to the entry code that I thought of and that I could come up with a way of doing. We have been saved from regressing many times by these tests. CET, and especially the CPL0 version of CET, is its own set of entry horror, and we need to keep these tests working. > > > > I assume the basic issue is that we call raise(), the context magically changes to 32-bit, but SSP has a 64-bit value, and horrors happen. So I think two things need to happen: > > > > 1. Someone needs to document what happens when IRET tries to put a 64-bit value into SSP but CS is compat. Because Intel has plenty of history of doing colossally broken things here. IOW you could easily be hitting a hardware design problem, not a software issue per se. > > > > 2. The test needs to work. Assuming the hardware doesn’t do something utterly broken, either the 32-bit code needs to be adjusted to avoid any CALL > > or RET, or you need to write a little raise_on_32bit_shstk() func that switches to an SSP that fits in 32 bits, calls raise(), and switches back. From memory, I didn’t think there was a CALl or RET, so I’m guessing that SSP is getting truncated when we round trip through CPL3 compat mode and the result is that the kernel invoked the signal handler with the wrong SSP. Whoops. > > > > Following up here, I think this needs attention from the H/W architects. > > From the SDM: > > SYSRET and SYSEXIT: > > IF ShadowStackEnabled(CPL) > SSP ← IA32_PL3_SSP; > FI; > > IRET: > > IF ShadowStackEnabled(CPL) > IF CPL = 3 > THEN tempSSP ← IA32_PL3_SSP; FI; > IF ((EFER.LMA AND CS.L) = 0 AND tempSSP[63:32] != 0) > THEN #GP(0); FI; > SSP ← tempSSP > > The semantics of actually executing in compat mode with SSP >= 2^32 > are unclear. If nothing else, VM exit will save the full SSP and a > subsequent VM entry will fail. Here is what I got after talking to the architect. If the guest is in 32-bit mode, but its VM guest state SSP field is 64-bit, the CPU only uses the lower 32 bits. The SDM currently states a consistency check of the guest SSP field, but that will be removed in the next version. Upon VM entry, the CPU only requires the guest SSP to be pseudo-canonical like the RIP and RSP. > I don't know what the actual effect of operand-size-32 SYSRET or > SYSEXIT with too big a PL3_SSP will be, but I think it needs to be > documented. Ideally it will not put the CPU in an invalid state. > Ideally it will also not fault, because SYSRET faults in particular > are fatal unless the vector uses IST, and please please please don't > force more ISTs on anyone. On SYSRET/SYSEXIT to a 32-bit context, the CPU only uses the lower 32 bits of the user-mode SSP, and will not go into an invalid state and will not fault. The SDM will be explicit about this. Yu-cheng > > So I think we may need to put this entire series on hold until we get > some answers, because I suspect we're going to have a nice little root > hole otherwise.