Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp996213imu; Fri, 7 Dec 2018 12:19:19 -0800 (PST) X-Google-Smtp-Source: AFSGD/WbUvnaitxodM4gfOlqrjVMEF0IZdMP2hJ+cs1csmNt6bTW1mr+VfGbTeRgxEmq/E2rup4x X-Received: by 2002:a17:902:2dc3:: with SMTP id p61mr3366119plb.166.1544213959570; Fri, 07 Dec 2018 12:19:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544213959; cv=none; d=google.com; s=arc-20160816; b=p1fXqljlnluOQ2Ew3GH/OHWVan31pP+3v6DQuvchGHF1ZmMfE4vcocf3RCd6Vfmu5m wd/X+xfPJcLpUwR56WjzjhsYvLB8ysADSg6SXMiSzbbk1DQN6d4AD3qqRx8jSMSJ0/2C T8QXzJnMnijJ9/YAeguOU3/YOrpMZtAvdPSHQhOlLR5BW/YzYkF+XSA819P4dbPVexrQ owVluvROYkHOzDl/yozwJQjGkNzb/RHrNq2GAjOZGC80+9YJYvTtWz0PZ0kByKo0H6KG SJ5CK0IXT71yb7+KzLfu5tZnLRkXNFi0KjIg63b6X5F8tlSvg/DCpeE9O/5rFtM6da/4 bbPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:to:references:message-id :content-transfer-encoding:cc:date:in-reply-to:from:subject :mime-version:dkim-signature; bh=pjVnQ5FDILgERAcDn7BNzHjEa7TnukijT6untP9L3tI=; b=WS0VJMaydPQz5n8qHzphI0Su/XkZBy7chC7iPAZSkKlHZeVxGUCkGKQ7Nu1cVIToTa ARlMBWtK6A8BVUCKU/+kRuFRrN0vjEn8OINGFJDJ9S5Iok1myROGqqXjbmSHnUrNPZso LWgj0RJaP6RBETiqAvcg2lk5DGK2byn6m+rW5JTCehJXH4jxBadBL/g6HYzLZ7p60Mil EedjQzyqYrHJ2lfTOWQBarBEF3agPRVtUd6s5kDVIO/kRs/4n9160EgSG0BP55WYD7Lb Z7o/FvF0YW2S2DaF9ssA7krLK5bYv3ieS62bgv8MyLuTYdeBdh7Vi64/4YIb43Urei3h OnNQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@amacapital-net.20150623.gappssmtp.com header.s=20150623 header.b=nzr0wqol; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id u3si3749077plb.99.2018.12.07.12.19.04; Fri, 07 Dec 2018 12:19:19 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@amacapital-net.20150623.gappssmtp.com header.s=20150623 header.b=nzr0wqol; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726239AbeLGURE (ORCPT + 99 others); Fri, 7 Dec 2018 15:17:04 -0500 Received: from mail-pf1-f196.google.com ([209.85.210.196]:39968 "EHLO mail-pf1-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726104AbeLGURE (ORCPT ); Fri, 7 Dec 2018 15:17:04 -0500 Received: by mail-pf1-f196.google.com with SMTP id i12so2439204pfo.7 for ; Fri, 07 Dec 2018 12:17:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amacapital-net.20150623.gappssmtp.com; s=20150623; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=pjVnQ5FDILgERAcDn7BNzHjEa7TnukijT6untP9L3tI=; b=nzr0wqol6Zw06x0MaAQzWsQyXEoGKJZE5AX1ZccaGLH1EUgCH77WDm+XQ34TNFN3Tb JsZRrYIpG1sbWnyT8rb1NxecasLuhk7HZRefr5sUhY/o8BvPDpEtdvDcBcY8frCoNTWF V4+UJEVVVO2tjuIx0QwkKvJ6A4sSJeEOdR9mmJk89+i0hHBMXJV9yUmIKi79yYusv8kl DYHoq+VLQT7c89vGXewBBqw05XEbffAnG+jIGVKQl8mYiOBWeg59fu7U6S99zpEAw7Fz 7bAKZQqrtArEAecL+WiVJaPYmGH1mjOHqtC+JtiHGREEHG2EQ2FX/8AUMYr+dVIfUHes Cyog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=pjVnQ5FDILgERAcDn7BNzHjEa7TnukijT6untP9L3tI=; b=kqYXRuWsJHTBQQJ/wXrOJOsE44v3tc8TIZgWrp3suwGnWn+ok3v2kyUYdtpKQmQd+e 9PRVSpUAlkfZzqZztwSgUt3FzBhrMhz4HiQFUjcVFWPpI4+mpvD6JfifArWjMBj0hPAe C6nHMRv4He/5x+LTT2STsKdbOWQ+o+fn+uYOA7yfbflyPCbjs2eNJrGf3+FzRqyU8rNu s2EW2WYuBK9/GP+7mnnpuOLQGfNqJv2ph1E2BBOoS5eQE50ri85b6akewP68ZNs98aYf f/GXK/6jB/Z5X4TAaWp1rhPwhr0Ljtk9E1UD9vIPEsHLIDXLvaEM67YwY+KAnf8Hbfdf ur5A== X-Gm-Message-State: AA+aEWaNm3dLFVgi0eFgCZZI2s1opgvATNHCpUuZILu0IN86znNjli+7 gXEII4dN6U8RhjBa6NpUWvOQOw== X-Received: by 2002:a62:8949:: with SMTP id v70mr3505778pfd.85.1544213822767; Fri, 07 Dec 2018 12:17:02 -0800 (PST) Received: from ?IPv6:2600:1010:b029:c17c:a5a6:ef43:f7fe:6ce0? ([2600:1010:b029:c17c:a5a6:ef43:f7fe:6ce0]) by smtp.gmail.com with ESMTPSA id m3sm4965938pgl.69.2018.12.07.12.17.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 07 Dec 2018 12:17:01 -0800 (PST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (1.0) Subject: Re: [RFC PATCH v2 4/4] x86/vdso: Add __vdso_sgx_enter_enclave() to wrap SGX enclave transitions From: Andy Lutomirski X-Mailer: iPhone Mail (16B92) In-Reply-To: <20181207200935.GE10404@linux.intel.com> Date: Fri, 7 Dec 2018 12:16:59 -0800 Cc: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , X86 ML , Dave Hansen , Peter Zijlstra , "H. Peter Anvin" , LKML , Jarkko Sakkinen , Josh Triplett , linux-sgx@vger.kernel.org, haitao.huang@linux.intel.com, Jethro Beekman , "Dr. Greg Wettstein" Content-Transfer-Encoding: quoted-printable Message-Id: <4CEB5945-9562-40FA-8CCA-A1675D55B001@amacapital.net> References: <20181206221922.31012-1-sean.j.christopherson@intel.com> <20181206221922.31012-5-sean.j.christopherson@intel.com> <20181207165145.GB10404@linux.intel.com> <20181207190257.GC10404@linux.intel.com> <20181207200935.GE10404@linux.intel.com> To: Sean Christopherson Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > On Dec 7, 2018, at 12:09 PM, Sean Christopherson wrote: >=20 >> On Fri, Dec 07, 2018 at 11:23:10AM -0800, Andy Lutomirski wrote: >>=20 >>>> On Dec 7, 2018, at 11:02 AM, Sean Christopherson wrote: >>>>=20 >>>> On Fri, Dec 07, 2018 at 09:56:09AM -0800, Andy Lutomirski wrote: >>>> On Fri, Dec 7, 2018 at 8:51 AM Sean Christopherson >>>> wrote: >>>>> I like that the exit handler allows userspace to trap/panic with the f= ull >>>>> call stack in place, and in a dedicated path, i.e. outside of the basi= c >>>>> enter/exit code. An exit handler probably doesn't fundamentally chang= e >>>>> what userspace can do with respect to debugging/reporting, but I think= >>>>> it would actually simplify some userspace implementations, e.g. I'd us= e >>>>> it in my tests like so: >>>>>=20 >>>>> long fault_handler(struct sgx_enclave_exit_info *exit_info, void *tcs,= void *priv) >>>>> { >>>>> if (exit_info->leaf =3D=3D SGX_EEXIT) >>>>> return 0; >>>>>=20 >>>>> >>>>> } >>>>>=20 >>>>=20 >>>> Hmm. That't not totally silly, although you could accomplish almost >>>> the same thing by wrapping the vDSO helper in another function. >>>=20 >>> True, but I think there's value in having the option to intercept an >>> exception at the exact(ish) point of failure, without having to return >>> up the stack. >>>=20 >>> The enclave has full access to the process' memory space, including the >>> untrsuted stack. It's not too far fetched to envision a scenario where >>> the enclave corrupts the stack even if the enclave isn't intentionally >>> using the stack, e.g. the host passes in variable that a points at the >>> stack instead of whatever memory is supposed to be shared between the >>> enclave and host. It'd be nice to have the ability to triage something >>> like that without having to e.g. configure breakpoints on the stack. >>=20 >> Ah, I see. You=E2=80=99re saying that, if the non-enclave stare is corrup= ted such >> that RIP is okay and RSP still points somewhere reasonable but the retur= n >> address is garbage, then we can at least get to the fault handler and pri= nt >> something? >=20 > Yep. Even for something more subtle like GPR corruption it could dump the= > entire call stack before attempting to return back up. >=20 >> This only works if the fault handler pointer itself is okay, though, whic= h >> somewhat limits the usefulness, given that its pointer is quite likely to= >> be on the stack very close to the return address. >=20 > Yeah, it's not a silver bullet by any means, but it does seem useful for a= t > least some scenarios. Even exploding when invoking the handler instead of= > at a random point might prove useful, e.g. "calling my exit handler explod= ed, > maybe my enclave corrupted the stack!". Here=E2=80=99s another idea: calculate some little hash or other checksum of= RSP, RBP, and perhaps a couple words on the stack, and do: call __vdso_enclave_corrupted_state If you get a mismatch after return. That function could be: call __vdso_enclave_corrupted_state: ud2 And now the debug trace makes it very clear what happened. This may or may not be worth the effort. But ISTM the enclave is almost as l= ikely to corrupt the host state and the. EEXIT as it is to corrupt the host s= tate and then fault. BTW, I read through Fortanix=E2=80=99s documentation of the Windows SGX call= ing conventions, and it seems to want RSI and RDI as out params. Letting th= e vDSO be used to invoke Windows-targeted enclaves seems useful. >=20 >> I really wish the ENCLU instruction had seen fit to preserve some registe= rs. >=20 > Speaking of preserving registers, the asm blob needs to mark RBX as > clobbered since it's modified for EEXIT. Have fun with that. The x86_32 compiler seems to really like having its PIC= register preserved, and you may get some lovely compiler errors.=