Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Fri, 7 Dec 2018 08:51:45 -0800
From:   Sean Christopherson <sean.j.christopherson@intel.com>
To:     Andy Lutomirski <luto@kernel.org>
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        X86 ML <x86@kernel.org>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Peter Zijlstra <peterz@infradead.org>,
        "H. Peter Anvin" <hpa@zytor.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>,
        Josh Triplett <josh@joshtriplett.org>,
        linux-sgx@vger.kernel.org,
        Haitao Huang <haitao.huang@linux.intel.com>,
        Jethro Beekman <jethro@fortanix.com>,
        "Dr. Greg" <greg@enjellic.com>
Subject: Re: [RFC PATCH v2 4/4] x86/vdso: Add __vdso_sgx_enter_enclave() to
 wrap SGX enclave transitions
Message-ID: <20181207165145.GB10404@linux.intel.com>
References: <20181206221922.31012-1-sean.j.christopherson@intel.com>
 <20181206221922.31012-5-sean.j.christopherson@intel.com>
 <CALCETrXRJ645=08fyeoMQ949fLB1TvhsgERFVx5mAHdViEjq8Q@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CALCETrXRJ645=08fyeoMQ949fLB1TvhsgERFVx5mAHdViEjq8Q@mail.gmail.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

+Cc: linux-sgx, Haitao, Greg and Jethro

My apologies for neglecting to cc the SGX folks, original thread is here:

https://lkml.kernel.org/r/20181206221922.31012-1-sean.j.christopherson@intel.com

On Thu, Dec 06, 2018 at 02:50:01PM -0800, Andy Lutomirski wrote:
> On Thu, Dec 6, 2018 at 2:19 PM Sean Christopherson
> <sean.j.christopherson@intel.com> wrote:
> >
>  +
> > +       /*
> > +        * Invoke the caller's exit handler if one was provided.  The return
> > +        * value tells us whether to re-enter the enclave (EENTER or ERESUME)
> > +        * or to return (EEXIT).
> > +        */
> > +       if (exit_handler) {
> > +               leaf = exit_handler(exit_info, tcs, priv);
> > +               if (leaf == SGX_EENTER || leaf == SGX_ERESUME)
> > +                       goto enter_enclave;
> > +               if (leaf == SGX_EEXIT)
> > +                       return 0;
> > +               return -EINVAL;
> > +       } else if (leaf != SGX_EEXIT) {
> > +               return -EFAULT;
> > +       }
> 
> This still seems overcomplicated to me.  How about letting the
> requested leaf (EENTER or ERESUME) be a parameter to the function and
> then just returning here?  As it stands, you're requiring any ERESUME
> that gets issued (other than the implicit ones) to be issued in the
> same call stack, which is very awkward if you're doing something like
> forwarding the fault to a different task over a socket and then
> waiting in epoll_wait() or similar before resuming the enclave.

Ah, yeah, wasn't thinking about usage models where the enclave could
get passed off to a different thread.

What about supporting both, i.e. keep the exit handler but make it 100%
optional?  And simplify the exit_handler to effectively return a boolean,
i.e. "exit or continue".

Something like this:

notrace long __vdso_sgx_enter_enclave(u32 op, void *tcs, void *priv,
				      struct sgx_enclave_exit_info *exit_info,
				      sgx_enclave_exit_handler *exit_handler)
{
	u64 rdi, rsi, rdx;
	u32 leaf;
	long ret;

	if (!tcs || !exit_info)
		return -EINVAL;

enter_enclave:
	if (op != SGX_EENTER && op != SGX_ERESUME)
		return -EINVAL;

        <same core code>

	/*
	 * Invoke the caller's exit handler if one was provided.  The return
	 * value tells us whether to re-enter the enclave (EENTER or ERESUME)
	 * or to return (EEXIT).
	 */
	if (exit_handler) {
		if (exit_handler(exit_info, tcs, priv)) {
			op = exit_info->leaf;
			goto enter_enclave;
		}
	}

	if (exit_info->leaf == SGX_EEXIT)
		return -EFAULT;

	return 0;
}


I like that the exit handler allows userspace to trap/panic with the full
call stack in place, and in a dedicated path, i.e. outside of the basic
enter/exit code.  An exit handler probably doesn't fundamentally change
what userspace can do with respect to debugging/reporting, but I think
it would actually simplify some userspace implementations, e.g. I'd use
it in my tests like so:

long fault_handler(struct sgx_enclave_exit_info *exit_info, void *tcs, void *priv)
{
	if (exit_info->leaf == SGX_EEXIT)
		return 0;

	<report exception and die/hang>
}