Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Tue, 8 Jan 2019 14:09:47 -0800
From:   Sean Christopherson <sean.j.christopherson@intel.com>
To:     "Huang, Kai" <kai.huang@intel.com>
Cc:     Andy Lutomirski <luto@kernel.org>,
        Jethro Beekman <jethro@fortanix.com>,
        Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        "x86@kernel.org" <x86@kernel.org>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Peter Zijlstra <peterz@infradead.org>,
        "H. Peter Anvin" <hpa@zytor.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-sgx@vger.kernel.org" <linux-sgx@vger.kernel.org>,
        Josh Triplett <josh@joshtriplett.org>,
        Haitao Huang <haitao.huang@linux.intel.com>,
        "Dr . Greg Wettstein" <greg@enjellic.com>
Subject: Re: x86/sgx: uapi change proposal
Message-ID: <20190108220946.GA30462@linux.intel.com>
References: <20181214215729.4221-1-sean.j.christopherson@intel.com>
 <7706b2aa71312e1f0009958bcab24e1e9d8d1237.camel@linux.intel.com>
 <598cd050-f0b5-d18c-96a0-915f02525e3e@fortanix.com>
 <20181219091148.GA5121@linux.intel.com>
 <613c6814-4e71-38e5-444a-545f0e286df8@fortanix.com>
 <20181219144515.GA30909@linux.intel.com>
 <CALCETrW_8LnitFamYUU7POHFOJFxLO7ESFUdS=aeG_SOvHVPkw@mail.gmail.com>
 <20181221162825.GB26865@linux.intel.com>
 <105F7BF4D0229846AF094488D65A0989355A45B6@PGSMSX112.gar.corp.intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <105F7BF4D0229846AF094488D65A0989355A45B6@PGSMSX112.gar.corp.intel.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, Jan 08, 2019 at 11:27:11AM -0800, Huang, Kai wrote:
> > >
> > > Can one of you explain why SGX_ENCLAVE_CREATE is better than just
> > > opening a new instance of /dev/sgx for each encalve?
> > 
> > Directly associating /dev/sgx with an enclave means /dev/sgx can't be used
> > to provide ioctl()'s for other SGX-related needs, e.g. to mmap() raw EPC and
> > expose it a VM.  Proposed layout in the link below.  I'll also respond to
> > Jarkko's question about exposing EPC through /dev/sgx instead of having
> > KVM allocate it on behalf of the VM.
> > 
> > https://lkml.kernel.org/r/20181218185349.GC30082@linux.intel.com
> 
> Hi Sean,
> 
> Sorry for replying to old email. But IMHO it is not a must that Qemu
> needs to open some /dev/sgx and allocate/mmap EPC for guest's virtual
> EPC slot, instead, KVM could create private slot, which is not visible
> to Qemu, for virtual EPC, and KVM could call core-SGX EPC allocation
> API directly.

That's possible, but it has several downsides.

  - Duplicates a lot of code in KVM for managing memory regions.
  - Artificially restricts userspace to a single EPC region, unless
    even more code is duplicated to handle multiple private regions.
  - Requires additional ioctls() or capabilities to probe EPC support
  - Does not fit with Qemu/KVM's memory model, e.g. all other types of
    memory are exposed to a guest through KVM_SET_USER_MEMORY_REGION. 
  - Prevents userspace from debugging a guest's enclave.  I'm not saying
    this is a likely scenario, but I also don't think we should preclude
    it without good reason.
  - KVM is now responsible for managing the lifecycle of EPC, e.g. what
    happens if an EPC cgroup limit is lowered on a running VM and
    KVM can't gracefully reclaim EPC?  The userspace hypervisor should
    ultimately decide how to handle such an event.
  - SGX logic is split between SGX and KVM, e.g. VA page management for
    oversubscription will likely be common to SGX and KVM.  From a long
    term maintenance perspective, this means that changes to the EPC
    management could potentially need to be Acked by KVM, and vice versa.

> I am not sure what's the good of allowing userspace to alloc/mmap a
> raw EPC region? Userspace is not allowed to touch EPC anyway, expect
> enclave code.
>
> To me KVM creates private EPC slot is cleaner than exposing /dev/sgx/epc
> and allowing userspace to map some raw EPC region.

Cleaner in the sense that it's faster to get basic support up and running
since there are fewer touchpoints, but there are long term ramifications
to cramming EPC management in KVM.

And at this point I'm not stating any absolutes, e.g. how EPC will be
handled by KVM.  What I'm pushing for is to not eliminate the possibility
of having the SGX subsystem own all EPC management, e.g. don't tie
/dev/sgx to a single enclave.

> 
> Thanks,
> -Kai