Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20181214215729.4221-1-sean.j.christopherson@intel.com>
 <7706b2aa71312e1f0009958bcab24e1e9d8d1237.camel@linux.intel.com>
 <598cd050-f0b5-d18c-96a0-915f02525e3e@fortanix.com> <20181219091148.GA5121@linux.intel.com>
 <613c6814-4e71-38e5-444a-545f0e286df8@fortanix.com> <20181219144515.GA30909@linux.intel.com>
 <CALCETrW_8LnitFamYUU7POHFOJFxLO7ESFUdS=aeG_SOvHVPkw@mail.gmail.com>
 <20181221162825.GB26865@linux.intel.com> <CALCETrX+KisMCbptrnPSO79-YF4E3nR1XHt+a7hCs1GXsxAbtw@mail.gmail.com>
 <20181221182454.GA27371@linux.intel.com>
In-Reply-To: <20181221182454.GA27371@linux.intel.com>
From:   Andy Lutomirski <luto@kernel.org>
Date:   Sun, 23 Dec 2018 12:41:49 -0800
Message-ID: <CALCETrVYJmutVEemMZRc=neFT4Ph+JUgapTWy5JS6KTH9eamHA@mail.gmail.com>
Subject: Re: x86/sgx: uapi change proposal
To:     Sean Christopherson <sean.j.christopherson@intel.com>
Cc:     Andy Lutomirski <luto@kernel.org>,
        Jethro Beekman <jethro@fortanix.com>,
        Jarkko Sakkinen <jarkko.sakkinen@linux.intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        "x86@kernel.org" <x86@kernel.org>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Peter Zijlstra <peterz@infradead.org>,
        "H. Peter Anvin" <hpa@zytor.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-sgx@vger.kernel.org" <linux-sgx@vger.kernel.org>,
        Josh Triplett <josh@joshtriplett.org>,
        Haitao Huang <haitao.huang@linux.intel.com>,
        "Dr . Greg Wettstein" <greg@enjellic.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

> On Dec 21, 2018, at 11:24 AM, Sean Christopherson <sean.j.christopherson@=
intel.com> wrote:
>
> On Fri, Dec 21, 2018 at 09:12:46AM -0800, Andy Lutomirski wrote:
>>> On Dec 21, 2018, at 9:28 AM, Sean Christopherson <sean.j.christopherson=
@intel.com> wrote:
>>>
>>> On Wed, Dec 19, 2018 at 06:58:48PM -0800, Andy Lutomirski wrote:
>>>>> On Dec 19, 2018, at 6:45 AM, Sean Christopherson <sean.j.christophers=
on@intel.com> wrote:
>>>>>
>>>>>>> On Wed, Dec 19, 2018 at 09:36:16AM +0000, Jethro Beekman wrote:
>>>>>
>>>>> I agree with Jethro, passing the enclave_fd as a param is obnoxious.
>>>>> And it means the user needs to open /dev/sgx to do anything with an
>>>>> enclave fd, e.g. the enclave fd might be passed to a builder thread,
>>>>> it shouldn't also need the device fd.
>>>>>
>>>>> E.g.:
>>>>>
>>>>>  sgx_fd =3D open("/dev/sgx", O_RDWR);
>>>>>  BUG_ON(sgx_fd < 0);
>>>>>
>>>>>  enclave_fd =3D ioctl(sgx_fd, SGX_ENCLAVE_CREATE, &ecreate);
>>>>>  BUG_ON(enclave_fd < 0);
>>>>>
>>>>>  ret =3D ioctl(enclave_fd, SGX_ENCLAVE_ADD_PAGE, &eadd);
>>>>>  BUG_ON(ret);
>>>>>
>>>>>  ...
>>>>>
>>>>>  ret =3D ioctl(enclave_fd, SGX_ENCLAVE_INIT, &einit);
>>>>>  BUG_ON(ret);
>>>>>
>>>>>  ...
>>>>>
>>>>>  close(enclave_fd);
>>>>>  close(sgx_fd);
>>>>>
>>>>>
>>>>> Take a look at virt/kvm/kvm_main.c to see how KVM manages anon inodes
>>>>> and ioctls for VMs and vCPUs.
>>>>
>>>> Can one of you explain why SGX_ENCLAVE_CREATE is better than just
>>>> opening a new instance of /dev/sgx for each encalve?
>>>
>>> Directly associating /dev/sgx with an enclave means /dev/sgx can't be
>>> used to provide ioctl()'s for other SGX-related needs, e.g. to mmap()
>>> raw EPC and expose it a VM.  Proposed layout in the link below.  I'll
>>> also respond to Jarkko's question about exposing EPC through /dev/sgx
>>> instead of having KVM allocate it on behalf of the VM.
>>
>> Hmm. I guess this makes some sense.  My instinct would be to do it a
>> little differently and have:
>>
>> /dev/sgx/enclave: Each instance is an enclave.
>>
>> /dev/sgx/epc: Used to get raw EPC for KVM. Might have different
>> permissions, perhaps 0660 and group kvm.
>>
>> /dev/sgx/something_else: For when SGX v3 adds something else :)
>
> Mmmm, I like this approach a lot.  It would allow userspace to easily
> manage permissions for each "feature", e.g. give all users access to
> /dev/sgx/epc but restrict /dev/sgx/enclave.
>
> And we could add e.g. /dev/sgx/admin if we wanted to exposed ioctls()
> that apply to all aspects of SGX.
>
> Do you know if /dev/sgx/epc could be dynamically created, e.g. by
> KVM when the kvm_intel module is loaded?

Presumably sgx would create a bus and enumerate the devices as needed.
Or I suppose these things could be platform or system devices. I=E2=80=99m =
not
really a device model expert, and the one time I tried to implement a
character device, I got so disgusted that I wrote a whole library for
it.  It=E2=80=99s still in limbo somewhere.


>  That would seal the deal for
> me as it'd keep open the option of having KVM handle oversubscription
> of guest EPC while exposing EPC through /dev/sgx instead of /dev/kvm.