Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <86495779-a9c5-45d5-0017-c491bf6354ab@intel.com>
Date:   Tue, 22 Feb 2022 14:39:31 -0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Firefox/91.0 Thunderbird/91.6.0
Subject: Re: [PATCH V2 00/32] x86/sgx and selftests/sgx: Support SGX2
Content-Language: en-US
To:     Nathaniel McCallum <nathaniel@profian.com>
CC:     <dave.hansen@linux.intel.com>, Jarkko Sakkinen <jarkko@kernel.org>,
        <tglx@linutronix.de>, <bp@alien8.de>,
        Andy Lutomirski <luto@kernel.org>, <mingo@redhat.com>,
        <linux-sgx@vger.kernel.org>, <x86@kernel.org>, <seanjc@google.com>,
        <kai.huang@intel.com>, <cathy.zhang@intel.com>,
        <cedric.xing@intel.com>, <haitao.huang@intel.com>,
        <mark.shanahan@intel.com>, <hpa@zytor.com>,
        <linux-kernel@vger.kernel.org>
References: <cover.1644274683.git.reinette.chatre@intel.com>
 <CAHAy0tR-64vxtFo4KXiJP_va2=WF++Q6gDaPksNxiaSB5wWvhA@mail.gmail.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <CAHAy0tR-64vxtFo4KXiJP_va2=WF++Q6gDaPksNxiaSB5wWvhA@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZkJIU05GOVhyd2RJNTV2bk55bmVIRDZCY3ZCRVQwTWViNXFmQVFnL0hJQVVY?=
 =?utf-8?B?QmlaUzY5TnBISkNjNW1OUjl5VmRPTlowWkZtMUJlRVlpYnNlS3JEemdLdG1R?=
 =?utf-8?B?dDQ4bkdxSm5mZkdxMEYzalFBWUIrMW93aXcxSUFiSzQyOG92Y25YU2QyUnph?=
 =?utf-8?B?ZG9MdVVLb0s0Tm1NVEh5dGdHYnV5akVMYlRwNU9CQnc2aDNoMUVvVnIrcFJw?=
 =?utf-8?B?eFcvUHAyYVQwbi94OWFRYWw5WHErU1N5dWVUV2pwcmMyT2RTQ20wZm9aK2lE?=
 =?utf-8?B?cTRqWFFRRkJnWFF6RnpuQjY4SEZuY1h4aDM3WnVCSEpsc0lkNWwyNmRIUHZT?=
 =?utf-8?B?Sk9KbVN0ejJDcnA5ZS96NDA4bXdtenl2MTcwUjlvOHZmTXJOV1VmeThpOTFH?=
 =?utf-8?B?OW5id09NM3ZNNmlJTXl0OEZleXIrSE84UmJ3VEJ0VkFBZGR2WU1SSXZLS2RO?=
 =?utf-8?B?SDZDV29vZFY2MFhibDF3M05IbTN2ZncvYUFxTXpKQVMxOTZTbnJKNlVTZ1VF?=
 =?utf-8?B?UVAwOGR6VUk2Y2RqWXFNN0tmdjkrVW9sc1kzR0toL3hhUzBvQk9Sakt6MXRj?=
 =?utf-8?B?bnErQVhrYktPTmxsWmJ4MCtkSG1QVDhyQ1RLSWc2VnVTdk1VNmpJT3AwYU9x?=
 =?utf-8?B?bHJJNlhNcnI3Z2NYc0dRZnl5SGdwNWhyVytXYUN1YzRGKytnS0hxV1kvZjAx?=
 =?utf-8?B?WE1GUkwvc3ZEK3ZqUHN2bngrWGFicVovdS9JbW5WbkY2WGpiak9iREh6TUdY?=
 =?utf-8?B?a0ZIb3RyQm11elFyanlReVp0VzM4YkJJVkhDZFdjV1VEbG5HWUdxMTY2cFFo?=
 =?utf-8?B?WXVzR1FFMjhQZ1hyS3NDMzZwVExKa1BYcy9QcDN5bEwxM3FKV3RQckFJTmhC?=
 =?utf-8?B?RUpCdzJZd2hwVEVzNWZCSGczSWxmQ1pCTTF0cUVHNkQrYWtSa3Bha1BMUXpm?=
 =?utf-8?B?STZnUjVYc3ZyVmxEcTBvQUU4cy9NQjgxTnRTZzNkYzZ6eTdqd3hDVUxWL1lt?=
 =?utf-8?B?WnNvWm5RajYydFdBS0JiRjVLcUxiOHJQRnhiWityVzJOK2s2SmxwTlB5WGtE?=
 =?utf-8?B?anBLdFQ1VUR2ak16NFZoYTMxeiswSGtKMjE5SlNVSmpMWklGTDZVNndzSWxq?=
 =?utf-8?B?OUJhekZDWDNoUHBRWDhTSi9FQitMazd4c1RkRDJiV1kwZHQrYm1VdFVwa2Y0?=
 =?utf-8?B?VEtuRWVSZC83YTdoZzR1TmZQYURGdnMrR2hwVjY0R0xZUFRzS2NTYmtXamw0?=
 =?utf-8?B?RXZSditQdmM4ZE94Q1BNT3RIUWtrU2FjeHhXamZjbkxkam9MeUFuRTFBd1hp?=
 =?utf-8?B?a2ZVR0hwN0MyUzM5VkxZOHR2Z3p6THJIZnRZZVhKLzZhcThWc21aYldUTXNl?=
 =?utf-8?B?VDlueWViMXA5RUJTWTdqWkd3bmx2ak5SLyt0VTZrbnFEM0tQVkpDc25UVHZv?=
 =?utf-8?B?THZGOU1nNXl0Zk9qaUtacWVJV2JEeS8wanh5L29nR0F1SVp5M2IraWVXY25X?=
 =?utf-8?B?TnpCSTQ5Mm41T3pxWTdrRzZHY2d5QzJsWjlhZXBvWVhDRUVQZDdSWkRZRGMw?=
 =?utf-8?B?NWFYSGNoN0VIcmU2clNKeGhTN0NjV3oyTjFUVFFEeU5hemh3UDZzOGpWNVFJ?=
 =?utf-8?B?UWNkbjVlZVFGSFBReCtlYndrbS9udVRObTlwYnQ4cE5SVHpmS0FUQXJ4U21H?=
 =?utf-8?B?SGIzVmVjNHB3WmpnR0V3U2tla25DZnNnRFlVQnA3WFBjNmhDWC9adkk5Y2FH?=
 =?utf-8?B?STdLcldORVdEOHZuQ2Q3TkJXMkhrWGMrL2xsZ2ZIZWZCOEtoaE96Q0JEbEdR?=
 =?utf-8?B?eTFjUmkzS1E1NlBLWFdFSTRuZWZPaWgyUnhjYlp4ejRaRE1pY2EzS285SUtG?=
 =?utf-8?B?N1dxeVdqR3dWb0tzSmZGVCtyYW84VGhIYndXVWhZaW1hb1lkQnJmTlcydU5N?=
 =?utf-8?B?aW1ldjJhZnFqSEVUWWlKMEJCZDdzcnRuUGlHaVdrZnVJdjBiVnh1UXdNdnVS?=
 =?utf-8?B?TXk5YzMvaEZ4SXQ1V3ovV0NqWjZzeEpHek1VY2JZMnJTb092d1NFZnFRTnFH?=
 =?utf-8?B?akpkL0QxaXFRUE1QUklmekN0WTAzNXB3TDBCbUFoWkg2Y2lGSDRubmpHSW9z?=
 =?utf-8?B?dGFDSXg1VWNNeDlsRUVsekxMSjZtWXM1SkFnVlQySGM4cHBtajFvMEZYdXB6?=
 =?utf-8?B?TW5oWGZqcnF1eGxobEdVTlRVTXpiN3kwTERMaC8vcWtWd1dsRzRiRUdwaGVX?=
 =?utf-8?Q?fd5MzO2vHPKL1doNKdJeedplrJ9rEUvx8WTuh2Tcgo=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 50325e33-0fe9-43f2-ce89-08d9f6543490
X-MS-Exchange-CrossTenant-AuthSource: BN0PR11MB5744.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Feb 2022 22:39:38.0463
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ziP5OsCWKSpn46ZEqBSysB2RPzFjo05vQEYrraKiGQoA4GTtHwe1g+rprebYoUEchYyGSGxOjufBtyOikhj/fCXhZqIUf8NpOlQLyyBPML4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN8PR11MB3843
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Nathaniel,

On 2/22/2022 12:27 PM, Nathaniel McCallum wrote:
> 1. This interface looks very odd to me. mmap() is the kernel interface
> for changing user space memory maps. Why are we introducing a new
> interface for this?

mmap() is the kernel interface used to create new mappings in the
virtual address space of the calling process. This is different from
the permissions and properties of the underlying file/memory being mapped.

A new interface is introduced because changes need to be made to the
permissions and properties of the underlying enclave. A new virtual
address space is not needed nor should existing VMAs be impacted.

This is similar to how mmap() is not used to change file permissions.

VMA permissions are separate from enclave page permissions as found in
the EPCM (Enclave Page Cache Map). The current implementation (SGX1) already
distinguishes between the VMA and EPCM permissions - for example, it is
already possible to create a read-only VMA from enclave pages that have
RW EPCM permissions. mmap() of a portion of EPC memory with a particular
permission does not imply that the underlying EPCM permissions (should)have
that permission. 

> You can just simply add a new mmap flag (i.e.
> MAP_SGX_TCS*) and then figure out which SGX instructions to execute
> based on the desired state of the memory maps. If you do this, none of
> the following ioctls are needed:
> 
> * SGX_IOC_ENCLAVE_RELAX_PERMISSIONS
> * SGX_IOC_ENCLAVE_RESTRICT_PERMISSIONS
> * SGX_IOC_ENCLAVE_REMOVE_PAGES
> * SGX_IOC_ENCLAVE_MODIFY_TYPE
> 
> It also means that languages don't have to grow support for all these
> ioctls. Instead, they can just reuse the existing mmap() bindings with
> the new flag. Also, multiple operations can be combined into a single
> mmap() call, amortizing the changes over a single context switch.
> 
> 2. Automatically adding pages with hard-coded permissions in a fault
> handler seems like a really bad idea.

Could you please elaborate why this is a bad idea?

> How do you distinguish between
> accesses which should result in an updated mapping and accesses that
> should result in a fault?

Accesses that should result in an updated mapping have two requirements:
(a) address accessed belongs to the enclave based on the address
    range specified during enclave create
(b) there is no backing enclave page for the address

> IMHO, all unmapped page accesses should
> result in a page fault. mmap() should be called first to identify the
> correct permissions for these pages.
> Then the page handler should be
> updated to use the permissions from the mapping when backfilling
> physical pages. If I understand correctly, this should also obviate

Regular enclave pages can _only_ be dynamically added with RW permission.

SGX2's support for adding regular pages to an enclave via the EAUG
instruction is architecturally set at RW. The OS cannot change those permissions
via the EAUG instruction nor can the OS do so with a different/additional
instruction because:
* the OS is not able to relax permissions since that can only be done from
within the enclave with ENCLU[EMODPE], thus it is not possible for the OS to
dynamically add pages via EAUG as RW and then relax permissions to RWX. 
* the OS is not able to EAUG a page and immediately attempt an EMODPR either
as Jarkko also recently inquired about:
https://lore.kernel.org/linux-sgx/80f3d7b9-e3d5-b2c0-7707-710bf6f5081e@intel.com/

> the need for the weird userspace callback to allow for execute
> permissions.

User policy integration would always be required to allow execute
permissions on a writable page. This is not expected to be a userspace
callback but instead integration with existing user policy subsystem(s).

> 
> 3. Implementing as I've suggested also means that we can lock down an
> enclave, for example - after code has been JITed, by closing the file
> descriptor. Once the file descriptor used to create the enclave is
> closed, no further mmap() can be performed on the enclave. Attempting
> to do EACCEPT on an unmapped page will generate a page fault.

This is not clear to me. If the file descriptor is closed and no further
mmap() is allowed then how would a process be able to enter the enclave
to execute code within it?

This series does indeed lock down the address range to ensure that it is
not possible to map memory that does not belong to the enclave after the
enclave is created. Please see:
https://lore.kernel.org/linux-sgx/1b833dbce6c937f71523f4aaf4b2181b9673519f.1644274683.git.reinette.chatre@intel.com/

> 
> * - I'm aware that a new flag might be frowned upon. I see a few other options:
> 1. reuse an existing flag which doesn't make sense in this context
> 2. communicate the page type in the offset argument
> 3. keep SGX_IOC_ENCLAVE_MODIFY_TYPE
> 

Reinette