Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <c63ad0cd-d517-0f1e-59e9-927d8ae15a1a@amd.com>
Date:   Mon, 17 Oct 2022 18:39:06 +0200
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.10.0
Subject: Re: [PATCH v8 1/8] mm/memfd: Introduce userspace inaccessible memfd
Content-Language: en-US
To:     "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>,
        Vlastimil Babka <vbabka@suse.cz>
Cc:     Chao Peng <chao.p.peng@linux.intel.com>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, linux-mm@kvack.org,
        linux-fsdevel@vger.kernel.org, linux-api@vger.kernel.org,
        linux-doc@vger.kernel.org, qemu-devel@nongnu.org,
        Paolo Bonzini <pbonzini@redhat.com>,
        Jonathan Corbet <corbet@lwn.net>,
        Sean Christopherson <seanjc@google.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        x86@kernel.org, "H . Peter Anvin" <hpa@zytor.com>,
        Hugh Dickins <hughd@google.com>,
        Jeff Layton <jlayton@kernel.org>,
        "J . Bruce Fields" <bfields@fieldses.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Shuah Khan <shuah@kernel.org>, Mike Rapoport <rppt@kernel.org>,
        Steven Price <steven.price@arm.com>,
        "Maciej S . Szmigiero" <mail@maciej.szmigiero.name>,
        Vishal Annapurve <vannapurve@google.com>,
        Yu Zhang <yu.c.zhang@linux.intel.com>, luto@kernel.org,
        jun.nakajima@intel.com, dave.hansen@intel.com, ak@linux.intel.com,
        david@redhat.com, aarcange@redhat.com, ddutile@redhat.com,
        dhildenb@redhat.com, Quentin Perret <qperret@google.com>,
        Michael Roth <michael.roth@amd.com>, mhocko@suse.com,
        Muchun Song <songmuchun@bytedance.com>, wei.w.wang@intel.com
References: <20220915142913.2213336-1-chao.p.peng@linux.intel.com>
 <20220915142913.2213336-2-chao.p.peng@linux.intel.com>
 <de680280-f6b1-9337-2ae4-4b2faf2b823b@suse.cz>
 <20221017161955.t4gditaztbwijgcn@box.shutemov.name>
From:   "Gupta, Pankaj" <pankaj.gupta@amd.com>
In-Reply-To: <20221017161955.t4gditaztbwijgcn@box.shutemov.name>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZXRHZTJ1c0syN2lqcUpBNWVaWi9vcG9oMjZOSk5MQ0Zjb0RYZDBiT2xacG9v?=
 =?utf-8?B?aHBiSEVycVc4b2dsWW54eTcrN2JLS0VVS0FaNHJsUm1wczJ3SHRJVEtSS0tF?=
 =?utf-8?B?My9mdm5FdmVPSFlkSllYZ3p1K1NzUENDRCtoQXBRY21VOXZGQkkyV2ZvYnFL?=
 =?utf-8?B?RkdmaEptSTZGQU1MUVcxMVd2V0dya0ZDSy9OZDY3dmpxQ1EwaUo2NVRhdWNv?=
 =?utf-8?B?dDRKUTZxSk4xdGtUdVMxYVVhV2dNL1NTNENyRHlDaDZ4Z1NWWjRZb2JrMnRE?=
 =?utf-8?B?WTRCRUpEQy91Tm9hSGRzdGlJQklaUWd0MUs0cDJMQUQzVk9JV2oxZUhMVGtv?=
 =?utf-8?B?NGlsbnJaOG1ybVBZbzZLTFJDdERYd0dkbTRvRUExL2hseXV5aTRFUjMzeCtG?=
 =?utf-8?B?b1VjRW16bHVKZnJqci8rK1NyNW1MUmg3M3FCdlprQ2I2bFhNS09wRW9Qczkx?=
 =?utf-8?B?d3FSS29YR0FkMWlBYTg5RWNQWlZkcFN3TE5qdWJ1WGtxSnZ5dVRqcUh0UDU3?=
 =?utf-8?B?MlhSa1lqQVRyYXkxSTU0Z1JtWXIrcGdMUmhpZ1Y4WWZZdWRMY090ektnWWZa?=
 =?utf-8?B?eDVPb24yNGFFSmZMYzhKRVh0Zy9Pb1FMcVlIdUErajlwU1dLaG1NZHp2dnNR?=
 =?utf-8?B?RmtzWTdqKzBMeWNvSXdMVUNydFJleDJmY0hEcXc4S1MwT2tqbjhkdnhHeFRx?=
 =?utf-8?B?andEWm5YL2llRkZBVnZPOHc0OGdoeFZ2R1A1ZW81UTZCa1ZoWlRvbUdLK3Ix?=
 =?utf-8?B?U0lZdHphT3lZYWJCS1lzaTNhV1MwOWxWWGgvb0JkNVhscm1QWEg4OU42TWFB?=
 =?utf-8?B?d0ZBU21uMjZXSW00ZXJLSDk4NVZuUEFEaEY1STZuRUl0YXZ3K1E1aXNIUkU3?=
 =?utf-8?B?eW5zelVZaFJPV0MzOVdrQmRyZ2pUb1N4bzk2R3diNFhLKzNsSnV0cTRseFRz?=
 =?utf-8?B?dmQxaFZ6cnZnTTl6OWpsdDdHeGJQUFBSajZZVjdaeGlkWEQwbkUzVzlzRHN4?=
 =?utf-8?B?cnJlWXRKZjFjSUtFcEZGaU5WYytkMGVmM0JOSitoZmJ2RUVWcjRVbkJNaUph?=
 =?utf-8?B?WG8yZVJXT3gwVTNUU0p4d3k1ZHdvMXdicTFBbjFlSW9KcmFja21sMG9TY1ll?=
 =?utf-8?B?Tit2dHoxSXF6SXpITDVrZGVDMWdiMzQ4cUt2dDFxdVk4SGszMFZxL1VneDc0?=
 =?utf-8?B?OUVjM211V1o0cHpXTjlEdlhkelFWU1B1NGN6YVJQTEFnWEtkVzhIOGl3Ym5t?=
 =?utf-8?B?UHdUTDFEL0U2NnpKalY3b00rRUgzYUV5Y0FZaGJYZlRCTWF6SzRCRm5TLzcw?=
 =?utf-8?B?anNQNlpFWkk5YU5VdXVtbTN6TjV4ejAzVlNSRGp5WElsVEFKMmx2TWk2VmMv?=
 =?utf-8?B?cmYyRGh3V2tWVldKTThLZ081RDFtYk5LUktubzNZVkFjMlNZekxpMTNMNnds?=
 =?utf-8?B?WURIUThRM1dxVVB0dDU2ak1GT09SbncyOUNobm1ZQ29OUGFNMDhNdWc3SWdR?=
 =?utf-8?B?bGNnaWdsVE5odTJRdW1BY3FLbUgvb2JHdXpnTGFpaU5nUnZ0dnVCK1lhNFlL?=
 =?utf-8?B?a0p2MVE2UFhLZnNtQ01GK3RRYmd2VDhLNEVwcDNINUNVejYydEZkWmlSVVRH?=
 =?utf-8?B?UzZBTmh5T01oME00UVVhcyt0VGttVnhJd3B1T2Q1Z1VUdzFyOXdGMUxvQ1BL?=
 =?utf-8?B?aFFubmJONS9NZENjejhuQzl1WG1JS2FoWU8wSVdMRllIVWYxSzlhY0FFRlRF?=
 =?utf-8?B?cXl2enY0V0xFajVwYkRLYSs2VDVJeVhzZkFQcC9GVElXaW5OSDZVQnNQcnFj?=
 =?utf-8?B?Rzd2Ny92NENkM05UYmQwY3RxTDNQNDJXZEt5TVl2VVpsZVA3UkNMVFpkbHVj?=
 =?utf-8?B?Tzh5eTJwN3k1TXN0QU5DNlh0c3VCTjBGUytPY04yUnFwVzdqVVJiT3hIb1lM?=
 =?utf-8?B?eXJtRklLTW5qaU5pdXZCbndJNjFyc3RkRTR1VGVsbmxYcFV5cENNbmk5Z2RP?=
 =?utf-8?B?cWFmbWhyZlhYVzEzUXZrQ2Q1VUtuUVFTRXFxbDRGZVA5UUM3dlYwSHRrYmdH?=
 =?utf-8?B?UWhmODRmb0hEU2F1ZERhQ2VNMi90bVpXeHVKdElmSjRiUDZ4MDg3bjR0d24r?=
 =?utf-8?Q?REi460e6t3cVyht6+rzaSyQr7?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 06df2695-d081-4b44-cf18-08dab05e2502
X-MS-Exchange-CrossTenant-AuthSource: CY4PR1201MB0181.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2022 16:39:22.3514
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 41dhMx0IR/+IPVAoO+iNDiVrpG0LA2kkZwlU8h1H6xSt2T0TcR55Mpy+nA/ISADesHNnhilEbg3wPq70BGla1g==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6828
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 10/17/2022 6:19 PM, Kirill A . Shutemov wrote:
> On Mon, Oct 17, 2022 at 03:00:21PM +0200, Vlastimil Babka wrote:
>> On 9/15/22 16:29, Chao Peng wrote:
>>> From: "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
>>>
>>> KVM can use memfd-provided memory for guest memory. For normal userspace
>>> accessible memory, KVM userspace (e.g. QEMU) mmaps the memfd into its
>>> virtual address space and then tells KVM to use the virtual address to
>>> setup the mapping in the secondary page table (e.g. EPT).
>>>
>>> With confidential computing technologies like Intel TDX, the
>>> memfd-provided memory may be encrypted with special key for special
>>> software domain (e.g. KVM guest) and is not expected to be directly
>>> accessed by userspace. Precisely, userspace access to such encrypted
>>> memory may lead to host crash so it should be prevented.
>>>
>>> This patch introduces userspace inaccessible memfd (created with
>>> MFD_INACCESSIBLE). Its memory is inaccessible from userspace through
>>> ordinary MMU access (e.g. read/write/mmap) but can be accessed via
>>> in-kernel interface so KVM can directly interact with core-mm without
>>> the need to map the memory into KVM userspace.
>>>
>>> It provides semantics required for KVM guest private(encrypted) memory
>>> support that a file descriptor with this flag set is going to be used as
>>> the source of guest memory in confidential computing environments such
>>> as Intel TDX/AMD SEV.
>>>
>>> KVM userspace is still in charge of the lifecycle of the memfd. It
>>> should pass the opened fd to KVM. KVM uses the kernel APIs newly added
>>> in this patch to obtain the physical memory address and then populate
>>> the secondary page table entries.
>>>
>>> The userspace inaccessible memfd can be fallocate-ed and hole-punched
>>> from userspace. When hole-punching happens, KVM can get notified through
>>> inaccessible_notifier it then gets chance to remove any mapped entries
>>> of the range in the secondary page tables.
>>>
>>> The userspace inaccessible memfd itself is implemented as a shim layer
>>> on top of real memory file systems like tmpfs/hugetlbfs but this patch
>>> only implemented tmpfs. The allocated memory is currently marked as
>>> unmovable and unevictable, this is required for current confidential
>>> usage. But in future this might be changed.
>>>
>>> Signed-off-by: Kirill A. Shutemov <kirill.shutemov@linux.intel.com>
>>> Signed-off-by: Chao Peng <chao.p.peng@linux.intel.com>
>>> ---
>>
>> ...
>>
>>> +static long inaccessible_fallocate(struct file *file, int mode,
>>> +				   loff_t offset, loff_t len)
>>> +{
>>> +	struct inaccessible_data *data = file->f_mapping->private_data;
>>> +	struct file *memfd = data->memfd;
>>> +	int ret;
>>> +
>>> +	if (mode & FALLOC_FL_PUNCH_HOLE) {
>>> +		if (!PAGE_ALIGNED(offset) || !PAGE_ALIGNED(len))
>>> +			return -EINVAL;
>>> +	}
>>> +
>>> +	ret = memfd->f_op->fallocate(memfd, mode, offset, len);
>>> +	inaccessible_notifier_invalidate(data, offset, offset + len);
>>
>> Wonder if invalidate should precede the actual hole punch, otherwise we open
>> a window where the page tables point to memory no longer valid?
> 
> Yes, you are right. Thanks for catching this.

I also noticed this. But then thought the memory would be anyways zeroed 
(hole punched) before this call?

> 
>>> +	return ret;
>>> +}
>>> +
>>
>> ...
>>
>>> +
>>> +static struct file_system_type inaccessible_fs = {
>>> +	.owner		= THIS_MODULE,
>>> +	.name		= "[inaccessible]",
>>
>> Dunno where exactly is this name visible, but shouldn't it better be
>> "[memfd:inaccessible]"?
> 
> Maybe. And skip brackets.
> 
>>
>>> +	.init_fs_context = inaccessible_init_fs_context,
>>> +	.kill_sb	= kill_anon_super,
>>> +};
>>> +
>>
>