Received: by 2002:a05:6a10:f347:0:0:0:0 with SMTP id d7csp534171pxu; Thu, 7 Jan 2021 11:07:08 -0800 (PST) X-Google-Smtp-Source: ABdhPJypWsv5ZfEYa0kjxZkeI3yN3MZ92T+k4QNVBsc7wZ3HzDOuUQZLRZb1yn1a/Hq1t+T6yFdi X-Received: by 2002:a17:906:d81:: with SMTP id m1mr176918eji.550.1610046428188; Thu, 07 Jan 2021 11:07:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1610046428; cv=none; d=google.com; s=arc-20160816; b=VqtDwkwBTMkq6YkggEdcxa56YJlT9pQEUhZaj6uMN2cgcOEybc1xiu5kapbTdlwyBd e5ub4A4iKqsstSMDkGRxw6SwIfqeD4UAz8u5qGurV4U0SSLZZWDslnhxaLTbxE1fAVkl FUOBvJ2ci7WtoM1JW5ZqzvVUyPELU8/LYltIp63SKW7vfNNpD1sbfR+JaZspWcnx2T4s VIO35IggqZ3skqttzRLKqKhBOtL8wOk99XzgGeoB+zrLREaE9AIFwXZxH1E5gL89egEA lOZLPSN4xkAknHeYMeq/h6P6UB0gGFMJLI5eCaGiZR87sJMsin15o2yua1N1mFpflA/1 h9uA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:mime-version:message-id:date :sender:dkim-signature; bh=1Ks8/3nZJCYS90R8JKk2puoyafhaeYHGuGPtcF6mxek=; b=Ku5GBxMYEYaJ7M4Edcm+WBuELsgoZaB2fOCvCTbNnX0S5J+ydVVLOubKgwER0EFd15 HhV15Ze8RHW+Z5aa6Lancf7bSO+0mzbL9k1gckOU1Q5x9m6Eiu+5wetBNHamCwkUxDEL qetcvOjhKvnFcGOCkLsfYE9RhUIqzp5tX30j1hI5cxUT2Hm/muI7IYJk8Ig9AbEPliua lRzyCxNoa+87RoCXxtqIsr/rutbw8+4bK+e94vzmZNUeH6yhKxHbquumwnX8gPo47wKw +77XkQpQyPi62F3MXBXKZ1srUV5goIW6WZMfeV3oOmLJHlpVjOXVpRkBFyk8FgQM5OsK Hqwg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=s33gwiE6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id en9si2544632ejb.519.2021.01.07.11.06.43; Thu, 07 Jan 2021 11:07:08 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=s33gwiE6; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729182AbhAGTFq (ORCPT + 99 others); Thu, 7 Jan 2021 14:05:46 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43102 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729081AbhAGTFq (ORCPT ); Thu, 7 Jan 2021 14:05:46 -0500 Received: from mail-pf1-x449.google.com (mail-pf1-x449.google.com [IPv6:2607:f8b0:4864:20::449]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CF2AFC0612F6 for ; Thu, 7 Jan 2021 11:05:05 -0800 (PST) Received: by mail-pf1-x449.google.com with SMTP id x21so4776295pff.14 for ; Thu, 07 Jan 2021 11:05:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=sender:date:message-id:mime-version:subject:from:to:cc; bh=1Ks8/3nZJCYS90R8JKk2puoyafhaeYHGuGPtcF6mxek=; b=s33gwiE6wEj4kJwKpZ+JBEnAPq2Iul6GLks8POMY9MgZc9LBZEzjlEFhyqZYiwx4id dMUAawf6ZO1RuNGq6qWn7XiCwJZaExLSJ/x0qIdCrZcZ5lvZbgLpOXACTZgO7pmlu87x Zv/9mH/Oxu+THRCrCKk23deOoYVx+kIKAa5hSJt7UI49J1EJO2RC9KiHnXTjT/vb1SAb yhuOQHXkHuckcPE1Q+qkGIReZWdZZ1KhdiTyGY799YowVmEqWHJf722bg2qLRozEV0Kd vZm4JkDx7ZwLeiNeB2q5xb7tZVq7cjBJEKYOlzBvaCQRi8AAGDzniY+am3PWed00CrYr UxpQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:message-id:mime-version:subject:from :to:cc; bh=1Ks8/3nZJCYS90R8JKk2puoyafhaeYHGuGPtcF6mxek=; b=kh2EISQpqpgbPecVq61qo6bjh/xFnMCgDv09UPec9g8bThcJSRWNu8aI2D8IQbmEf7 IsiBSwiAOyrF/BSzIuhOmW8q9ESLD6QCL3ZQd4WGHN6dw4R2I2ckFB/g8vqCcHV+xxhM oTyYY0f6E200DaeBqo0dTp+7ufyrj5+Zn2FnvDWcU8zArbSXKblCVsyv3ip0XsWWox1T oQrr75eO4dbkbJiPgOz0dV8LpIBTmcM1gzupm0UF+9YWbZ5gQEx5cxU9OyR/zquQuU+J Lr6RAomEoXrEjTfEN01uOSbOPlU+F9F2hgmw6c1z/hlgkl9u45cn/QVhX50ZhDTerFYd 137A== X-Gm-Message-State: AOAM530BvvRygYCauAohq0gs/hxUTjMHXMEKoK8kFO4fqffsiJ3i4kzL WtfWA/bazYIP1Y4FBiVo1dOuEurNTSvsaCy3PEu7 Sender: "axelrasmussen via sendgmr" X-Received: from ajr0.svl.corp.google.com ([2620:15c:2cd:203:f693:9fff:feef:c8f8]) (user=axelrasmussen job=sendgmr) by 2002:a17:90a:c085:: with SMTP id o5mr10737346pjs.210.1610046305274; Thu, 07 Jan 2021 11:05:05 -0800 (PST) Date: Thu, 7 Jan 2021 11:04:51 -0800 Message-Id: <20210107190453.3051110-1-axelrasmussen@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.29.2.729.g45daf8777d-goog Subject: [RFC PATCH 0/2] userfaultfd: handle minor faults, add UFFDIO_CONTINUE From: Axel Rasmussen To: Alexander Viro , Alexey Dobriyan , Andrea Arcangeli , Andrew Morton , Anshuman Khandual , Catalin Marinas , Chinwen Chang , Huang Ying , Ingo Molnar , Jann Horn , Jerome Glisse , Lokesh Gidra , "Matthew Wilcox (Oracle)" , Michael Ellerman , "=?UTF-8?q?Michal=20Koutn=C3=BD?=" , Michel Lespinasse , Mike Kravetz , Mike Rapoport , Nicholas Piggin , Peter Xu , Shaohua Li , Shawn Anastasio , Steven Rostedt , Steven Price , Vlastimil Babka Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org, Adam Ruprecht , Axel Rasmussen , Cannon Matthews , "Dr . David Alan Gilbert" , David Rientjes , Oliver Upton Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Overview ======== This series adds a new userfaultfd registration mode, UFFDIO_REGISTER_MODE_MINOR. This allows userspace to intercept "minor" faults. By "minor" fault, I mean the following situation: Let there exist two mappings (i.e., VMAs) to the same page(s) (shared memory). One of the mappings is registered with userfaultfd (in minor mode), and the other is not. Via the non-UFFD mapping, the underlying pages have already been allocated & filled with some contents. The UFFD mapping has not yet been faulted in; when it is touched for the first time, this results in what I'm calling a "minor" fault. As a concrete example, when working with hugetlbfs, we have huge_pte_none(), but find_lock_page() finds an existing page. We also add a new ioctl to resolve such faults: UFFDIO_CONTINUE. The idea is, userspace resolves the fault by either a) doing nothing if the contents are already correct, or b) updating the underlying contents using the second, non-UFFD mapping (via memcpy/memset or similar, or something fancier like RDMA, or etc...). In either case, userspace issues UFFDIO_CONTINUE to tell the kernel "I have ensured the page contents are correct, carry on setting up the mapping". Use Case ======== Consider the use case of VM live migration (e.g. under QEMU/KVM): 1. While a VM is still running, we copy the contents of its memory to a target machine. The pages are populated on the target by writing to the non-UFFD mapping, using the setup described above. The VM is still running (and therefore its memory is likely changing), so this may be repeated several times, until we decide the target is "up to date enough". 2. We pause the VM on the source, and start executing on the target machine. During this gap, the VM's user(s) will *see* a pause, so it is desirable to minimize this window. 3. Between the last time any page was copied from the source to the target, and when the VM was paused, the contents of that page may have changed - and therefore the copy we have on the target machine is out of date. Although we can keep track of which pages are out of date, for VMs with large amounts of memory, it is "slow" to transfer this information to the target machine. We want to resume execution before such a transfer would complete. 4. So, the guest begins executing on the target machine. The first time it touches its memory (via the UFFD-registered mapping), userspace wants to intercept this fault. Userspace checks whether or not the page is up to date, and if not, copies the updated page from the source machine, via the non-UFFD mapping. Finally, whether a copy was performed or not, userspace issues a UFFDIO_CONTINUE ioctl to tell the kernel "I have ensured the page contents are correct, carry on setting up the mapping". We don't have to do all of the final updates on-demand. The userfaultfd manager can, in the background, also copy over updated pages once it receives the map of which pages are up-to-date or not. Interaction with Existing APIs ============================== Because it's possible to combine registration modes (e.g. a single VMA can be userfaultfd-registered MINOR | MISSING), and because it's up to userspace how to resolve faults once they are received, I spent some time thinking through how the existing API interacts with the new feature. UFFDIO_CONTINUE cannot be used to resolve non-minor faults, as it does not allocate a new page. If UFFDIO_CONTINUE is used on a non-minor fault: - For non-shared memory or shmem, -EINVAL is returned. - For hugetlb, -EFAULT is returned. UFFDIO_COPY and UFFDIO_ZEROPAGE cannot be used to resolve minor faults. Without modifications, the existing codepath assumes a new page needs to be allocated. This is okay, since userspace must have a second non-UFFD-registered mapping anyway, thus there isn't much reason to want to use these in any case (just memcpy or memset or similar). - If UFFDIO_COPY is used on a minor fault, -EEXIST is returned. - If UFFDIO_ZEROPAGE is used on a minor fault, -EEXIST is returned (or -EINVAL in the case of hugetlb, as UFFDIO_ZEROPAGE is unsupported in any case). - UFFDIO_WRITEPROTECT simply doesn't work with shared memory, and returns -ENOENT in that case (regardless of the kind of fault). Remaining Work ============== This patchset doesn't include updates to userfaultfd's documentation or selftests. This will be added before I send a non-RFC version of this series (I want to find out if there are strong objections to the API surface before spending the time to document it.) Currently the patchset only supports hugetlbfs. There is no reason it can't work with shmem, but I expect hugetlbfs to be much more commonly used since we're talking about backing guest memory for VMs. I plan to implement shmem support in a follow-up patch series. Axel Rasmussen (2): userfaultfd: add minor fault registration mode userfaultfd: add UFFDIO_CONTINUE ioctl fs/proc/task_mmu.c | 1 + fs/userfaultfd.c | 143 ++++++++++++++++++++++++------- include/linux/mm.h | 1 + include/linux/userfaultfd_k.h | 14 ++- include/trace/events/mmflags.h | 1 + include/uapi/linux/userfaultfd.h | 36 +++++++- mm/hugetlb.c | 42 +++++++-- mm/userfaultfd.c | 86 ++++++++++++++----- 8 files changed, 261 insertions(+), 63 deletions(-) -- 2.29.2.729.g45daf8777d-goog