by adding an int *locked parameter to get_user_pages() callers to this function
can now utilise VM_FAULT_RETRY functionality.
Taken in conjunction with the patch series adding the same parameter to
get_user_pages_remote() this means all slow-path get_user_pages*() functions
will now have the ability to utilise VM_FAULT_RETRY.
Additionally get_user_pages() and get_user_pages_remote() previously mirrored
one another in functionality differing only in the ability to specify task/mm,
this patch series reinstates this relationship.
This patch series should not introduce any functional changes.
Lorenzo Stoakes (2):
mm: add locked parameter to get_user_pages()
mm: remove get_user_pages_locked()
arch/cris/arch-v32/drivers/cryptocop.c | 2 +
arch/ia64/kernel/err_inject.c | 2 +-
arch/x86/mm/mpx.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 2 +-
drivers/gpu/drm/radeon/radeon_ttm.c | 2 +-
drivers/gpu/drm/via/via_dmablit.c | 2 +-
drivers/infiniband/core/umem.c | 2 +-
drivers/infiniband/hw/mthca/mthca_memfree.c | 3 +-
drivers/infiniband/hw/qib/qib_user_pages.c | 2 +-
drivers/infiniband/hw/usnic/usnic_uiom.c | 2 +-
drivers/media/v4l2-core/videobuf-dma-sg.c | 2 +-
drivers/misc/mic/scif/scif_rma.c | 1 +
drivers/misc/sgi-gru/grufault.c | 3 +-
drivers/platform/goldfish/goldfish_pipe.c | 2 +-
drivers/rapidio/devices/rio_mport_cdev.c | 2 +-
.../interface/vchiq_arm/vchiq_2835_arm.c | 3 +-
.../vc04_services/interface/vchiq_arm/vchiq_arm.c | 3 +-
drivers/virt/fsl_hypervisor.c | 2 +-
include/linux/mm.h | 4 +-
mm/frame_vector.c | 4 +-
mm/gup.c | 62 ++++++++--------------
mm/ksm.c | 3 +-
mm/mempolicy.c | 2 +-
mm/nommu.c | 10 +---
virt/kvm/kvm_main.c | 4 +-
25 files changed, 55 insertions(+), 73 deletions(-)
get_user_pages() now has an int *locked parameter which renders
get_user_pages_locked() redundant, so remove it.
This patch should not introduce any functional changes.
Signed-off-by: Lorenzo Stoakes <[email protected]>
---
include/linux/mm.h | 2 --
mm/frame_vector.c | 4 ++--
mm/gup.c | 56 +++++++++++++++++++-----------------------------------
mm/nommu.c | 8 --------
4 files changed, 22 insertions(+), 48 deletions(-)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 396984e..4db3147 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1278,8 +1278,6 @@ long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm,
long get_user_pages(unsigned long start, unsigned long nr_pages,
unsigned int gup_flags, struct page **pages,
struct vm_area_struct **vmas, int *locked);
-long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
- unsigned int gup_flags, struct page **pages, int *locked);
long __get_user_pages_unlocked(struct task_struct *tsk, struct mm_struct *mm,
unsigned long start, unsigned long nr_pages,
struct page **pages, unsigned int gup_flags);
diff --git a/mm/frame_vector.c b/mm/frame_vector.c
index db77dcb..c5ce1b1 100644
--- a/mm/frame_vector.c
+++ b/mm/frame_vector.c
@@ -55,8 +55,8 @@ int get_vaddr_frames(unsigned long start, unsigned int nr_frames,
if (!(vma->vm_flags & (VM_IO | VM_PFNMAP))) {
vec->got_ref = true;
vec->is_pfns = false;
- ret = get_user_pages_locked(start, nr_frames,
- gup_flags, (struct page **)(vec->ptrs), &locked);
+ ret = get_user_pages(start, nr_frames,
+ gup_flags, (struct page **)(vec->ptrs), NULL, &locked);
goto out;
}
diff --git a/mm/gup.c b/mm/gup.c
index 6b5539e..6cec693 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -826,37 +826,6 @@ static __always_inline long __get_user_pages_locked(struct task_struct *tsk,
}
/*
- * We can leverage the VM_FAULT_RETRY functionality in the page fault
- * paths better by using either get_user_pages_locked() or
- * get_user_pages_unlocked().
- *
- * get_user_pages_locked() is suitable to replace the form:
- *
- * down_read(&mm->mmap_sem);
- * do_something()
- * get_user_pages(tsk, mm, ..., pages, NULL, NULL);
- * up_read(&mm->mmap_sem);
- *
- * to:
- *
- * int locked = 1;
- * down_read(&mm->mmap_sem);
- * do_something()
- * get_user_pages_locked(tsk, mm, ..., pages, &locked);
- * if (locked)
- * up_read(&mm->mmap_sem);
- */
-long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
- unsigned int gup_flags, struct page **pages,
- int *locked)
-{
- return __get_user_pages_locked(current, current->mm, start, nr_pages,
- pages, NULL, locked, true,
- gup_flags | FOLL_TOUCH);
-}
-EXPORT_SYMBOL(get_user_pages_locked);
-
-/*
* Same as get_user_pages_unlocked(...., FOLL_TOUCH) but it allows to
* pass additional gup_flags as last parameter (like FOLL_HWPOISON).
*
@@ -954,11 +923,6 @@ EXPORT_SYMBOL(get_user_pages_unlocked);
* use the correct cache flushing APIs.
*
* See also get_user_pages_fast, for performance critical applications.
- *
- * get_user_pages should be phased out in favor of
- * get_user_pages_locked|unlocked or get_user_pages_fast. Nothing
- * should use get_user_pages because it cannot pass
- * FAULT_FLAG_ALLOW_RETRY to handle_mm_fault.
*/
long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm,
unsigned long start, unsigned long nr_pages,
@@ -976,6 +940,26 @@ EXPORT_SYMBOL(get_user_pages_remote);
* less-flexible calling convention where we assume that the task
* and mm being operated on are the current task's. We also
* obviously don't pass FOLL_REMOTE in here.
+ *
+ * We can leverage the VM_FAULT_RETRY functionality in the page fault
+ * paths better by using either get_user_pages()'s locked parameter or
+ * get_user_pages_unlocked().
+ *
+ * get_user_pages()'s locked parameter is suitable to replace the form:
+ *
+ * down_read(&mm->mmap_sem);
+ * do_something()
+ * get_user_pages(tsk, mm, ..., pages, NULL, NULL);
+ * up_read(&mm->mmap_sem);
+ *
+ * to:
+ *
+ * int locked = 1;
+ * down_read(&mm->mmap_sem);
+ * do_something()
+ * get_user_pages(tsk, mm, ..., pages, NULL, &locked);
+ * if (locked)
+ * up_read(&mm->mmap_sem);
*/
long get_user_pages(unsigned long start, unsigned long nr_pages,
unsigned int gup_flags, struct page **pages,
diff --git a/mm/nommu.c b/mm/nommu.c
index 82aaa33..3d38d40 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -168,14 +168,6 @@ long get_user_pages(unsigned long start, unsigned long nr_pages,
}
EXPORT_SYMBOL(get_user_pages);
-long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
- unsigned int gup_flags, struct page **pages,
- int *locked)
-{
- return get_user_pages(start, nr_pages, gup_flags, pages, NULL, NULL);
-}
-EXPORT_SYMBOL(get_user_pages_locked);
-
long __get_user_pages_unlocked(struct task_struct *tsk, struct mm_struct *mm,
unsigned long start, unsigned long nr_pages,
struct page **pages, unsigned int gup_flags)
--
2.10.1
This patch adds an int *locked parameter to get_user_pages() to allow
VM_FAULT_RETRY faulting behaviour similar to get_user_pages_[un]locked().
It additionally clears the way for get_user_pages_locked() to be removed as its
sole remaining useful characteristic was to allow for VM_FAULT_RETRY behaviour
when faulting in pages.
It should not introduce any functional changes, however it does allow for
subsequent changes to get_user_pages() callers to take advantage of
VM_FAULT_RETRY.
Signed-off-by: Lorenzo Stoakes <[email protected]>
---
arch/cris/arch-v32/drivers/cryptocop.c | 2 ++
arch/ia64/kernel/err_inject.c | 2 +-
arch/x86/mm/mpx.c | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 2 +-
drivers/gpu/drm/radeon/radeon_ttm.c | 2 +-
drivers/gpu/drm/via/via_dmablit.c | 2 +-
drivers/infiniband/core/umem.c | 2 +-
drivers/infiniband/hw/mthca/mthca_memfree.c | 3 ++-
drivers/infiniband/hw/qib/qib_user_pages.c | 2 +-
drivers/infiniband/hw/usnic/usnic_uiom.c | 2 +-
drivers/media/v4l2-core/videobuf-dma-sg.c | 2 +-
drivers/misc/mic/scif/scif_rma.c | 1 +
drivers/misc/sgi-gru/grufault.c | 3 ++-
drivers/platform/goldfish/goldfish_pipe.c | 2 +-
drivers/rapidio/devices/rio_mport_cdev.c | 2 +-
.../staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c | 3 ++-
drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c | 3 ++-
drivers/virt/fsl_hypervisor.c | 2 +-
include/linux/mm.h | 2 +-
mm/gup.c | 8 ++++----
mm/ksm.c | 3 ++-
mm/mempolicy.c | 2 +-
mm/nommu.c | 4 ++--
virt/kvm/kvm_main.c | 4 ++--
24 files changed, 35 insertions(+), 27 deletions(-)
diff --git a/arch/cris/arch-v32/drivers/cryptocop.c b/arch/cris/arch-v32/drivers/cryptocop.c
index 0068fd4..7444b45 100644
--- a/arch/cris/arch-v32/drivers/cryptocop.c
+++ b/arch/cris/arch-v32/drivers/cryptocop.c
@@ -2723,6 +2723,7 @@ static int cryptocop_ioctl_process(struct inode *inode, struct file *filp, unsig
noinpages,
0, /* read access only for in data */
inpages,
+ NULL,
NULL);
if (err < 0) {
@@ -2737,6 +2738,7 @@ static int cryptocop_ioctl_process(struct inode *inode, struct file *filp, unsig
nooutpages,
FOLL_WRITE, /* write access for out data */
outpages,
+ NULL,
NULL);
up_read(¤t->mm->mmap_sem);
if (err < 0) {
diff --git a/arch/ia64/kernel/err_inject.c b/arch/ia64/kernel/err_inject.c
index 5ed0ea9..99f3fa2 100644
--- a/arch/ia64/kernel/err_inject.c
+++ b/arch/ia64/kernel/err_inject.c
@@ -142,7 +142,7 @@ store_virtual_to_phys(struct device *dev, struct device_attribute *attr,
u64 virt_addr=simple_strtoull(buf, NULL, 16);
int ret;
- ret = get_user_pages(virt_addr, 1, FOLL_WRITE, NULL, NULL);
+ ret = get_user_pages(virt_addr, 1, FOLL_WRITE, NULL, NULL, NULL);
if (ret<=0) {
#ifdef ERR_INJ_DEBUG
printk("Virtual address %lx is not existing.\n",virt_addr);
diff --git a/arch/x86/mm/mpx.c b/arch/x86/mm/mpx.c
index e4f8009..b74a7b2 100644
--- a/arch/x86/mm/mpx.c
+++ b/arch/x86/mm/mpx.c
@@ -546,7 +546,7 @@ static int mpx_resolve_fault(long __user *addr, int write)
int nr_pages = 1;
gup_ret = get_user_pages((unsigned long)addr, nr_pages,
- write ? FOLL_WRITE : 0, NULL, NULL);
+ write ? FOLL_WRITE : 0, NULL, NULL, NULL);
/*
* get_user_pages() returns number of pages gotten.
* 0 means we failed to fault in and get anything,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index dcaf691..3d9a195 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -584,7 +584,7 @@ int amdgpu_ttm_tt_get_user_pages(struct ttm_tt *ttm, struct page **pages)
list_add(&guptask.list, >t->guptasks);
spin_unlock(>t->guptasklock);
- r = get_user_pages(userptr, num_pages, flags, p, NULL);
+ r = get_user_pages(userptr, num_pages, flags, p, NULL, NULL);
spin_lock(>t->guptasklock);
list_del(&guptask.list);
diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c
index 3de5e6e..8b0c069 100644
--- a/drivers/gpu/drm/radeon/radeon_ttm.c
+++ b/drivers/gpu/drm/radeon/radeon_ttm.c
@@ -567,7 +567,7 @@ static int radeon_ttm_tt_pin_userptr(struct ttm_tt *ttm)
struct page **pages = ttm->pages + pinned;
r = get_user_pages(userptr, num_pages, write ? FOLL_WRITE : 0,
- pages, NULL);
+ pages, NULL, NULL);
if (r < 0)
goto release_pages;
diff --git a/drivers/gpu/drm/via/via_dmablit.c b/drivers/gpu/drm/via/via_dmablit.c
index 1a3ad76..39877fe 100644
--- a/drivers/gpu/drm/via/via_dmablit.c
+++ b/drivers/gpu/drm/via/via_dmablit.c
@@ -242,7 +242,7 @@ via_lock_all_dma_pages(drm_via_sg_info_t *vsg, drm_via_dmablit_t *xfer)
ret = get_user_pages((unsigned long)xfer->mem_addr,
vsg->num_pages,
(vsg->direction == DMA_FROM_DEVICE) ? FOLL_WRITE : 0,
- vsg->pages, NULL);
+ vsg->pages, NULL, NULL);
up_read(¤t->mm->mmap_sem);
if (ret != vsg->num_pages) {
diff --git a/drivers/infiniband/core/umem.c b/drivers/infiniband/core/umem.c
index 224ad27..1d5278f 100644
--- a/drivers/infiniband/core/umem.c
+++ b/drivers/infiniband/core/umem.c
@@ -194,7 +194,7 @@ struct ib_umem *ib_umem_get(struct ib_ucontext *context, unsigned long addr,
ret = get_user_pages(cur_base,
min_t(unsigned long, npages,
PAGE_SIZE / sizeof (struct page *)),
- gup_flags, page_list, vma_list);
+ gup_flags, page_list, vma_list, NULL);
if (ret < 0)
goto out;
diff --git a/drivers/infiniband/hw/mthca/mthca_memfree.c b/drivers/infiniband/hw/mthca/mthca_memfree.c
index c6fe89d..45d037f 100644
--- a/drivers/infiniband/hw/mthca/mthca_memfree.c
+++ b/drivers/infiniband/hw/mthca/mthca_memfree.c
@@ -472,7 +472,8 @@ int mthca_map_user_db(struct mthca_dev *dev, struct mthca_uar *uar,
goto out;
}
- ret = get_user_pages(uaddr & PAGE_MASK, 1, FOLL_WRITE, pages, NULL);
+ ret = get_user_pages(uaddr & PAGE_MASK, 1, FOLL_WRITE, pages, NULL,
+ NULL);
if (ret < 0)
goto out;
diff --git a/drivers/infiniband/hw/qib/qib_user_pages.c b/drivers/infiniband/hw/qib/qib_user_pages.c
index 75f0862..433dc3a 100644
--- a/drivers/infiniband/hw/qib/qib_user_pages.c
+++ b/drivers/infiniband/hw/qib/qib_user_pages.c
@@ -69,7 +69,7 @@ static int __qib_get_user_pages(unsigned long start_page, size_t num_pages,
ret = get_user_pages(start_page + got * PAGE_SIZE,
num_pages - got,
FOLL_WRITE | FOLL_FORCE,
- p + got, NULL);
+ p + got, NULL, NULL);
if (ret < 0)
goto bail_release;
}
diff --git a/drivers/infiniband/hw/usnic/usnic_uiom.c b/drivers/infiniband/hw/usnic/usnic_uiom.c
index 1ccee6e..68d583f 100644
--- a/drivers/infiniband/hw/usnic/usnic_uiom.c
+++ b/drivers/infiniband/hw/usnic/usnic_uiom.c
@@ -145,7 +145,7 @@ static int usnic_uiom_get_pages(unsigned long addr, size_t size, int writable,
ret = get_user_pages(cur_base,
min_t(unsigned long, npages,
PAGE_SIZE / sizeof(struct page *)),
- gup_flags, page_list, NULL);
+ gup_flags, page_list, NULL, NULL);
if (ret < 0)
goto out;
diff --git a/drivers/media/v4l2-core/videobuf-dma-sg.c b/drivers/media/v4l2-core/videobuf-dma-sg.c
index 1db0af6..66c24e6 100644
--- a/drivers/media/v4l2-core/videobuf-dma-sg.c
+++ b/drivers/media/v4l2-core/videobuf-dma-sg.c
@@ -186,7 +186,7 @@ static int videobuf_dma_init_user_locked(struct videobuf_dmabuf *dma,
data, size, dma->nr_pages);
err = get_user_pages(data & PAGE_MASK, dma->nr_pages,
- flags, dma->pages, NULL);
+ flags, dma->pages, NULL, NULL);
if (err != dma->nr_pages) {
dma->nr_pages = (err >= 0) ? err : 0;
diff --git a/drivers/misc/mic/scif/scif_rma.c b/drivers/misc/mic/scif/scif_rma.c
index f806a44..6719896 100644
--- a/drivers/misc/mic/scif/scif_rma.c
+++ b/drivers/misc/mic/scif/scif_rma.c
@@ -1398,6 +1398,7 @@ int __scif_pin_pages(void *addr, size_t len, int *out_prot,
nr_pages,
(prot & SCIF_PROT_WRITE) ? FOLL_WRITE : 0,
pinned_pages->pages,
+ NULL,
NULL);
up_write(&mm->mmap_sem);
if (nr_pages != pinned_pages->nr_pages) {
diff --git a/drivers/misc/sgi-gru/grufault.c b/drivers/misc/sgi-gru/grufault.c
index 6fb773d..c6bda94 100644
--- a/drivers/misc/sgi-gru/grufault.c
+++ b/drivers/misc/sgi-gru/grufault.c
@@ -198,7 +198,8 @@ static int non_atomic_pte_lookup(struct vm_area_struct *vma,
#else
*pageshift = PAGE_SHIFT;
#endif
- if (get_user_pages(vaddr, 1, write ? FOLL_WRITE : 0, &page, NULL) <= 0)
+ if (get_user_pages(vaddr, 1, write ? FOLL_WRITE : 0, &page, NULL, NULL)
+ <= 0)
return -EFAULT;
*paddr = page_to_phys(page);
put_page(page);
diff --git a/drivers/platform/goldfish/goldfish_pipe.c b/drivers/platform/goldfish/goldfish_pipe.c
index 1aba2c7..bb5c0fe 100644
--- a/drivers/platform/goldfish/goldfish_pipe.c
+++ b/drivers/platform/goldfish/goldfish_pipe.c
@@ -310,7 +310,7 @@ static ssize_t goldfish_pipe_read_write(struct file *filp, char __user *buffer,
*/
down_read(¤t->mm->mmap_sem);
ret = get_user_pages(address, 1, is_write ? 0 : FOLL_WRITE,
- &page, NULL);
+ &page, NULL, NULL);
up_read(¤t->mm->mmap_sem);
if (ret < 0)
break;
diff --git a/drivers/rapidio/devices/rio_mport_cdev.c b/drivers/rapidio/devices/rio_mport_cdev.c
index 9013a58..0e507ca 100644
--- a/drivers/rapidio/devices/rio_mport_cdev.c
+++ b/drivers/rapidio/devices/rio_mport_cdev.c
@@ -894,7 +894,7 @@ rio_dma_transfer(struct file *filp, u32 transfer_mode,
(unsigned long)xfer->loc_addr & PAGE_MASK,
nr_pages,
dir == DMA_FROM_DEVICE ? FOLL_WRITE : 0,
- page_list, NULL);
+ page_list, NULL, NULL);
up_read(¤t->mm->mmap_sem);
if (pinned != nr_pages) {
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
index 1091b9f..f3f0bab 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_2835_arm.c
@@ -425,7 +425,8 @@ create_pagelist(char __user *buf, size_t count, unsigned short type,
num_pages,
(type == PAGELIST_READ) ? FOLL_WRITE : 0,
pages,
- NULL /*vmas */);
+ NULL /*vmas */,
+ NULL /* locked */);
up_read(&task->mm->mmap_sem);
if (actual_pages != num_pages) {
diff --git a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
index 7b6cd4d..86828d9 100644
--- a/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
+++ b/drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
@@ -1479,7 +1479,8 @@ dump_phys_mem(void *virt_addr, uint32_t num_bytes)
num_pages, /* len */
0, /* gup_flags */
pages, /* pages (array of page pointers) */
- NULL); /* vmas */
+ NULL, /* vmas */
+ NULL); /* locked */
up_read(¤t->mm->mmap_sem);
prev_idx = -1;
diff --git a/drivers/virt/fsl_hypervisor.c b/drivers/virt/fsl_hypervisor.c
index 150ce2a..896bc31 100644
--- a/drivers/virt/fsl_hypervisor.c
+++ b/drivers/virt/fsl_hypervisor.c
@@ -246,7 +246,7 @@ static long ioctl_memcpy(struct fsl_hv_ioctl_memcpy __user *p)
down_read(¤t->mm->mmap_sem);
num_pinned = get_user_pages(param.local_vaddr - lb_offset,
num_pages, (param.source == -1) ? 0 : FOLL_WRITE,
- pages, NULL);
+ pages, NULL, NULL);
up_read(¤t->mm->mmap_sem);
if (num_pinned != num_pages) {
diff --git a/include/linux/mm.h b/include/linux/mm.h
index a92c8d7..396984e 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1277,7 +1277,7 @@ long get_user_pages_remote(struct task_struct *tsk, struct mm_struct *mm,
struct vm_area_struct **vmas);
long get_user_pages(unsigned long start, unsigned long nr_pages,
unsigned int gup_flags, struct page **pages,
- struct vm_area_struct **vmas);
+ struct vm_area_struct **vmas, int *locked);
long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
unsigned int gup_flags, struct page **pages, int *locked);
long __get_user_pages_unlocked(struct task_struct *tsk, struct mm_struct *mm,
diff --git a/mm/gup.c b/mm/gup.c
index ec4f827..6b5539e 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -834,7 +834,7 @@ static __always_inline long __get_user_pages_locked(struct task_struct *tsk,
*
* down_read(&mm->mmap_sem);
* do_something()
- * get_user_pages(tsk, mm, ..., pages, NULL);
+ * get_user_pages(tsk, mm, ..., pages, NULL, NULL);
* up_read(&mm->mmap_sem);
*
* to:
@@ -886,7 +886,7 @@ EXPORT_SYMBOL(__get_user_pages_unlocked);
* get_user_pages_unlocked() is suitable to replace the form:
*
* down_read(&mm->mmap_sem);
- * get_user_pages(tsk, mm, ..., pages, NULL);
+ * get_user_pages(tsk, mm, ..., pages, NULL, NULL);
* up_read(&mm->mmap_sem);
*
* with:
@@ -979,10 +979,10 @@ EXPORT_SYMBOL(get_user_pages_remote);
*/
long get_user_pages(unsigned long start, unsigned long nr_pages,
unsigned int gup_flags, struct page **pages,
- struct vm_area_struct **vmas)
+ struct vm_area_struct **vmas, int *locked)
{
return __get_user_pages_locked(current, current->mm, start, nr_pages,
- pages, vmas, NULL, false,
+ pages, vmas, locked, true,
gup_flags | FOLL_TOUCH);
}
EXPORT_SYMBOL(get_user_pages);
diff --git a/mm/ksm.c b/mm/ksm.c
index 9ae6011..b7468dd 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -358,7 +358,8 @@ static inline bool ksm_test_exit(struct mm_struct *mm)
/*
* We use break_ksm to break COW on a ksm page: it's a stripped down
*
- * if (get_user_pages(addr, 1, 1, 1, &page, NULL) == 1)
+ * if (get_user_pages(addr, 1, FOLL_WRITE | FOLL_FORCE, &page,
+ * NULL, NULL) == 1)
* put_page(page);
*
* but taking great care only to touch a ksm page, in a VM_MERGEABLE vma,
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 0b859af..4be72c7 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -850,7 +850,7 @@ static int lookup_node(unsigned long addr)
struct page *p;
int err;
- err = get_user_pages(addr & PAGE_MASK, 1, 0, &p, NULL);
+ err = get_user_pages(addr & PAGE_MASK, 1, 0, &p, NULL, NULL);
if (err >= 0) {
err = page_to_nid(p);
put_page(p);
diff --git a/mm/nommu.c b/mm/nommu.c
index 8b8faaf..82aaa33 100644
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -161,7 +161,7 @@ static long __get_user_pages(struct task_struct *tsk, struct mm_struct *mm,
*/
long get_user_pages(unsigned long start, unsigned long nr_pages,
unsigned int gup_flags, struct page **pages,
- struct vm_area_struct **vmas)
+ struct vm_area_struct **vmas, int *locked)
{
return __get_user_pages(current, current->mm, start, nr_pages,
gup_flags, pages, vmas, NULL);
@@ -172,7 +172,7 @@ long get_user_pages_locked(unsigned long start, unsigned long nr_pages,
unsigned int gup_flags, struct page **pages,
int *locked)
{
- return get_user_pages(start, nr_pages, gup_flags, pages, NULL);
+ return get_user_pages(start, nr_pages, gup_flags, pages, NULL, NULL);
}
EXPORT_SYMBOL(get_user_pages_locked);
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 2907b7b..353bcb2 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -1351,14 +1351,14 @@ static int get_user_page_nowait(unsigned long start, int write,
if (write)
flags |= FOLL_WRITE;
- return get_user_pages(start, 1, flags, page, NULL);
+ return get_user_pages(start, 1, flags, page, NULL, NULL);
}
static inline int check_user_page_hwpoison(unsigned long addr)
{
int rc, flags = FOLL_HWPOISON | FOLL_WRITE;
- rc = get_user_pages(addr, 1, flags, NULL, NULL);
+ rc = get_user_pages(addr, 1, flags, NULL, NULL, NULL);
return rc == -EHWPOISON;
}
--
2.10.1
On 31/10/2016 11:02, Lorenzo Stoakes wrote:
> - *
> - * get_user_pages should be phased out in favor of
> - * get_user_pages_locked|unlocked or get_user_pages_fast. Nothing
> - * should use get_user_pages because it cannot pass
> - * FAULT_FLAG_ALLOW_RETRY to handle_mm_fault.
This comment should be preserved in some way. In addition, removing
get_user_pages_locked() makes it harder (compared to a simple "git grep
-w") to identify callers that lack allow-retry functionality). So I'm
not sure about the benefits of these patches.
If all callers were changed, then sure removing the _locked suffix would
be a good idea.
Paolo
On Mon, Oct 31, 2016 at 12:45:36PM +0100, Paolo Bonzini wrote:
>
>
> On 31/10/2016 11:02, Lorenzo Stoakes wrote:
> > - *
> > - * get_user_pages should be phased out in favor of
> > - * get_user_pages_locked|unlocked or get_user_pages_fast. Nothing
> > - * should use get_user_pages because it cannot pass
> > - * FAULT_FLAG_ALLOW_RETRY to handle_mm_fault.
>
> This comment should be preserved in some way. In addition, removing
Could you clarify what you think should be retained?
The comment seems to me to be largely rendered redundant by this change -
get_user_pages() now offers identical behaviour, and of course the latter part
of the comment ('because it cannot pass FAULT_FLAG_ALLOW_RETRY') is rendered
incorrect by this change.
It could be replaced with a recommendation to make use of VM_FAULT_RETRY logic
if possible. Note that the line above retains the reference to
'get_user_pages_fast()' - 'See also get_user_pages_fast, for performance
critical applications.'
One important thing to note here is this is a comment on get_user_pages_remote()
for which it was already incorrect syntactically (I'm guessing once
get_user_pages_remote() was added it 'stole' get_user_pages()'s comment :)
> get_user_pages_locked() makes it harder (compared to a simple "git grep
> -w") to identify callers that lack allow-retry functionality). So I'm
> not sure about the benefits of these patches.
>
That's a fair point, and certainly this cleanup series is less obviously helpful
than previous ones.
However, there are a few points in its favour:
1. get_user_pages_remote() was the mirror of get_user_pages() prior to adding an
int *locked parameter to the former (necessary to allow for the unexport of
__get_user_pages_unlocked()), differing only in task/mm being specified and
FOLL_REMOTE being set. This patch series keeps these functions 'synchronised'
in this respect.
2. There is currently only one caller of get_user_pages_locked() in
mm/frame_vector.c which seems to suggest this function isn't widely
used/known.
3. This change results in all slow-path get_user_pages*() functions having the
ability to use VM_FAULT_RETRY logic rather than 'defaulting' to using
get_user_pages() that doesn't let you do this even if you wanted to.
4. It's somewhat confusing/redundant having both get_user_pages_locked() and
get_user_pages() functions which both require mmap_sem to be held (i.e. both
are 'locked' versions.)
> If all callers were changed, then sure removing the _locked suffix would
> be a good idea.
It seems many callers cannot release mmap_sem so VM_FAULT_RETRY logic couldn't
happen anyway in this cases and in these cases we couldn't change the caller.
Overall, an alternative here might be to remove get_user_pages() and update
get_user_pages_locked() to add a 'vmas' parameter, however doing this renders
get_user_pages_unlocked() asymmetric as it would lack an vmas parameter (adding
one there would make no sense as VM_FAULT_RETRY logic invalidates VMAs) though
perhaps not such a big issue as it makes sense as to why this is the case.
get_user_pages_unlocked() definitely seems to be a useful helper and therefore
makes sense to retain.
Of course another alternative is to leave things be :)
On 31/10/2016 14:48, Lorenzo Stoakes wrote:
> On Mon, Oct 31, 2016 at 12:45:36PM +0100, Paolo Bonzini wrote:
>>
>>
>> On 31/10/2016 11:02, Lorenzo Stoakes wrote:
>>> - *
>>> - * get_user_pages should be phased out in favor of
>>> - * get_user_pages_locked|unlocked or get_user_pages_fast. Nothing
>>> - * should use get_user_pages because it cannot pass
>>> - * FAULT_FLAG_ALLOW_RETRY to handle_mm_fault.
>>
>> This comment should be preserved in some way. In addition, removing
>
> Could you clarify what you think should be retained?
>
> The comment seems to me to be largely rendered redundant by this change -
> get_user_pages() now offers identical behaviour, and of course the latter part
> of the comment ('because it cannot pass FAULT_FLAG_ALLOW_RETRY') is rendered
> incorrect by this change.
>
> It could be replaced with a recommendation to make use of VM_FAULT_RETRY logic
> if possible.
Yes, exactly. locked == NULL should be avoided whenever mmap_sem can
be dropped, and the comment indirectly said so. Now most of those cases
actually are those where you can just call get_user_pages_unlocked.
>> get_user_pages_locked() makes it harder (compared to a simple "git grep
>> -w") to identify callers that lack allow-retry functionality). So I'm
>> not sure about the benefits of these patches.
>
> That's a fair point, and certainly this cleanup series is less obviously helpful
> than previous ones.
>
> However, there are a few points in its favour:
>
> 1. get_user_pages_remote() was the mirror of get_user_pages() prior to adding an
> int *locked parameter to the former (necessary to allow for the unexport of
> __get_user_pages_unlocked()), differing only in task/mm being specified and
> FOLL_REMOTE being set. This patch series keeps these functions 'synchronised'
> in this respect.
>
> 2. There is currently only one caller of get_user_pages_locked() in
> mm/frame_vector.c which seems to suggest this function isn't widely
> used/known.
Or not widely necessary. :)
> 3. This change results in all slow-path get_user_pages*() functions having the
> ability to use VM_FAULT_RETRY logic rather than 'defaulting' to using
> get_user_pages() that doesn't let you do this even if you wanted to.
This is only true if someone does the work though. From a quick look
at your series, the following files can be trivially changed to use
get_user_pages_unlocked:
- drivers/gpu/drm/via/via_dmablit.c
- drivers/platform/goldfish/goldfish_pipe.c
- drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
- drivers/rapidio/devices/rio_mport_cdev.c
- drivers/virt/fsl_hypervisor.c
Also, videobuf_dma_init_user could be changed to use retry by adding a
*locked argument to videobuf_dma_init_user_locked, prototype patch
after my signature.
Everything else is probably best kept using get_user_pages.
> 4. It's somewhat confusing/redundant having both get_user_pages_locked() and
> get_user_pages() functions which both require mmap_sem to be held (i.e. both
> are 'locked' versions.)
>
>> If all callers were changed, then sure removing the _locked suffix would
>> be a good idea.
>
> It seems many callers cannot release mmap_sem so VM_FAULT_RETRY logic couldn't
> happen anyway in this cases and in these cases we couldn't change the caller.
But then get_user_pages_locked remains a less-common case, so perhaps
it's a good thing to give it a longer name!
> Overall, an alternative here might be to remove get_user_pages() and update
> get_user_pages_locked() to add a 'vmas' parameter, however doing this renders
> get_user_pages_unlocked() asymmetric as it would lack an vmas parameter (adding
> one there would make no sense as VM_FAULT_RETRY logic invalidates VMAs) though
> perhaps not such a big issue as it makes sense as to why this is the case.
Adding the 'vmas' parameter to get_user_pages_locked would make little
sense. Since VM_FAULT_RETRY invalidates it and g_u_p_locked can and
does retry, it would generally not be useful.
So I think we have the right API now:
- do not have lock? Use get_user_pages_unlocked, get retry for free,
no need to handle mmap_sem and the locked argument; cannot get back vmas.
- have and cannot drop lock? User get_user_pages, no need to pass NULL
for the locked argument; can get back vmas.
- have but can drop lock (rare case)? Use get_user_pages_locked,
cannot get back vams.
Paolo
> get_user_pages_unlocked() definitely seems to be a useful helper and therefore
> makes sense to retain.
> Of course another alternative is to leave things be :)
>
diff --git a/drivers/media/v4l2-core/videobuf-dma-sg.c b/drivers/media/v4l2-core/videobuf-dma-sg.c
index 1db0af6c7f94..dae4eb8e9d5b 100644
--- a/drivers/media/v4l2-core/videobuf-dma-sg.c
+++ b/drivers/media/v4l2-core/videobuf-dma-sg.c
@@ -152,7 +152,8 @@ static void videobuf_dma_init(struct videobuf_dmabuf *dma)
}
static int videobuf_dma_init_user_locked(struct videobuf_dmabuf *dma,
- int direction, unsigned long data, unsigned long size)
+ int direction, unsigned long data, unsigned long size,
+ int *locked)
{
unsigned long first, last;
int err, rw = 0;
@@ -185,8 +186,17 @@ static int videobuf_dma_init_user_locked(struct videobuf_dmabuf *dma,
dprintk(1, "init user [0x%lx+0x%lx => %d pages]\n",
data, size, dma->nr_pages);
- err = get_user_pages(data & PAGE_MASK, dma->nr_pages,
- flags, dma->pages, NULL);
+ if (locked && !*locked) {
+ down_read(¤t->mm->mmap_sem);
+ *locked = 1;
+ }
+
+ /*
+ * If the caller cannot have mmap_sem dropped, it passes locked == NULL
+ * so get_user_pages_locked will not release it.
+ */
+ err = get_user_pages_locked(data & PAGE_MASK, dma->nr_pages,
+ flags, dma->pages, locked);
if (err != dma->nr_pages) {
dma->nr_pages = (err >= 0) ? err : 0;
@@ -200,10 +210,11 @@ static int videobuf_dma_init_user(struct videobuf_dmabuf *dma, int direction,
unsigned long data, unsigned long size)
{
int ret;
+ int locked = 0;
- down_read(¤t->mm->mmap_sem);
- ret = videobuf_dma_init_user_locked(dma, direction, data, size);
- up_read(¤t->mm->mmap_sem);
+ ret = videobuf_dma_init_user_locked(dma, direction, data, size, &locked);
+ if (locked)
+ up_read(¤t->mm->mmap_sem);
return ret;
}
@@ -540,7 +551,7 @@ static int __videobuf_iolock(struct videobuf_queue *q,
err = videobuf_dma_init_user_locked(&mem->dma,
DMA_FROM_DEVICE,
- vb->baddr, vb->bsize);
+ vb->baddr, vb->bsize, NULL);
if (0 != err)
return err;
}
On Mon, Oct 31, 2016 at 06:55:33PM +0100, Paolo Bonzini wrote:
> > 2. There is currently only one caller of get_user_pages_locked() in
> > mm/frame_vector.c which seems to suggest this function isn't widely
> > used/known.
>
> Or not widely necessary. :)
Well, quite :)
>
> > 3. This change results in all slow-path get_user_pages*() functions having the
> > ability to use VM_FAULT_RETRY logic rather than 'defaulting' to using
> > get_user_pages() that doesn't let you do this even if you wanted to.
>
> This is only true if someone does the work though. From a quick look
> at your series, the following files can be trivially changed to use
> get_user_pages_unlocked:
>
> - drivers/gpu/drm/via/via_dmablit.c
> - drivers/platform/goldfish/goldfish_pipe.c
> - drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c
> - drivers/rapidio/devices/rio_mport_cdev.c
> - drivers/virt/fsl_hypervisor.c
>
Ah indeed, I rather glossed through the callers and noticed that at least some
held locks long enough or were called with a lock held sufficient that I wasn't
sure it could be released.
> Also, videobuf_dma_init_user could be changed to use retry by adding a
> *locked argument to videobuf_dma_init_user_locked, prototype patch
> after my signature.
>
Very nice!
> Everything else is probably best kept using get_user_pages.
>
> > 4. It's somewhat confusing/redundant having both get_user_pages_locked() and
> > get_user_pages() functions which both require mmap_sem to be held (i.e. both
> > are 'locked' versions.)
> >
> >> If all callers were changed, then sure removing the _locked suffix would
> >> be a good idea.
> >
> > It seems many callers cannot release mmap_sem so VM_FAULT_RETRY logic couldn't
> > happen anyway in this cases and in these cases we couldn't change the caller.
>
> But then get_user_pages_locked remains a less-common case, so perhaps
> it's a good thing to give it a longer name!
My (somewhat minor) concern was that there would be confusion due to the
existence of the triumvirate of g_u_p()/g_u_p_unlocked()/g_u_p_locked(), however
the comments do a decent enough job of explaining the situation.
>
> > Overall, an alternative here might be to remove get_user_pages() and update
> > get_user_pages_locked() to add a 'vmas' parameter, however doing this renders
> > get_user_pages_unlocked() asymmetric as it would lack an vmas parameter (adding
> > one there would make no sense as VM_FAULT_RETRY logic invalidates VMAs) though
> > perhaps not such a big issue as it makes sense as to why this is the case.
>
> Adding the 'vmas' parameter to get_user_pages_locked would make little
> sense. Since VM_FAULT_RETRY invalidates it and g_u_p_locked can and
> does retry, it would generally not be useful.
I meant only in the case where we'd remove get_user_pages() and instead be left
with get_user_pages_[un]locked() only, meaning we'd have to add some means of
retrieving vmas if locked was set NULL, of course in cases where locked is not
NULL it makes no sense to add it.
>
> So I think we have the right API now:
>
> - do not have lock? Use get_user_pages_unlocked, get retry for free,
> no need to handle mmap_sem and the locked argument; cannot get back vmas.
>
> - have and cannot drop lock? User get_user_pages, no need to pass NULL
> for the locked argument; can get back vmas.
>
> - have but can drop lock (rare case)? Use get_user_pages_locked,
> cannot get back vams.
Yeah I think this is sane as it is actually, this patch set was a lot less
convincing of a cleanup than prior ones and overall it seems we are better off
with the existing API.
I wonder whether a better patch series to come out of this would be to find
cases where the lock could be dropped (i.e. the ones you mention above) and to
switch to using get_user_pages_[un]locked() where it makes sense to.
I am happy to look into these cases (though of course I must leave your
suggested patch here to you :)
On Mon, Oct 31, 2016 at 10:02:27AM +0000, Lorenzo Stoakes wrote:
> This patch adds an int *locked parameter to get_user_pages() to allow
> VM_FAULT_RETRY faulting behaviour similar to get_user_pages_[un]locked().
>
> It additionally clears the way for get_user_pages_locked() to be removed as its
> sole remaining useful characteristic was to allow for VM_FAULT_RETRY behaviour
> when faulting in pages.
>
> It should not introduce any functional changes, however it does allow for
> subsequent changes to get_user_pages() callers to take advantage of
> VM_FAULT_RETRY.
For the cris-part:
Acked-by: Jesper Nilsson <[email protected]>
> Signed-off-by: Lorenzo Stoakes <[email protected]>
/^JN - Jesper Nilsson
--
Jesper Nilsson -- [email protected]
On Mon, Nov 07, 2016 at 11:49:18AM +0100, Jesper Nilsson wrote:
> For the cris-part:
> Acked-by: Jesper Nilsson <[email protected]>
Thanks very much for that, however just to avoid any confusion, I realised this
series was not not the right way forward after discussion with Paolo and rather
it makes more sense to keep the API as it is and to update callers where it
makes sense to.