2019-08-11 08:13:43

by Christoph Hellwig

[permalink] [raw]
Subject: add a not device managed memremap_pages

Hi Dan and Jason,

Bharata has been working on secure page management for kvmppc guests,
and one I thing I noticed is that he had to fake up a struct device
just so that it could be passed to the devm_memremap_pages
instrastructure for device private memory.

This series adds non-device managed versions of the
devm_request_free_mem_region and devm_memremap_pages functions for
his use case.


2019-08-11 08:14:00

by Christoph Hellwig

[permalink] [raw]
Subject: [PATCH 3/5] memremap: remove the dev field in struct dev_pagemap

The dev field in struct dev_pagemap is only used to print dev_name in
two places, which are at best nice to have. Just remove the field
and thus the name in those two messages.

Signed-off-by: Christoph Hellwig <[email protected]>
---
include/linux/memremap.h | 1 -
mm/memremap.c | 6 +-----
mm/page_alloc.c | 2 +-
3 files changed, 2 insertions(+), 7 deletions(-)

diff --git a/include/linux/memremap.h b/include/linux/memremap.h
index f8a5b2a19945..8f0013e18e14 100644
--- a/include/linux/memremap.h
+++ b/include/linux/memremap.h
@@ -109,7 +109,6 @@ struct dev_pagemap {
struct percpu_ref *ref;
struct percpu_ref internal_ref;
struct completion done;
- struct device *dev;
enum memory_type type;
unsigned int flags;
u64 pci_p2pdma_bus_offset;
diff --git a/mm/memremap.c b/mm/memremap.c
index 6ee03a816d67..600a14cbe663 100644
--- a/mm/memremap.c
+++ b/mm/memremap.c
@@ -96,7 +96,6 @@ static void dev_pagemap_cleanup(struct dev_pagemap *pgmap)
static void devm_memremap_pages_release(void *data)
{
struct dev_pagemap *pgmap = data;
- struct device *dev = pgmap->dev;
struct resource *res = &pgmap->res;
unsigned long pfn;
int nid;
@@ -123,8 +122,7 @@ static void devm_memremap_pages_release(void *data)

untrack_pfn(NULL, PHYS_PFN(res->start), resource_size(res));
pgmap_array_delete(res);
- dev_WARN_ONCE(dev, pgmap->altmap.alloc,
- "%s: failed to free all reserved pages\n", __func__);
+ WARN_ONCE(pgmap->altmap.alloc, "failed to free all reserved pages\n");
}

static void dev_pagemap_percpu_release(struct percpu_ref *ref)
@@ -245,8 +243,6 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)
goto err_array;
}

- pgmap->dev = dev;
-
error = xa_err(xa_store_range(&pgmap_array, PHYS_PFN(res->start),
PHYS_PFN(res->end), pgmap, GFP_KERNEL));
if (error)
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 272c6de1bf4e..b39baa2b1faf 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -5982,7 +5982,7 @@ void __ref memmap_init_zone_device(struct zone *zone,
}
}

- pr_info("%s initialised, %lu pages in %ums\n", dev_name(pgmap->dev),
+ pr_info("%s initialised %lu pages in %ums\n", __func__,
size, jiffies_to_msecs(jiffies - start));
}

--
2.20.1

2019-08-11 08:14:56

by Christoph Hellwig

[permalink] [raw]
Subject: [PATCH 4/5] memremap: don't use a separate devm action for devmap_managed_enable_get

Just clean up for early failures and then piggy back on
devm_memremap_pages_release. This helps with a pending not device
managed version of devm_memremap_pages.

Signed-off-by: Christoph Hellwig <[email protected]>
---
mm/memremap.c | 15 ++++++++++-----
1 file changed, 10 insertions(+), 5 deletions(-)

diff --git a/mm/memremap.c b/mm/memremap.c
index 600a14cbe663..09a087ca30ff 100644
--- a/mm/memremap.c
+++ b/mm/memremap.c
@@ -21,13 +21,13 @@ DEFINE_STATIC_KEY_FALSE(devmap_managed_key);
EXPORT_SYMBOL(devmap_managed_key);
static atomic_t devmap_managed_enable;

-static void devmap_managed_enable_put(void *data)
+static void devmap_managed_enable_put(void)
{
if (atomic_dec_and_test(&devmap_managed_enable))
static_branch_disable(&devmap_managed_key);
}

-static int devmap_managed_enable_get(struct device *dev, struct dev_pagemap *pgmap)
+static int devmap_managed_enable_get(struct dev_pagemap *pgmap)
{
if (!pgmap->ops || !pgmap->ops->page_free) {
WARN(1, "Missing page_free method\n");
@@ -36,13 +36,16 @@ static int devmap_managed_enable_get(struct device *dev, struct dev_pagemap *pgm

if (atomic_inc_return(&devmap_managed_enable) == 1)
static_branch_enable(&devmap_managed_key);
- return devm_add_action_or_reset(dev, devmap_managed_enable_put, NULL);
+ return 0;
}
#else
-static int devmap_managed_enable_get(struct device *dev, struct dev_pagemap *pgmap)
+static int devmap_managed_enable_get(struct dev_pagemap *pgmap)
{
return -EINVAL;
}
+static void devmap_managed_enable_put(void)
+{
+}
#endif /* CONFIG_DEV_PAGEMAP_OPS */

static void pgmap_array_delete(struct resource *res)
@@ -123,6 +126,7 @@ static void devm_memremap_pages_release(void *data)
untrack_pfn(NULL, PHYS_PFN(res->start), resource_size(res));
pgmap_array_delete(res);
WARN_ONCE(pgmap->altmap.alloc, "failed to free all reserved pages\n");
+ devmap_managed_enable_put();
}

static void dev_pagemap_percpu_release(struct percpu_ref *ref)
@@ -212,7 +216,7 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)
}

if (need_devmap_managed) {
- error = devmap_managed_enable_get(dev, pgmap);
+ error = devmap_managed_enable_get(pgmap);
if (error)
return ERR_PTR(error);
}
@@ -321,6 +325,7 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)
err_array:
dev_pagemap_kill(pgmap);
dev_pagemap_cleanup(pgmap);
+ devmap_managed_enable_put();
return ERR_PTR(error);
}
EXPORT_SYMBOL_GPL(devm_memremap_pages);
--
2.20.1

2019-08-11 08:14:57

by Christoph Hellwig

[permalink] [raw]
Subject: [PATCH 2/5] resource: add a not device managed request_free_mem_region variant

Just add a simple macro that passes a NULL dev argument to
dev_request_free_mem_region, and call request_mem_region in the
function for that particular case.

Signed-off-by: Christoph Hellwig <[email protected]>
---
include/linux/ioport.h | 2 ++
kernel/resource.c | 5 ++++-
2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/include/linux/ioport.h b/include/linux/ioport.h
index 0dcc48cafa80..528ae6cbb1b4 100644
--- a/include/linux/ioport.h
+++ b/include/linux/ioport.h
@@ -297,6 +297,8 @@ static inline bool resource_overlaps(struct resource *r1, struct resource *r2)

struct resource *devm_request_free_mem_region(struct device *dev,
struct resource *base, unsigned long size, const char *name);
+#define request_free_mem_region(base, size, name) \
+ devm_request_free_mem_region(NULL, base, size, name)

#endif /* __ASSEMBLY__ */
#endif /* _LINUX_IOPORT_H */
diff --git a/kernel/resource.c b/kernel/resource.c
index 0ddc558586a7..3a826b3cc883 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -1671,7 +1671,10 @@ struct resource *devm_request_free_mem_region(struct device *dev,
REGION_DISJOINT)
continue;

- res = devm_request_mem_region(dev, addr, size, name);
+ if (dev)
+ res = devm_request_mem_region(dev, addr, size, name);
+ else
+ res = request_mem_region(addr, size, name);
if (!res)
return ERR_PTR(-ENOMEM);
res->desc = IORES_DESC_DEVICE_PRIVATE_MEMORY;
--
2.20.1

2019-08-11 08:14:57

by Christoph Hellwig

[permalink] [raw]
Subject: [PATCH 1/5] resource: pass a name argument to devm_request_free_mem_region

Add an explicit resource name argument to devm_request_free_mem_region.
Besides allowing drivers to request multiple regions per device with
different names, this also prepares for a not device managed version of
the function.

Signed-off-by: Christoph Hellwig <[email protected]>
---
drivers/gpu/drm/nouveau/nouveau_dmem.c | 3 ++-
include/linux/ioport.h | 2 +-
kernel/resource.c | 5 +++--
3 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/nouveau/nouveau_dmem.c b/drivers/gpu/drm/nouveau/nouveau_dmem.c
index 1333220787a1..aedf18a44789 100644
--- a/drivers/gpu/drm/nouveau/nouveau_dmem.c
+++ b/drivers/gpu/drm/nouveau/nouveau_dmem.c
@@ -605,7 +605,8 @@ nouveau_dmem_init(struct nouveau_drm *drm)
* and latter if we want to do thing like over commit then we
* could revisit this.
*/
- res = devm_request_free_mem_region(device, &iomem_resource, size);
+ res = devm_request_free_mem_region(device, &iomem_resource, size,
+ dev_name(device));
if (IS_ERR(res))
goto out_free;
drm->dmem->pagemap.type = MEMORY_DEVICE_PRIVATE;
diff --git a/include/linux/ioport.h b/include/linux/ioport.h
index 5b6a7121c9f0..0dcc48cafa80 100644
--- a/include/linux/ioport.h
+++ b/include/linux/ioport.h
@@ -296,7 +296,7 @@ static inline bool resource_overlaps(struct resource *r1, struct resource *r2)
}

struct resource *devm_request_free_mem_region(struct device *dev,
- struct resource *base, unsigned long size);
+ struct resource *base, unsigned long size, const char *name);

#endif /* __ASSEMBLY__ */
#endif /* _LINUX_IOPORT_H */
diff --git a/kernel/resource.c b/kernel/resource.c
index 7ea4306503c5..0ddc558586a7 100644
--- a/kernel/resource.c
+++ b/kernel/resource.c
@@ -1650,13 +1650,14 @@ EXPORT_SYMBOL(resource_list_free);
* @dev: device struct to bind the resource to
* @size: size in bytes of the device memory to add
* @base: resource tree to look in
+ * @name: identifying name for the new resource
*
* This function tries to find an empty range of physical address big enough to
* contain the new resource, so that it can later be hotplugged as ZONE_DEVICE
* memory, which in turn allocates struct pages.
*/
struct resource *devm_request_free_mem_region(struct device *dev,
- struct resource *base, unsigned long size)
+ struct resource *base, unsigned long size, const char *name)
{
resource_size_t end, addr;
struct resource *res;
@@ -1670,7 +1671,7 @@ struct resource *devm_request_free_mem_region(struct device *dev,
REGION_DISJOINT)
continue;

- res = devm_request_mem_region(dev, addr, size, dev_name(dev));
+ res = devm_request_mem_region(dev, addr, size, name);
if (!res)
return ERR_PTR(-ENOMEM);
res->desc = IORES_DESC_DEVICE_PRIVATE_MEMORY;
--
2.20.1

2019-08-11 08:15:28

by Christoph Hellwig

[permalink] [raw]
Subject: [PATCH 5/5] memremap: provide a not device managed memremap_pages

The kvmppc ultravisor code wants a device private memory pool that is
system wide and not attached to a device. Instead of faking up one
provide a low-level memremap_pages for it. Note that this function is
not exported, and doesn't have a cleanup routine associated with it to
discourage use from more driver like users.

Signed-off-by: Christoph Hellwig <[email protected]>
---
include/linux/memremap.h | 1 +
mm/memremap.c | 74 ++++++++++++++++++++++++----------------
2 files changed, 45 insertions(+), 30 deletions(-)

diff --git a/include/linux/memremap.h b/include/linux/memremap.h
index 8f0013e18e14..eac23e88a94a 100644
--- a/include/linux/memremap.h
+++ b/include/linux/memremap.h
@@ -123,6 +123,7 @@ static inline struct vmem_altmap *pgmap_altmap(struct dev_pagemap *pgmap)
}

#ifdef CONFIG_ZONE_DEVICE
+void *memremap_pages(struct dev_pagemap *pgmap, int nid);
void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap);
void devm_memunmap_pages(struct device *dev, struct dev_pagemap *pgmap);
struct dev_pagemap *get_dev_pagemap(unsigned long pfn,
diff --git a/mm/memremap.c b/mm/memremap.c
index 09a087ca30ff..7b7575330db4 100644
--- a/mm/memremap.c
+++ b/mm/memremap.c
@@ -137,27 +137,12 @@ static void dev_pagemap_percpu_release(struct percpu_ref *ref)
complete(&pgmap->done);
}

-/**
- * devm_memremap_pages - remap and provide memmap backing for the given resource
- * @dev: hosting device for @res
- * @pgmap: pointer to a struct dev_pagemap
- *
- * Notes:
- * 1/ At a minimum the res and type members of @pgmap must be initialized
- * by the caller before passing it to this function
- *
- * 2/ The altmap field may optionally be initialized, in which case
- * PGMAP_ALTMAP_VALID must be set in pgmap->flags.
- *
- * 3/ The ref field may optionally be provided, in which pgmap->ref must be
- * 'live' on entry and will be killed and reaped at
- * devm_memremap_pages_release() time, or if this routine fails.
- *
- * 4/ res is expected to be a host memory range that could feasibly be
- * treated as a "System RAM" range, i.e. not a device mmio range, but
- * this is not enforced.
+/*
+ * This version is not intended for system resources only, and there is no
+ * way to clean up the resource acquisitions. If you need to clean up you
+ * probably want dev_memremap_pages below.
*/
-void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)
+void *memremap_pages(struct dev_pagemap *pgmap, int nid)
{
struct resource *res = &pgmap->res;
struct dev_pagemap *conflict_pgmap;
@@ -168,7 +153,7 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)
.altmap = pgmap_altmap(pgmap),
};
pgprot_t pgprot = PAGE_KERNEL;
- int error, nid, is_ram;
+ int error, is_ram;
bool need_devmap_managed = true;

switch (pgmap->type) {
@@ -223,7 +208,7 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)

conflict_pgmap = get_dev_pagemap(PHYS_PFN(res->start), NULL);
if (conflict_pgmap) {
- dev_WARN(dev, "Conflicting mapping in same section\n");
+ WARN(1, "Conflicting mapping in same section\n");
put_dev_pagemap(conflict_pgmap);
error = -ENOMEM;
goto err_array;
@@ -231,7 +216,7 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)

conflict_pgmap = get_dev_pagemap(PHYS_PFN(res->end), NULL);
if (conflict_pgmap) {
- dev_WARN(dev, "Conflicting mapping in same section\n");
+ WARN(1, "Conflicting mapping in same section\n");
put_dev_pagemap(conflict_pgmap);
error = -ENOMEM;
goto err_array;
@@ -252,7 +237,6 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)
if (error)
goto err_array;

- nid = dev_to_node(dev);
if (nid < 0)
nid = numa_mem_id();

@@ -308,12 +292,6 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)
PHYS_PFN(res->start),
PHYS_PFN(resource_size(res)), pgmap);
percpu_ref_get_many(pgmap->ref, pfn_end(pgmap) - pfn_first(pgmap));
-
- error = devm_add_action_or_reset(dev, devm_memremap_pages_release,
- pgmap);
- if (error)
- return ERR_PTR(error);
-
return __va(res->start);

err_add_memory:
@@ -328,6 +306,42 @@ void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)
devmap_managed_enable_put();
return ERR_PTR(error);
}
+
+/**
+ * devm_memremap_pages - remap and provide memmap backing for the given resource
+ * @dev: hosting device for @res
+ * @pgmap: pointer to a struct dev_pagemap
+ *
+ * Notes:
+ * 1/ At a minimum the res and type members of @pgmap must be initialized
+ * by the caller before passing it to this function
+ *
+ * 2/ The altmap field may optionally be initialized, in which case
+ * PGMAP_ALTMAP_VALID must be set in pgmap->flags.
+ *
+ * 3/ The ref field may optionally be provided, in which pgmap->ref must be
+ * 'live' on entry and will be killed and reaped at
+ * devm_memremap_pages_release() time, or if this routine fails.
+ *
+ * 4/ res is expected to be a host memory range that could feasibly be
+ * treated as a "System RAM" range, i.e. not a device mmio range, but
+ * this is not enforced.
+ */
+void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap)
+{
+ int error;
+ void *ret;
+
+ ret = memremap_pages(pgmap, dev_to_node(dev));
+ if (IS_ERR(ret))
+ return ret;
+
+ error = devm_add_action_or_reset(dev, devm_memremap_pages_release,
+ pgmap);
+ if (error)
+ return ERR_PTR(error);
+ return ret;
+}
EXPORT_SYMBOL_GPL(devm_memremap_pages);

void devm_memunmap_pages(struct device *dev, struct dev_pagemap *pgmap)
--
2.20.1

2019-08-11 22:56:47

by Jason Gunthorpe

[permalink] [raw]
Subject: Re: [PATCH 2/5] resource: add a not device managed request_free_mem_region variant

On Sun, Aug 11, 2019 at 10:12:44AM +0200, Christoph Hellwig wrote:
> Just add a simple macro that passes a NULL dev argument to
> dev_request_free_mem_region, and call request_mem_region in the
> function for that particular case.
>
> Signed-off-by: Christoph Hellwig <[email protected]>
> include/linux/ioport.h | 2 ++
> kernel/resource.c | 5 ++++-
> 2 files changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/include/linux/ioport.h b/include/linux/ioport.h
> index 0dcc48cafa80..528ae6cbb1b4 100644
> +++ b/include/linux/ioport.h
> @@ -297,6 +297,8 @@ static inline bool resource_overlaps(struct resource *r1, struct resource *r2)
>
> struct resource *devm_request_free_mem_region(struct device *dev,
> struct resource *base, unsigned long size, const char *name);
> +#define request_free_mem_region(base, size, name) \
> + devm_request_free_mem_region(NULL, base, size, name)
>
> #endif /* __ASSEMBLY__ */
> #endif /* _LINUX_IOPORT_H */
> diff --git a/kernel/resource.c b/kernel/resource.c
> index 0ddc558586a7..3a826b3cc883 100644
> +++ b/kernel/resource.c
> @@ -1671,7 +1671,10 @@ struct resource *devm_request_free_mem_region(struct device *dev,
> REGION_DISJOINT)
> continue;
>
> - res = devm_request_mem_region(dev, addr, size, name);
> + if (dev)
> + res = devm_request_mem_region(dev, addr, size, name);
> + else
> + res = request_mem_region(addr, size, name);

It is a bit jarring to have something called devm_* that doesn't
actually do the devm_ part on some paths.

Maybe this function should be called __request_free_mem_region() with
another name wrapper macro?

Jason

2019-08-11 23:00:04

by Jason Gunthorpe

[permalink] [raw]
Subject: Re: [PATCH 5/5] memremap: provide a not device managed memremap_pages

On Sun, Aug 11, 2019 at 10:12:47AM +0200, Christoph Hellwig wrote:
> The kvmppc ultravisor code wants a device private memory pool that is
> system wide and not attached to a device. Instead of faking up one
> provide a low-level memremap_pages for it. Note that this function is
> not exported, and doesn't have a cleanup routine associated with it to
> discourage use from more driver like users.
>
> Signed-off-by: Christoph Hellwig <[email protected]>
> include/linux/memremap.h | 1 +
> mm/memremap.c | 74 ++++++++++++++++++++++++----------------
> 2 files changed, 45 insertions(+), 30 deletions(-)
>
> diff --git a/include/linux/memremap.h b/include/linux/memremap.h
> index 8f0013e18e14..eac23e88a94a 100644
> +++ b/include/linux/memremap.h
> @@ -123,6 +123,7 @@ static inline struct vmem_altmap *pgmap_altmap(struct dev_pagemap *pgmap)
> }
>
> #ifdef CONFIG_ZONE_DEVICE
> +void *memremap_pages(struct dev_pagemap *pgmap, int nid);
> void *devm_memremap_pages(struct device *dev, struct dev_pagemap *pgmap);
> void devm_memunmap_pages(struct device *dev, struct dev_pagemap *pgmap);
> struct dev_pagemap *get_dev_pagemap(unsigned long pfn,
> diff --git a/mm/memremap.c b/mm/memremap.c
> index 09a087ca30ff..7b7575330db4 100644
> +++ b/mm/memremap.c
> @@ -137,27 +137,12 @@ static void dev_pagemap_percpu_release(struct percpu_ref *ref)
> complete(&pgmap->done);
> }
>
> -/**
> - * devm_memremap_pages - remap and provide memmap backing for the given resource
> - * @dev: hosting device for @res
> - * @pgmap: pointer to a struct dev_pagemap
> - *
> - * Notes:
> - * 1/ At a minimum the res and type members of @pgmap must be initialized
> - * by the caller before passing it to this function
> - *
> - * 2/ The altmap field may optionally be initialized, in which case
> - * PGMAP_ALTMAP_VALID must be set in pgmap->flags.
> - *
> - * 3/ The ref field may optionally be provided, in which pgmap->ref must be
> - * 'live' on entry and will be killed and reaped at
> - * devm_memremap_pages_release() time, or if this routine fails.
> - *
> - * 4/ res is expected to be a host memory range that could feasibly be
> - * treated as a "System RAM" range, i.e. not a device mmio range, but
> - * this is not enforced.
> +/*
> + * This version is not intended for system resources only, and there is no

Was 'is not' what was intended here? I'm having a hard time reading
this.

Jason

2019-08-12 07:41:49

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH 2/5] resource: add a not device managed request_free_mem_region variant

On Sun, Aug 11, 2019 at 10:52:58PM +0000, Jason Gunthorpe wrote:
> It is a bit jarring to have something called devm_* that doesn't
> actually do the devm_ part on some paths.
>
> Maybe this function should be called __request_free_mem_region() with
> another name wrapper macro?

Seems like a little more churn than required, but I could do it.

2019-08-12 07:43:19

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH 5/5] memremap: provide a not device managed memremap_pages

On Sun, Aug 11, 2019 at 10:56:07PM +0000, Jason Gunthorpe wrote:
> > + * This version is not intended for system resources only, and there is no
>
> Was 'is not' what was intended here? I'm having a hard time reading
> this.

s/not//g

2019-08-12 14:53:39

by Bharata B Rao

[permalink] [raw]
Subject: Re: [PATCH 5/5] memremap: provide a not device managed memremap_pages

On Sun, Aug 11, 2019 at 10:12:47AM +0200, Christoph Hellwig wrote:
> The kvmppc ultravisor code wants a device private memory pool that is
> system wide and not attached to a device. Instead of faking up one
> provide a low-level memremap_pages for it. Note that this function is
> not exported, and doesn't have a cleanup routine associated with it to
> discourage use from more driver like users.

The kvmppc secure pages management code will be part of kvm-hv which
can be built as module too. So it would require memremap_pages() to be
exported.

Additionally, non-dev version of the cleanup routine
devm_memremap_pages_release() or equivalent would also be requried.
With device being present, put_device() used to take care of this
cleanup.

Regards,
Bharata.

2019-08-12 15:01:34

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH 5/5] memremap: provide a not device managed memremap_pages

On Mon, Aug 12, 2019 at 08:20:58PM +0530, Bharata B Rao wrote:
> On Sun, Aug 11, 2019 at 10:12:47AM +0200, Christoph Hellwig wrote:
> > The kvmppc ultravisor code wants a device private memory pool that is
> > system wide and not attached to a device. Instead of faking up one
> > provide a low-level memremap_pages for it. Note that this function is
> > not exported, and doesn't have a cleanup routine associated with it to
> > discourage use from more driver like users.
>
> The kvmppc secure pages management code will be part of kvm-hv which
> can be built as module too. So it would require memremap_pages() to be
> exported.
>
> Additionally, non-dev version of the cleanup routine
> devm_memremap_pages_release() or equivalent would also be requried.
> With device being present, put_device() used to take care of this
> cleanup.

Oh well. We can add them fairly easily if we really need to, but I
tried to avoid that. Can you try to see if this works non-modular
for you for now until we hear more feedback from Dan?

2019-08-13 04:57:29

by Bharata B Rao

[permalink] [raw]
Subject: Re: [PATCH 5/5] memremap: provide a not device managed memremap_pages

On Mon, Aug 12, 2019 at 05:00:12PM +0200, Christoph Hellwig wrote:
> On Mon, Aug 12, 2019 at 08:20:58PM +0530, Bharata B Rao wrote:
> > On Sun, Aug 11, 2019 at 10:12:47AM +0200, Christoph Hellwig wrote:
> > > The kvmppc ultravisor code wants a device private memory pool that is
> > > system wide and not attached to a device. Instead of faking up one
> > > provide a low-level memremap_pages for it. Note that this function is
> > > not exported, and doesn't have a cleanup routine associated with it to
> > > discourage use from more driver like users.
> >
> > The kvmppc secure pages management code will be part of kvm-hv which
> > can be built as module too. So it would require memremap_pages() to be
> > exported.
> >
> > Additionally, non-dev version of the cleanup routine
> > devm_memremap_pages_release() or equivalent would also be requried.
> > With device being present, put_device() used to take care of this
> > cleanup.
>
> Oh well. We can add them fairly easily if we really need to, but I
> tried to avoid that. Can you try to see if this works non-modular
> for you for now until we hear more feedback from Dan?

Yes, this patchset works non-modular and with kvm-hv as module, it
works with devm_memremap_pages_release() and release_mem_region() in the
cleanup path. The cleanup path will be required in the non-modular
case too for proper recovery from failures.

Regards,
Bharata.

2019-08-14 06:14:06

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH 5/5] memremap: provide a not device managed memremap_pages

On Tue, Aug 13, 2019 at 10:26:11AM +0530, Bharata B Rao wrote:
> Yes, this patchset works non-modular and with kvm-hv as module, it
> works with devm_memremap_pages_release() and release_mem_region() in the
> cleanup path. The cleanup path will be required in the non-modular
> case too for proper recovery from failures.

Can you check if the version here:

git://git.infradead.org/users/hch/misc.git pgmap-remove-dev

Gitweb:

http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/pgmap-remove-dev

works for you fully before I resend?

>
> Regards,
> Bharata.
---end quoted text---

2019-08-14 08:59:28

by Bharata B Rao

[permalink] [raw]
Subject: Re: [PATCH 5/5] memremap: provide a not device managed memremap_pages

On Wed, Aug 14, 2019 at 08:11:50AM +0200, Christoph Hellwig wrote:
> On Tue, Aug 13, 2019 at 10:26:11AM +0530, Bharata B Rao wrote:
> > Yes, this patchset works non-modular and with kvm-hv as module, it
> > works with devm_memremap_pages_release() and release_mem_region() in the
> > cleanup path. The cleanup path will be required in the non-modular
> > case too for proper recovery from failures.
>
> Can you check if the version here:
>
> git://git.infradead.org/users/hch/misc.git pgmap-remove-dev
>
> Gitweb:
>
> http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/pgmap-remove-dev
>
> works for you fully before I resend?

Yes, this works for us. This and migrate-vma-cleanup series helps to
really simplify the kvmppc secure pages management code. Thanks.

Regards,
Bharata.

2019-08-14 11:27:08

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH 5/5] memremap: provide a not device managed memremap_pages

On Wed, Aug 14, 2019 at 02:28:26PM +0530, Bharata B Rao wrote:
> > http://git.infradead.org/users/hch/misc.git/shortlog/refs/heads/pgmap-remove-dev
> >
> > works for you fully before I resend?
>
> Yes, this works for us. This and migrate-vma-cleanup series helps to
> really simplify the kvmppc secure pages management code. Thanks.

Thanks. I'm going to resend it once we've made a bit of progress
on the migrate_vma series that I resent this morning. There are
a few more lose ends in this area with implications for the driver
API, so I might have a few more patches for you to test in a bit.