This patch set is to fix some dax kmem driver issues and then it can
support to use pmem as normal RAM in arm64 qemu guest.
Jia He (2):
drivers/dax/kmem: use default numa_mem_id if target_node is invalid
drivers/dax/kmem: give a warning if CONFIG_DEV_DAX_PMEM_COMPAT is
enabled
drivers/dax/kmem.c | 11 ++++++++---
1 file changed, 8 insertions(+), 3 deletions(-)
--
2.17.1
In some platforms(e.g arm64 guest), the NFIT info might not be ready.
Then target_node might be -1. But if there is a default numa_mem_id(),
we can use it to avoid unnecessary fatal EINVL error.
devm_memremap_pages() also uses this logic if nid is invalid, we can
keep the same page with it.
Signed-off-by: Jia He <[email protected]>
---
drivers/dax/kmem.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
index a02318c6d28a..ad62d551d94e 100644
--- a/drivers/dax/kmem.c
+++ b/drivers/dax/kmem.c
@@ -33,9 +33,9 @@ int dev_dax_kmem_probe(struct device *dev)
*/
numa_node = dev_dax->target_node;
if (numa_node < 0) {
- dev_warn(dev, "rejecting DAX region %pR with invalid node: %d\n",
- res, numa_node);
- return -EINVAL;
+ dev_warn(dev, "DAX %pR with invalid node, assume it as %d\n",
+ res, numa_node, numa_mem_id());
+ numa_node = numa_mem_id();
}
/* Hotplug starting at the beginning of the next block: */
--
2.17.1
commit c221c0b0308f ("device-dax: "Hotplug" persistent memory for use
like normal RAM") helps to add persistent memory as normal RAM blocks.
But this driver doesn't work if CONFIG_DEV_DAX_PMEM_COMPAT is enabled.
Here is the debugging call trace when CONFIG_DEV_DAX_PMEM_COMPAT is
enabled.
[ 4.443730] devm_memremap_pages+0x4b9/0x540
[ 4.443733] dev_dax_probe+0x112/0x220 [device_dax]
[ 4.443735] dax_pmem_compat_probe+0x58/0x92 [dax_pmem_compat]
[ 4.443737] nvdimm_bus_probe+0x6b/0x150
[ 4.443739] really_probe+0xf5/0x3d0
[ 4.443740] driver_probe_device+0x11b/0x130
[ 4.443741] device_driver_attach+0x58/0x60
[ 4.443742] __driver_attach+0xa3/0x140
Then the dax0.0 device will be registered as "nd" bus instead of
"dax" bus. This causes the error as follows:
root@ubuntu:~# echo dax0.0 > /sys/bus/dax/drivers/device_dax/unbind
-bash: echo: write error: No such device
This gives a warning to notify the user.
Signed-off-by: Jia He <[email protected]>
---
drivers/dax/kmem.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
index ad62d551d94e..b77f0e880598 100644
--- a/drivers/dax/kmem.c
+++ b/drivers/dax/kmem.c
@@ -93,6 +93,11 @@ static struct dax_device_driver device_dax_kmem_driver = {
static int __init dax_kmem_init(void)
{
+ if (IS_ENABLED(CONFIG_DEV_DAX_PMEM_COMPAT)) {
+ pr_warn("CONFIG_DEV_DAX_PMEM_COMPAT is not compatible\n");
+ pr_warn("kmem dax driver might not be workable\n");
+ }
+
return dax_driver_register(&device_dax_kmem_driver);
}
--
2.17.1
Jia He <[email protected]> writes:
> commit c221c0b0308f ("device-dax: "Hotplug" persistent memory for use
> like normal RAM") helps to add persistent memory as normal RAM blocks.
> But this driver doesn't work if CONFIG_DEV_DAX_PMEM_COMPAT is enabled.
>
> Here is the debugging call trace when CONFIG_DEV_DAX_PMEM_COMPAT is
> enabled.
> [ 4.443730] devm_memremap_pages+0x4b9/0x540
> [ 4.443733] dev_dax_probe+0x112/0x220 [device_dax]
> [ 4.443735] dax_pmem_compat_probe+0x58/0x92 [dax_pmem_compat]
> [ 4.443737] nvdimm_bus_probe+0x6b/0x150
> [ 4.443739] really_probe+0xf5/0x3d0
> [ 4.443740] driver_probe_device+0x11b/0x130
> [ 4.443741] device_driver_attach+0x58/0x60
> [ 4.443742] __driver_attach+0xa3/0x140
>
> Then the dax0.0 device will be registered as "nd" bus instead of
> "dax" bus. This causes the error as follows:
> root@ubuntu:~# echo dax0.0 > /sys/bus/dax/drivers/device_dax/unbind
> -bash: echo: write error: No such device
>
> This gives a warning to notify the user.
>
> Signed-off-by: Jia He <[email protected]>
> ---
> drivers/dax/kmem.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
> index ad62d551d94e..b77f0e880598 100644
> --- a/drivers/dax/kmem.c
> +++ b/drivers/dax/kmem.c
> @@ -93,6 +93,11 @@ static struct dax_device_driver device_dax_kmem_driver = {
>
> static int __init dax_kmem_init(void)
> {
> + if (IS_ENABLED(CONFIG_DEV_DAX_PMEM_COMPAT)) {
> + pr_warn("CONFIG_DEV_DAX_PMEM_COMPAT is not compatible\n");
> + pr_warn("kmem dax driver might not be workable\n");
> + }
> +
> return dax_driver_register(&device_dax_kmem_driver);
> }
This logic is wrong (and the error message is *very* confusing). You
can have the driver configured, but not loaded. In that case, the kmem
driver will load and work properly.
When using daxctl to online memory, you already get the following
message:
libdaxctl: daxctl_dev_disable: dax0.0: error: device model is dax-class
That's still not very helpful. It would be better if the message
suggested a fix (like using migrate-device-model). Vishal?
Cheers,
Jeff
On Thu, 2019-08-22 at 14:55 -0400, Jeff Moyer wrote:
>
> When using daxctl to online memory, you already get the following
> message:
>
> libdaxctl: daxctl_dev_disable: dax0.0: error: device model is dax-class
>
> That's still not very helpful. It would be better if the message
> suggested a fix (like using migrate-device-model). Vishal?
Yes, it is on my list to improve this. Currently the man page shows the
above error message, and talks about migrate-device-model, but I
received more feedback to add another bread crumb in the printed message
pointing to migrate-device-model. I'll send a patch for it soon.
Thanks,
-Vishal
Hi
Ping.
The target_node will be -1 if numa disabled. IIUC, it is a generic issue, not only on arm64.
--
Cheers,
Justin (Jia He)
> -----Original Message-----
> From: Jia He <[email protected]>
> Sent: 2019??8??16?? 19:19
> To: Dan Williams <[email protected]>; Vishal Verma
> <[email protected]>
> Cc: Keith Busch <[email protected]>; Dave Jiang
> <[email protected]>; [email protected]; linux-
> [email protected]; Justin He (Arm Technology China)
> <[email protected]>
> Subject: [PATCH 1/2] drivers/dax/kmem: use default numa_mem_id if
> target_node is invalid
>
> In some platforms(e.g arm64 guest), the NFIT info might not be ready.
> Then target_node might be -1. But if there is a default numa_mem_id(),
> we can use it to avoid unnecessary fatal EINVL error.
>
> devm_memremap_pages() also uses this logic if nid is invalid, we can
> keep the same page with it.
>
> Signed-off-by: Jia He <[email protected]>
> ---
> drivers/dax/kmem.c | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c
> index a02318c6d28a..ad62d551d94e 100644
> --- a/drivers/dax/kmem.c
> +++ b/drivers/dax/kmem.c
> @@ -33,9 +33,9 @@ int dev_dax_kmem_probe(struct device *dev)
> */
> numa_node = dev_dax->target_node;
> if (numa_node < 0) {
> - dev_warn(dev, "rejecting DAX region %pR with invalid
> node: %d\n",
> - res, numa_node);
> - return -EINVAL;
> + dev_warn(dev, "DAX %pR with invalid node, assume it
> as %d\n",
> + res, numa_node, numa_mem_id());
> + numa_node = numa_mem_id();
> }
>
> /* Hotplug starting at the beginning of the next block: */
> --
> 2.17.1
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.