The reserved_mem array is used to store data for the different
reserved memory regions defined in the DT of a device. The array
stores information such as region name, node, start-address, and size
of the reserved memory regions.
The array is currently statically allocated with a size of
MAX_RESERVED_REGIONS(64). This means that any system that specifies a
number of reserved memory regions greater than MAX_RESERVED_REGIONS(64)
will not have enough space to store the information for all the regions.
Therefore, this series extends the use of the static array for
reserved_mem, and introduces a dynamically allocated array using
memblock_alloc() based on the number of reserved memory regions
specified in the DT.
Some architectures such as arm64 require the page tables to be setup
before memblock allocated memory is writable. Therefore, the dynamic
allocation of the reserved_mem array will need to be done after the
page tables have been setup on these architectures. In most cases that
will be after paging_init().
Reserved memory regions can be divided into 2 groups.
i) Statically-placed reserved memory regions
i.e. regions defined in the DT using the @reg property.
ii) Dynamically-placed reserved memory regions.
i.e. regions specified in the DT using the @alloc_ranges
and @size properties.
It is possible to call memblock_reserve() and memblock_mark_nomap() on
the statically-placed reserved memory regions and not need to save them
to the reserved_mem array until memory is allocated for it using
memblock, which will be after the page tables have been setup.
For the dynamically-placed reserved memory regions, it is not possible
to wait to store its information because the starting address is
allocated only at run time, and hence they need to be stored somewhere
after they are allocated.
Waiting until after the page tables have been setup to allocate memory
for the dynamically-placed regions is also not an option because the
allocations will come from memory that have already been added to the
page tables, which is not good for memory that is supposed to be
reserved and/or marked as nomap.
Therefore, this series splits up the processing of the reserved memory
regions into two stages, of which the first stage is carried out by
early_init_fdt_scan_reserved_mem() and the second is carried out by
fdt_init_reserved_mem().
The early_init_fdt_scan_reserved_mem(), which is called before the page
tables are setup is used to:
1. Call memblock_reserve() and memblock_mark_nomap() on all the
statically-placed reserved memory regions as needed.
2. Allocate memory from memblock for the dynamically-placed reserved
memory regions and store them in the static array for reserved_mem.
memblock_reserve() and memblock_mark_nomap() are also called as
needed on all the memory allocated for the dynamically-placed
regions.
3. Count the total number of reserved memory regions found in the DT.
fdt_init_reserved_mem(), which should be called after the page tables
have been setup, is used to carry out the following:
1. Allocate memory for the reserved_mem array based on the number of
reserved memory regions counted as mentioned above.
2. Copy all the information for the dynamically-placed reserved memory
regions from the static array into the new allocated memory for the
reserved_mem array.
3. Add the information for the statically-placed reserved memory into
reserved_mem array.
4. Run the region specific init functions for each of the reserve memory
regions saved in the reserved_mem array.
Once the above steps have been completed and the init process is done
running, the original statically allocated reserved_mem array of size
MAX_RESERVED_REGIONS(64) will be automatically freed back to buddy
because it is no longer needed. This is done by marking the array as an
"__initdata" object in Patch 0018.
Note:
- Per Architecture, this series is effectively only 10 patches. The
code for each architecture is split up into separate patches to
allow each architecture to be tested independently of changes from
other architectures. Should this series be accepted, this should
allow for each arcitecture change to be picked up independently as
well.
Patch 0001: Splits up the processing of the reserved memory regions
between early_init_fdt_scan_reserved_mem and fdt_init_reserved_mem.
Patch 0002: Introduces a copy of early_init_fdt_scan_reserved_mem()
which is used to separate it from fdt_init_reserved_mem() so that the
two functions can be called independently of each other.
Patch 0003 - Patch 0016: Duplicated change for each architecture to
call early_init_fdt_scan_reserved_mem() and fdt_init_reserved_mem()
at their appropriate locations. Here fdt_init_reserved_mem() is called
either before of after the page tables have been setup depending on
the architecture requirements.
Patch 0017: Deletes the early_init_fdt_scan_reserved_mem() function
since all architectures are now using the copy introduced in
Patch 0002.
Patch 0018: Dynamically allocate memory for the reserved_mem array
based on the total number of reserved memory regions specified in the
DT.
Patch 0019 - Patch 0029: Duplicated change for each architecture to
move the fdt_init_reserved_mem() function call to below the
unflatten_devicetree() function call. This is so that the unflatten
devicetree APIs can be used to process the reserved memory regions.
Patch 0030: Make code changes to start using the unflatten devicetree
APIs to access the reserved memory regions defined in the DT.
Patch 0031: Rename fdt_* functions as dt_* to refelct that the
flattened devicetree (fdt) APIs have been replaced with the unflatten
devicetree APIs.
Patch 0032 - Patch 0045: Duplicated change for each architecture to
switch from the use of fdt_init_reserved_mem() to
dt_init_reserved_mem(), which is the same function but the later uses
the unflatten devicetree APIs.
Patch 0046: Delete the fdt_init_reserved_mem() function as all
architectures have switched to using dt_init_reserved_mem() which was
introduced in Patch 0031.
- The limitation to this approach is that there is still a limit of
64 for dynamically-placed reserved memory regions. But from my current
analysis, these types of reserved memory regions are generally less
in number when compared to the statically-placed reserved memory
regions.
- I have looked through all architectures and placed the call to
memblock_alloc() for the reserved_mem array at points where I
believe memblock allocated memory are available to be written to.
I currently only have access to an arm64 device and this is where I am
testing the functionality of this series. Hence, I will need help from
architecture maintainers to test this series on other architectures to
ensure that the code is functioning properly on there.
Previous patch revisions:
1. [RFC V1 Patchset]:
https://lore.kernel.org/all/[email protected]/
2. [RFC V2 Patchset]:
https://lore.kernel.org/all/[email protected]/
- Extend changes to all other relevant architectures.
- Add code to use unflatten devicetree APIs to process the reserved
memory regions.
Oreoluwa Babatunde (46):
of: reserved_mem: Change the order that reserved_mem regions are
stored
of: reserved_mem: Introduce new early reserved memory scan function
ARC: reserved_mem: Implement the new processing order for reserved
memory
ARM: reserved_mem: Implement the new processing order for reserved
memory
arm64: reserved_mem: Implement the new processing order for reserved
memory
csky: reserved_mem: Implement the new processing order for reserved
memory
Loongarch: reserved_mem: Implement the new processing order for
reserved memory
microblaze: reserved_mem: Implement the new processing order for
reserved memory
mips: reserved_mem: Implement the new processing order for reserved
memory
nios2: reserved_mem: Implement the new processing order for reserved
memory
openrisc: reserved_mem: Implement the new processing order for
reserved memory
powerpc: reserved_mem: Implement the new processing order for reserved
memory
riscv: reserved_mem: Implement the new processing order for reserved
memory
sh: reserved_mem: Implement the new processing order for reserved
memory
um: reserved_mem: Implement the new processing order for reserved
memory
xtensa: reserved_mem: Implement the new processing order for reserved
memory
of: reserved_mem: Delete the early_init_fdt_scan_reserved_mem()
function
of: reserved_mem: Add code to dynamically allocate reserved_mem array
ARC: resrved_mem: Move fdt_init_reserved_mem() below
unflatten_device_tree()
ARM: resrved_mem: Move fdt_init_reserved_mem() below
unflatten_device_tree()
arm64: resrved_mem: Move fdt_init_reserved_mem() below
unflatten_device_tree()
csky: resrved_mem: Move fdt_init_reserved_mem() below
unflatten_device_tree()
microblaze: resrved_mem: Move fdt_init_reserved_mem() below
unflatten_device_tree()
mips: resrved_mem: Move fdt_init_reserved_mem() below
unflatten_device_tree()
nios2: resrved_mem: Move fdt_init_reserved_mem() below
unflatten_device_tree()
powerpc: resrved_mem: Move fdt_init_reserved_mem() below
unflatten_device_tree()
riscv: resrved_mem: Move fdt_init_reserved_mem() below
unflatten_device_tree()
um: resrved_mem: Move fdt_init_reserved_mem() below
unflatten_device_tree()
xtensa: resrved_mem: Move fdt_init_reserved_mem() below
unflatten_device_tree()
of: reserved_mem: Add code to use unflattened DT for reserved_mem
nodes
of: reserved_mem: Rename fdt_* functions to refelct use of unflattened
devicetree APIs
ARC: reserved_mem: Switch fdt_init_reserved_mem() to
dt_init_reserved_mem()
ARM: reserved_mem: Switch fdt_init_reserved_mem() to
dt_init_reserved_mem()
arm64: reserved_mem: Switch fdt_init_reserved_mem() to
dt_init_reserved_mem()
csky: reserved_mem: Switch fdt_init_reserved_mem() to
dt_init_reserved_mem()
loongarch: reserved_mem: Switch fdt_init_reserved_mem to
dt_init_reserved_mem
microblaze: reserved_mem: Switch fdt_init_reserved_mem to
dt_init_reserved_mem
mips: reserved_mem: Switch fdt_init_reserved_mem() to
dt_init_reserved_mem()
nios2: reserved_mem: Switch fdt_init_reserved_mem() to
dt_init_reserved_mem()
openrisc: reserved_mem: Switch fdt_init_reserved_mem to
dt_init_reserved_mem
powerpc: reserved_mem: Switch fdt_init_reserved_mem() to
dt_init_reserved_mem()
riscv: reserved_mem: Switch fdt_init_reserved_mem() to
dt_init_reserved_mem()
sh: reserved_mem: Switch fdt_init_reserved_mem() to
dt_init_reserved_mem()
um: reserved_mem: Switch fdt_init_reserved_mem() to
dt_init_reserved_mem()
xtensa: reserved_mem: Switch fdt_init_reserved_mem() to
dt_init_reserved_mem()
of: reserved_mem: Delete the fdt_init_reserved_mem() function
arch/arc/kernel/setup.c | 2 +
arch/arc/mm/init.c | 2 +-
arch/arm/kernel/setup.c | 4 +
arch/arm/mm/init.c | 2 +-
arch/arm64/kernel/setup.c | 3 +
arch/arm64/mm/init.c | 2 +-
arch/csky/kernel/setup.c | 4 +-
arch/loongarch/kernel/setup.c | 4 +-
arch/microblaze/kernel/setup.c | 3 +
arch/microblaze/mm/init.c | 2 +-
arch/mips/kernel/setup.c | 4 +-
arch/nios2/kernel/setup.c | 5 +-
arch/openrisc/kernel/setup.c | 4 +-
arch/powerpc/kernel/prom.c | 2 +-
arch/powerpc/kernel/setup-common.c | 3 +
arch/riscv/kernel/setup.c | 3 +
arch/riscv/mm/init.c | 2 +-
arch/sh/boards/of-generic.c | 4 +-
arch/um/kernel/dtb.c | 4 +-
arch/xtensa/kernel/setup.c | 2 +
arch/xtensa/mm/init.c | 2 +-
drivers/of/fdt.c | 42 +++++--
drivers/of/of_private.h | 5 +-
drivers/of/of_reserved_mem.c | 178 +++++++++++++++++++++--------
include/linux/of_fdt.h | 4 +-
include/linux/of_reserved_mem.h | 11 +-
kernel/dma/coherent.c | 4 +-
kernel/dma/contiguous.c | 8 +-
kernel/dma/swiotlb.c | 10 +-
29 files changed, 234 insertions(+), 91 deletions(-)
--
2.17.1
The unflattened devicetree structure is available to be used not long
after the page tables have been set up on most architectures, and is
available even before that on other architectures.
Hence, move the call to fdt_init_reserved_mem() to after
unflatten_device_tree() is called so that the reserved memory nodes can
be accessed using the unflattened device tree APIs.
Using the unflattened devicetree APIs is more efficient than using the
flattened devicetree APIs.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/arm64/kernel/setup.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index 2a9e98104af7..426f9cc45ce2 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -347,8 +347,6 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
paging_init();
- fdt_init_reserved_mem();
-
acpi_table_upgrade();
/* Parse the ACPI tables for possible boot-time configuration */
@@ -357,6 +355,8 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
if (acpi_disabled)
unflatten_device_tree();
+ fdt_init_reserved_mem();
+
bootmem_init();
kasan_init();
--
2.17.1
Switch from using fdt_init_reserved_mem() to dt_init_reserved_mem() to
reflect the use of the unflatten devicetree APIs to process the
reserved memory regions.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/sh/boards/of-generic.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/sh/boards/of-generic.c b/arch/sh/boards/of-generic.c
index 7bec409f077c..a9af31404167 100644
--- a/arch/sh/boards/of-generic.c
+++ b/arch/sh/boards/of-generic.c
@@ -112,7 +112,7 @@ static void __init sh_of_mem_reserve(void)
{
early_init_fdt_reserve_self();
early_fdt_scan_reserved_mem();
- fdt_init_reserved_mem();
+ dt_init_reserved_mem();
}
static void __init sh_of_setup(char **cmdline_p)
--
2.17.1
Delete the fdt_init_reserved_mem() function as all architectures have
been switched to using dt_init_reserved_mem(), which is basically a copy
of the function, but uses the unflatten devicetree APIs instead of the
fdt APIs.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
drivers/of/of_reserved_mem.c | 54 ---------------------------------
include/linux/of_reserved_mem.h | 3 --
2 files changed, 57 deletions(-)
diff --git a/drivers/of/of_reserved_mem.c b/drivers/of/of_reserved_mem.c
index 3650efab0afd..3d1ab2325217 100644
--- a/drivers/of/of_reserved_mem.c
+++ b/drivers/of/of_reserved_mem.c
@@ -382,60 +382,6 @@ static void __init __rmem_check_for_overlap(void)
}
}
-/**
- * fdt_init_reserved_mem() - allocate and init all saved reserved memory regions
- */
-void __init fdt_init_reserved_mem(void)
-{
- int i, ret;
-
- ret = alloc_reserved_mem_array();
- if (ret)
- pr_err("Failed to allocate memory for reserved_mem array with err: %d", ret);
-
- dt_scan_reserved_mem_reg_nodes();
-
- /* check for overlapping reserved regions */
- __rmem_check_for_overlap();
-
- for (i = 0; i < reserved_mem_count; i++) {
- struct reserved_mem *rmem = &reserved_mem[i];
- struct device_node *node = rmem->dev_node;
- int len;
- const __be32 *prop;
- int err = 0;
- bool nomap;
-
- nomap = of_get_property(node, "no-map", NULL) != NULL;
- prop = of_get_property(node, "phandle", &len);
- if (!prop)
- prop = of_get_property(node, "linux,phandle", &len);
- if (prop)
- rmem->phandle = of_read_number(prop, len/4);
-
- err = __reserved_mem_init_node(rmem);
- if (err != 0 && err != -ENOENT) {
- pr_info("node %s compatible matching fail\n",
- rmem->name);
- if (nomap)
- memblock_clear_nomap(rmem->base, rmem->size);
- else
- memblock_phys_free(rmem->base,
- rmem->size);
- } else {
- phys_addr_t end = rmem->base + rmem->size - 1;
- bool reusable =
- (of_get_property(node, "reusable", NULL)) != NULL;
-
- pr_info("%pa..%pa (%lu KiB) %s %s %s\n",
- &rmem->base, &end, (unsigned long)(rmem->size / SZ_1K),
- nomap ? "nomap" : "map",
- reusable ? "reusable" : "non-reusable",
- rmem->name ? rmem->name : "unknown");
- }
- }
-}
-
/**
* dt_init_reserved_mem() - allocate and init all saved reserved memory regions
*/
diff --git a/include/linux/of_reserved_mem.h b/include/linux/of_reserved_mem.h
index b1f71a4894aa..dd67b9b2488e 100644
--- a/include/linux/of_reserved_mem.h
+++ b/include/linux/of_reserved_mem.h
@@ -33,7 +33,6 @@ typedef int (*reservedmem_of_init_fn)(struct reserved_mem *rmem);
_OF_DECLARE(reservedmem, name, compat, init, reservedmem_of_init_fn)
void dt_init_reserved_mem(void);
-void fdt_init_reserved_mem(void);
int of_reserved_mem_device_init_by_idx(struct device *dev,
struct device_node *np, int idx);
int of_reserved_mem_device_init_by_name(struct device *dev,
@@ -50,8 +49,6 @@ struct reserved_mem *of_reserved_mem_lookup(struct device_node *np);
static inline void dt_init_reserved_mem(void) { }
-static inline void fdt_init_reserved_mem(void) { }
-
static inline int of_reserved_mem_device_init_by_idx(struct device *dev,
struct device_node *np, int idx)
{
--
2.17.1
Call early_fdt_scan_reserved_mem() in place of
early_init_fdt_scan_reserved_mem() to carry out the first stage of the
reserved memory processing only.
The early_fdt_scan_reserved_mem() function is used to scan through the
DT and mark all the reserved memory regions as reserved or nomap as
needed, as well as allocate the memory required by the
dynamically-placed
reserved memory regions.
The second stage of the reserved memory processing is done by
fdt_init_reserved_mem(). This function is used to store the information
of the statically-placed reserved memory nodes in the reserved_mem
array as well as call the region specific initialization function on all
the stored reserved memory regions.
The call to fdt_init_reserved_mem() is placed after setup_memory()
in preparation for the dynamic allocation of the reserved_mem array
using memblock. This is because memblock allocated memory is not
writable until after the page tables have been setup on the microblaze
architecture.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/microblaze/kernel/setup.c | 3 +++
arch/microblaze/mm/init.c | 2 +-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/arch/microblaze/kernel/setup.c b/arch/microblaze/kernel/setup.c
index f417333eccae..631faa4613ec 100644
--- a/arch/microblaze/kernel/setup.c
+++ b/arch/microblaze/kernel/setup.c
@@ -29,6 +29,7 @@
#include <linux/pci.h>
#include <linux/cache.h>
#include <linux/of.h>
+#include <linux/of_reserved_mem.h>
#include <linux/dma-mapping.h>
#include <asm/cacheflush.h>
#include <asm/entry.h>
@@ -54,6 +55,8 @@ void __init setup_arch(char **cmdline_p)
setup_memory();
+ fdt_init_reserved_mem();
+
console_verbose();
unflatten_device_tree();
diff --git a/arch/microblaze/mm/init.c b/arch/microblaze/mm/init.c
index 3827dc76edd8..9e73a1433dfa 100644
--- a/arch/microblaze/mm/init.c
+++ b/arch/microblaze/mm/init.c
@@ -262,7 +262,7 @@ asmlinkage void __init mmu_init(void)
parse_early_param();
- early_init_fdt_scan_reserved_mem();
+ early_fdt_scan_reserved_mem();
/* CMA initialization */
dma_contiguous_reserve(memory_start + lowmem_size - 1);
--
2.17.1
Switch from using fdt_init_reserved_mem() to dt_init_reserved_mem() to
reflect the use of the unflatten devicetree APIs to process the
reserved memory regions.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/riscv/kernel/setup.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index 0601ed1e4ce6..3abc41217ede 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -271,7 +271,7 @@ void __init setup_arch(char **cmdline_p)
unflatten_device_tree();
#endif
- fdt_init_reserved_mem();
+ dt_init_reserved_mem();
misc_mem_init();
init_resources();
--
2.17.1
The unflattened devicetree APIs are available to be used not long after
the processing is done for the reserved memory regions on some
architectures, and is available even before that on other architectures.
Therefore, use the unflattened devicetree APIs to process and store
information for the reserved memory regions.
Using the unflattened devicetree APIs is more efficient than using the
flattened devicetree APIs.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
drivers/of/fdt.c | 49 ----------------------
drivers/of/of_private.h | 4 +-
drivers/of/of_reserved_mem.c | 74 ++++++++++++++++++++++++++-------
include/linux/of_fdt.h | 1 -
include/linux/of_reserved_mem.h | 2 +-
kernel/dma/coherent.c | 4 +-
kernel/dma/contiguous.c | 8 ++--
kernel/dma/swiotlb.c | 10 ++---
8 files changed, 72 insertions(+), 80 deletions(-)
diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
index ebd2fa9e0114..2bc01ffdabfe 100644
--- a/drivers/of/fdt.c
+++ b/drivers/of/fdt.c
@@ -558,55 +558,6 @@ static int __init __reserved_mem_check_root(unsigned long node)
return 0;
}
-/*
- * Save the reserved_mem reg nodes in the reserved_mem array
- */
-void __init fdt_scan_reserved_mem_reg_nodes(void)
-
-{
- int t_len = (dt_root_addr_cells + dt_root_size_cells) * sizeof(__be32);
- const void *fdt = initial_boot_params;
- phys_addr_t base, size;
- const __be32 *prop;
- int node, child;
- int len;
-
- node = fdt_path_offset(fdt, "/reserved-memory");
- if (node < 0) {
- pr_err("Reserved memory: Did not find reserved-memory node\n");
- return;
- }
-
- if (__reserved_mem_check_root(node) != 0) {
- pr_err("Reserved memory: unsupported node format, ignoring\n");
- return;
- }
-
- fdt_for_each_subnode(child, fdt, node) {
- const char *uname;
-
- prop = of_get_flat_dt_prop(child, "reg", &len);
- if (!prop)
- continue;
-
- if (!of_fdt_device_is_available(fdt, child))
- continue;
-
- uname = fdt_get_name(fdt, child, NULL);
- if (len && len % t_len != 0) {
- pr_err("Reserved memory: invalid reg property in '%s', skipping node.\n",
- uname);
- continue;
- }
-
- base = dt_mem_next_cell(dt_root_addr_cells, &prop);
- size = dt_mem_next_cell(dt_root_size_cells, &prop);
-
- if (size)
- fdt_reserved_mem_save_node(child, uname, base, size);
- }
-}
-
/*
* fdt_scan_reserved_mem() - scan a single FDT node for reserved memory.
*/
diff --git a/drivers/of/of_private.h b/drivers/of/of_private.h
index c338e1c019c7..f7da22108e7a 100644
--- a/drivers/of/of_private.h
+++ b/drivers/of/of_private.h
@@ -177,7 +177,7 @@ static inline struct device_node *__of_get_dma_parent(const struct device_node *
#endif
void update_reserved_mem_max_cnt(int max_count);
-void fdt_reserved_mem_save_node(unsigned long node, const char *uname,
- phys_addr_t base, phys_addr_t size);
+void fdt_reserved_mem_save_node(struct device_node *node, const char *uname,
+ phys_addr_t base, phys_addr_t size);
#endif /* _LINUX_OF_PRIVATE_H */
diff --git a/drivers/of/of_reserved_mem.c b/drivers/of/of_reserved_mem.c
index 419b062cb41f..645b02e27492 100644
--- a/drivers/of/of_reserved_mem.c
+++ b/drivers/of/of_reserved_mem.c
@@ -96,11 +96,58 @@ static int alloc_reserved_mem_array(void)
return -1;
}
+/*
+ * Save the reserved_mem reg nodes in the reserved_mem array
+ */
+static void __init dt_scan_reserved_mem_reg_nodes(void)
+{
+ int t_len = (dt_root_addr_cells + dt_root_size_cells) * sizeof(__be32);
+ struct device_node *node, *child;
+ phys_addr_t base, size;
+ const __be32 *prop;
+ int len;
+
+ node = of_find_node_by_path("/reserved-memory");
+ if (node < 0) {
+ pr_err("Reserved memory: Did not find reserved-memory node\n");
+ return;
+ }
+
+ for_each_child_of_node(node, child) {
+ const char *uname;
+ struct reserved_mem *rmem;
+
+ if (!of_device_is_available(child))
+ continue;
+
+ prop = of_get_property(child, "reg", &len);
+ if (!prop) {
+ rmem = of_reserved_mem_lookup(child);
+ if (rmem)
+ rmem->dev_node = child;
+ continue;
+ }
+
+ uname = of_node_full_name(child);
+ if (len && len % t_len != 0) {
+ pr_err("Reserved memory: invalid reg property in '%s', skipping node.\n",
+ uname);
+ continue;
+ }
+
+ base = dt_mem_next_cell(dt_root_addr_cells, &prop);
+ size = dt_mem_next_cell(dt_root_size_cells, &prop);
+
+ if (size)
+ fdt_reserved_mem_save_node(child, uname, base, size);
+ }
+}
+
/*
* fdt_reserved_mem_save_node() - save fdt node for second pass initialization
*/
-void __init fdt_reserved_mem_save_node(unsigned long node, const char *uname,
- phys_addr_t base, phys_addr_t size)
+void __init fdt_reserved_mem_save_node(struct device_node *node, const char *uname,
+ phys_addr_t base, phys_addr_t size)
{
struct reserved_mem *rmem = &reserved_mem[reserved_mem_count];
@@ -109,7 +156,7 @@ void __init fdt_reserved_mem_save_node(unsigned long node, const char *uname,
return;
}
- rmem->fdt_node = node;
+ rmem->dev_node = node;
rmem->name = uname;
rmem->base = base;
rmem->size = size;
@@ -252,7 +299,7 @@ int __init __reserved_mem_alloc_size(unsigned long node, const char *uname)
uname, (unsigned long)(size / SZ_1M));
return -ENOMEM;
}
- fdt_reserved_mem_save_node(node, uname, base, size);
+ fdt_reserved_mem_save_node(NULL, uname, base, size);
return 0;
}
@@ -272,7 +319,7 @@ static int __init __reserved_mem_init_node(struct reserved_mem *rmem)
reservedmem_of_init_fn initfn = i->data;
const char *compat = i->compatible;
- if (!of_flat_dt_is_compatible(rmem->fdt_node, compat))
+ if (!of_device_is_compatible(rmem->dev_node, compat))
continue;
ret = initfn(rmem);
@@ -305,11 +352,6 @@ static int __init __rmem_cmp(const void *a, const void *b)
if (ra->size > rb->size)
return 1;
- if (ra->fdt_node < rb->fdt_node)
- return -1;
- if (ra->fdt_node > rb->fdt_node)
- return 1;
-
return 0;
}
@@ -351,23 +393,23 @@ void __init fdt_init_reserved_mem(void)
if (ret)
pr_err("Failed to allocate memory for reserved_mem array with err: %d", ret);
- fdt_scan_reserved_mem_reg_nodes();
+ dt_scan_reserved_mem_reg_nodes();
/* check for overlapping reserved regions */
__rmem_check_for_overlap();
for (i = 0; i < reserved_mem_count; i++) {
struct reserved_mem *rmem = &reserved_mem[i];
- unsigned long node = rmem->fdt_node;
+ struct device_node *node = rmem->dev_node;
int len;
const __be32 *prop;
int err = 0;
bool nomap;
- nomap = of_get_flat_dt_prop(node, "no-map", NULL) != NULL;
- prop = of_get_flat_dt_prop(node, "phandle", &len);
+ nomap = of_get_property(node, "no-map", NULL) != NULL;
+ prop = of_get_property(node, "phandle", &len);
if (!prop)
- prop = of_get_flat_dt_prop(node, "linux,phandle", &len);
+ prop = of_get_property(node, "linux,phandle", &len);
if (prop)
rmem->phandle = of_read_number(prop, len/4);
@@ -383,7 +425,7 @@ void __init fdt_init_reserved_mem(void)
} else {
phys_addr_t end = rmem->base + rmem->size - 1;
bool reusable =
- (of_get_flat_dt_prop(node, "reusable", NULL)) != NULL;
+ (of_get_property(node, "reusable", NULL)) != NULL;
pr_info("%pa..%pa (%lu KiB) %s %s %s\n",
&rmem->base, &end, (unsigned long)(rmem->size / SZ_1K),
diff --git a/include/linux/of_fdt.h b/include/linux/of_fdt.h
index 9b85bbc5d9f5..fb7b437141bd 100644
--- a/include/linux/of_fdt.h
+++ b/include/linux/of_fdt.h
@@ -73,7 +73,6 @@ extern int early_init_dt_scan_root(void);
extern bool early_init_dt_scan(void *params);
extern bool early_init_dt_verify(void *params);
extern void early_init_dt_scan_nodes(void);
-extern void fdt_scan_reserved_mem_reg_nodes(void);
extern const char *of_flat_dt_get_machine_name(void);
extern const void *of_flat_dt_match_machine(const void *default_match,
diff --git a/include/linux/of_reserved_mem.h b/include/linux/of_reserved_mem.h
index 2a3178920bae..e92babd669c2 100644
--- a/include/linux/of_reserved_mem.h
+++ b/include/linux/of_reserved_mem.h
@@ -10,7 +10,7 @@ struct reserved_mem_ops;
struct reserved_mem {
const char *name;
- unsigned long fdt_node;
+ struct device_node *dev_node;
unsigned long phandle;
const struct reserved_mem_ops *ops;
phys_addr_t base;
diff --git a/kernel/dma/coherent.c b/kernel/dma/coherent.c
index ff5683a57f77..5ab26fe11c29 100644
--- a/kernel/dma/coherent.c
+++ b/kernel/dma/coherent.c
@@ -362,9 +362,9 @@ static const struct reserved_mem_ops rmem_dma_ops = {
static int __init rmem_dma_setup(struct reserved_mem *rmem)
{
- unsigned long node = rmem->fdt_node;
+ struct device_node *node = rmem->dev_node;
- if (of_get_flat_dt_prop(node, "reusable", NULL))
+ if (of_get_property(node, "reusable", NULL))
return -EINVAL;
#ifdef CONFIG_ARM
diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
index f005c66f378c..b54cf128a9d9 100644
--- a/kernel/dma/contiguous.c
+++ b/kernel/dma/contiguous.c
@@ -462,8 +462,8 @@ static const struct reserved_mem_ops rmem_cma_ops = {
static int __init rmem_cma_setup(struct reserved_mem *rmem)
{
- unsigned long node = rmem->fdt_node;
- bool default_cma = of_get_flat_dt_prop(node, "linux,cma-default", NULL);
+ struct device_node *node = rmem->dev_node;
+ bool default_cma = of_get_property(node, "linux,cma-default", NULL);
struct cma *cma;
int err;
@@ -473,8 +473,8 @@ static int __init rmem_cma_setup(struct reserved_mem *rmem)
return -EBUSY;
}
- if (!of_get_flat_dt_prop(node, "reusable", NULL) ||
- of_get_flat_dt_prop(node, "no-map", NULL))
+ if (!of_get_property(node, "reusable", NULL) ||
+ of_get_property(node, "no-map", NULL))
return -EINVAL;
if (!IS_ALIGNED(rmem->base | rmem->size, CMA_MIN_ALIGNMENT_BYTES)) {
diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c
index b079a9a8e087..ea1f734c8c35 100644
--- a/kernel/dma/swiotlb.c
+++ b/kernel/dma/swiotlb.c
@@ -1732,12 +1732,12 @@ static const struct reserved_mem_ops rmem_swiotlb_ops = {
static int __init rmem_swiotlb_setup(struct reserved_mem *rmem)
{
- unsigned long node = rmem->fdt_node;
+ struct device_node *node = rmem->dev_node;
- if (of_get_flat_dt_prop(node, "reusable", NULL) ||
- of_get_flat_dt_prop(node, "linux,cma-default", NULL) ||
- of_get_flat_dt_prop(node, "linux,dma-default", NULL) ||
- of_get_flat_dt_prop(node, "no-map", NULL))
+ if (of_get_property(node, "reusable", NULL) ||
+ of_get_property(node, "linux,cma-default", NULL) ||
+ of_get_property(node, "linux,dma-default", NULL) ||
+ of_get_property(node, "no-map", NULL))
return -EINVAL;
rmem->ops = &rmem_swiotlb_ops;
--
2.17.1
Switch from using fdt_init_reserved_mem() to dt_init_reserved_mem() to
reflect the use of the unflatten devicetree APIs to process the
reserved memory regions.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/arc/kernel/setup.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arc/kernel/setup.c b/arch/arc/kernel/setup.c
index c5e3c4abb249..6b904771c158 100644
--- a/arch/arc/kernel/setup.c
+++ b/arch/arc/kernel/setup.c
@@ -527,7 +527,7 @@ void __init setup_arch(char **cmdline_p)
/* copy flat DT out of .init and then unflatten it */
unflatten_and_copy_device_tree();
- fdt_init_reserved_mem();
+ dt_init_reserved_mem();
/* Can be issue if someone passes cmd line arg "ro"
* But that is unlikely so keeping it as it is
*/
--
2.17.1
Call early_fdt_scan_reserved_mem() in place of
early_init_fdt_scan_reserved_mem() to carry out the first stage of the
reserved memory processing only.
The early_fdt_scan_reserved_mem() function is used to scan through the
DT and mark all the reserved memory regions as reserved or nomap as
needed, as well as allocate the memory required by the
dynamically-placed
reserved memory regions.
The second stage of the reserved memory processing is done by
fdt_init_reserved_mem(). This function is used to store the information
of the statically-placed reserved memory nodes in the reserved_mem
array as well as call the region specific initialization function on all
the stored reserved memory regions.
The call to fdt_init_reserved_mem() is placed right after
early_fdt_scan_reserved_mem() because memblock allocated memory should
already be writable at this point.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/sh/boards/of-generic.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/sh/boards/of-generic.c b/arch/sh/boards/of-generic.c
index f7f3e618e85b..7bec409f077c 100644
--- a/arch/sh/boards/of-generic.c
+++ b/arch/sh/boards/of-generic.c
@@ -8,6 +8,7 @@
#include <linux/of.h>
#include <linux/of_clk.h>
#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
#include <linux/clocksource.h>
#include <linux/irqchip.h>
#include <asm/machvec.h>
@@ -110,7 +111,8 @@ static int noopi(void)
static void __init sh_of_mem_reserve(void)
{
early_init_fdt_reserve_self();
- early_init_fdt_scan_reserved_mem();
+ early_fdt_scan_reserved_mem();
+ fdt_init_reserved_mem();
}
static void __init sh_of_setup(char **cmdline_p)
--
2.17.1
Switch from using fdt_init_reserved_mem() to dt_init_reserved_mem() to
reflect the use of the unflatten devicetree APIs to process the
reserved memory regions.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/csky/kernel/setup.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/csky/kernel/setup.c b/arch/csky/kernel/setup.c
index d8c65819877b..eefbbfdba535 100644
--- a/arch/csky/kernel/setup.c
+++ b/arch/csky/kernel/setup.c
@@ -75,7 +75,7 @@ void __init setup_arch(char **cmdline_p)
unflatten_and_copy_device_tree();
- fdt_init_reserved_mem();
+ dt_init_reserved_mem();
#ifdef CONFIG_SMP
setup_smp();
#endif
--
2.17.1
The unflattened devicetree structure is available to be used not long
after the page tables have been set up on most architectures, and is
available even before that on other architectures.
Hence, move the call to fdt_init_reserved_mem() to after
unflatten_device_tree() is called so that the reserved memory nodes can
be accessed using the unflattened device tree APIs.
Using the unflattened devicetree APIs is more efficient than using the
flattened devicetree APIs.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/powerpc/kernel/prom.c | 2 --
arch/powerpc/kernel/setup-common.c | 3 +++
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c
index 5f6307ea3069..3cf8213b114c 100644
--- a/arch/powerpc/kernel/prom.c
+++ b/arch/powerpc/kernel/prom.c
@@ -27,7 +27,6 @@
#include <linux/memblock.h>
#include <linux/of.h>
#include <linux/of_fdt.h>
-#include <linux/of_reserved_mem.h>
#include <linux/libfdt.h>
#include <linux/cpu.h>
#include <linux/pgtable.h>
@@ -621,7 +620,6 @@ static void __init early_reserve_mem_dt(void)
early_init_fdt_reserve_self();
early_fdt_scan_reserved_mem();
- fdt_init_reserved_mem();
dt_root = of_get_flat_dt_root();
diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index 9b142b9d5187..96bd4c964943 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -32,6 +32,7 @@
#include <linux/memblock.h>
#include <linux/of.h>
#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
#include <linux/of_irq.h>
#include <linux/hugetlb.h>
#include <linux/pgtable.h>
@@ -890,6 +891,8 @@ void __init setup_arch(char **cmdline_p)
/* Unflatten the device-tree passed by prom_init or kexec */
unflatten_device_tree();
+ fdt_init_reserved_mem();
+
/*
* Initialize cache line/block info from device-tree (on ppc64) or
* just cputable (on ppc32).
--
2.17.1
Call early_fdt_scan_reserved_mem() in place of
early_init_fdt_scan_reserved_mem() to carry out the first stage of the
reserved memory processing only.
The early_fdt_scan_reserved_mem() function is used to scan through the
DT and mark all the reserved memory regions as reserved or nomap as
needed, as well as allocate the memory required by the
dynamically-placed
reserved memory regions.
The second stage of the reserved memory processing is done by
fdt_init_reserved_mem(). This function is used to store the information
of the statically-placed reserved memory nodes in the reserved_mem
array as well as call the region specific initialization function on all
the stored reserved memory regions.
The call to fdt_init_reserved_mem() is placed right after
early_fdt_scan_reserved_mem() because memblock allocated memory should
already be writable at this point.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/openrisc/kernel/setup.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/openrisc/kernel/setup.c b/arch/openrisc/kernel/setup.c
index 9cf7fb60441f..2c7059a0484b 100644
--- a/arch/openrisc/kernel/setup.c
+++ b/arch/openrisc/kernel/setup.c
@@ -31,6 +31,7 @@
#include <linux/serial.h>
#include <linux/initrd.h>
#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
#include <linux/of.h>
#include <linux/device.h>
@@ -86,7 +87,8 @@ static void __init setup_memory(void)
#endif /* CONFIG_BLK_DEV_INITRD */
early_init_fdt_reserve_self();
- early_init_fdt_scan_reserved_mem();
+ early_fdt_scan_reserved_mem();
+ fdt_init_reserved_mem();
memblock_dump_all();
}
--
2.17.1
Switch from using fdt_init_reserved_mem() to dt_init_reserved_mem() to
reflect the use of the unflatten devicetree APIs to process the
reserved memory regions.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/arm64/kernel/setup.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index 426f9cc45ce2..646de760c675 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -355,7 +355,7 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
if (acpi_disabled)
unflatten_device_tree();
- fdt_init_reserved_mem();
+ dt_init_reserved_mem();
bootmem_init();
--
2.17.1
Switch from using fdt_init_reserved_mem() to dt_init_reserved_mem() to
reflect the use of the unflatten devicetree APIs to process the
reserved memory regions.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/arm/kernel/setup.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 36fa18e80ab3..04d3a3693a02 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -1173,7 +1173,7 @@ void __init setup_arch(char **cmdline_p)
unflatten_device_tree();
- fdt_init_reserved_mem();
+ dt_init_reserved_mem();
arm_dt_init_cpu_maps();
psci_dt_init();
--
2.17.1
Delete the early_init_fdt_scan_reserved_mem() function definition since
this function is no longer being used anywhere in the kernel.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
drivers/of/fdt.c | 29 -----------------------------
include/linux/of_fdt.h | 2 --
2 files changed, 31 deletions(-)
diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
index 6bda033936af..c6e8560946f4 100644
--- a/drivers/of/fdt.c
+++ b/drivers/of/fdt.c
@@ -682,35 +682,6 @@ static void __init fdt_reserve_elfcorehdr(void)
elfcorehdr_size >> 10, elfcorehdr_addr);
}
-/**
- * early_init_fdt_scan_reserved_mem() - create reserved memory regions
- *
- * This function grabs memory from early allocator for device exclusive use
- * defined in device tree structures. It should be called by arch specific code
- * once the early allocator (i.e. memblock) has been fully activated.
- */
-void __init early_init_fdt_scan_reserved_mem(void)
-{
- int n;
- u64 base, size;
-
- if (!initial_boot_params)
- return;
-
- fdt_scan_reserved_mem();
- fdt_reserve_elfcorehdr();
-
- /* Process header /memreserve/ fields */
- for (n = 0; ; n++) {
- fdt_get_mem_rsv(initial_boot_params, n, &base, &size);
- if (!size)
- break;
- memblock_reserve(base, size);
- }
-
- fdt_init_reserved_mem();
-}
-
/**
* early_fdt_scan_reserved_mem() - create reserved memory regions
*
diff --git a/include/linux/of_fdt.h b/include/linux/of_fdt.h
index 9b849c5c3917..9b85bbc5d9f5 100644
--- a/include/linux/of_fdt.h
+++ b/include/linux/of_fdt.h
@@ -62,7 +62,6 @@ extern int early_init_dt_scan_chosen(char *cmdline);
extern int early_init_dt_scan_memory(void);
extern void early_init_dt_check_for_usable_mem_range(void);
extern int early_init_dt_scan_chosen_stdout(void);
-extern void early_init_fdt_scan_reserved_mem(void);
extern void early_fdt_scan_reserved_mem(void);
extern void early_init_fdt_reserve_self(void);
extern void early_init_dt_add_memory_arch(u64 base, u64 size);
@@ -88,7 +87,6 @@ extern void early_get_first_memblock_info(void *, phys_addr_t *);
#else /* CONFIG_OF_EARLY_FLATTREE */
static inline void early_init_dt_check_for_usable_mem_range(void) {}
static inline int early_init_dt_scan_chosen_stdout(void) { return -ENODEV; }
-static inline void early_init_fdt_scan_reserved_mem(void) {}
static inline void early_fdt_scan_reserved_mem(void) {}
static inline void early_init_fdt_reserve_self(void) {}
static inline const char *of_flat_dt_get_machine_name(void) { return NULL; }
--
2.17.1
Call early_fdt_scan_reserved_mem() in place of
early_init_fdt_scan_reserved_mem() to carry out the first stage of the
reserved memory processing only.
The early_fdt_scan_reserved_mem() function is used to scan through the
DT and mark all the reserved memory regions as reserved or nomap as
needed, as well as allocate the memory required by the
dynamically-placed
reserved memory regions.
The second stage of the reserved memory processing is done by
fdt_init_reserved_mem(). This function is used to store the information
of the statically-placed reserved memory nodes in the reserved_mem
array as well as call the region specific initialization function on all
the stored reserved memory regions.
The call to fdt_init_reserved_mem() is placed right after
early_fdt_scan_reserved_mem() because memblock allocated memory should
be already writable at this point.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/powerpc/kernel/prom.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/prom.c b/arch/powerpc/kernel/prom.c
index 0b5878c3125b..5f6307ea3069 100644
--- a/arch/powerpc/kernel/prom.c
+++ b/arch/powerpc/kernel/prom.c
@@ -27,6 +27,7 @@
#include <linux/memblock.h>
#include <linux/of.h>
#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
#include <linux/libfdt.h>
#include <linux/cpu.h>
#include <linux/pgtable.h>
@@ -619,7 +620,8 @@ static void __init early_reserve_mem_dt(void)
const __be32 *prop;
early_init_fdt_reserve_self();
- early_init_fdt_scan_reserved_mem();
+ early_fdt_scan_reserved_mem();
+ fdt_init_reserved_mem();
dt_root = of_get_flat_dt_root();
--
2.17.1
The unflattened devicetree structure is available to be used not long
after the page tables have been set up on most architectures, and is
available even before that on other architectures.
Hence, move the call to fdt_init_reserved_mem() to after
unflatten_device_tree() is called so that the reserved memory nodes can
be accessed using the unflattened device tree APIs.
Using the unflattened devicetree APIs is more efficient than using the
flattened devicetree APIs.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/arm/kernel/setup.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm/kernel/setup.c b/arch/arm/kernel/setup.c
index 8cf3709ed985..36fa18e80ab3 100644
--- a/arch/arm/kernel/setup.c
+++ b/arch/arm/kernel/setup.c
@@ -1163,8 +1163,6 @@ void __init setup_arch(char **cmdline_p)
paging_init(mdesc);
- fdt_init_reserved_mem();
-
kasan_init();
request_standard_resources(mdesc);
@@ -1175,6 +1173,8 @@ void __init setup_arch(char **cmdline_p)
unflatten_device_tree();
+ fdt_init_reserved_mem();
+
arm_dt_init_cpu_maps();
psci_dt_init();
#ifdef CONFIG_SMP
--
2.17.1
The unflattened devicetree structure is available to be used not long
after the page tables have been set up on most architectures, and is
available even before that on other architectures.
Hence, move the call to fdt_init_reserved_mem() to after
unflatten_device_tree() is called so that the reserved memory nodes can
be accessed using the unflattened device tree APIs.
Using the unflattened devicetree APIs is more efficient than using the
flattened devicetree APIs.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/riscv/kernel/setup.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/riscv/kernel/setup.c b/arch/riscv/kernel/setup.c
index ea4fbc8e0ea1..0601ed1e4ce6 100644
--- a/arch/riscv/kernel/setup.c
+++ b/arch/riscv/kernel/setup.c
@@ -262,8 +262,6 @@ void __init setup_arch(char **cmdline_p)
efi_init();
paging_init();
- fdt_init_reserved_mem();
-
/* Parse the ACPI tables for possible boot-time configuration */
acpi_boot_table_init();
@@ -272,6 +270,8 @@ void __init setup_arch(char **cmdline_p)
#else
unflatten_device_tree();
#endif
+
+ fdt_init_reserved_mem();
misc_mem_init();
init_resources();
--
2.17.1
The unflattened devicetree structure is available to be used not long
after the page tables have been set up on most architectures, and is
available even before that on other architectures.
Hence, move the call to fdt_init_reserved_mem() to after
unflatten_device_tree() is called so that the reserved memory nodes can
be accessed using the unflattened device tree APIs.
Using the unflattened devicetree APIs is more efficient than using the
flattened devicetree APIs.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/xtensa/kernel/setup.c | 2 ++
arch/xtensa/mm/init.c | 2 --
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/xtensa/kernel/setup.c b/arch/xtensa/kernel/setup.c
index bdec4a773af0..cb2fb993de76 100644
--- a/arch/xtensa/kernel/setup.c
+++ b/arch/xtensa/kernel/setup.c
@@ -25,6 +25,7 @@
#include <linux/cpu.h>
#include <linux/of.h>
#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
#if defined(CONFIG_VGA_CONSOLE) || defined(CONFIG_DUMMY_CONSOLE)
# include <linux/console.h>
@@ -357,6 +358,7 @@ void __init setup_arch(char **cmdline_p)
kasan_init();
unflatten_and_copy_device_tree();
+ fdt_init_reserved_mem();
#ifdef CONFIG_SMP
smp_init_cpus();
#endif
diff --git a/arch/xtensa/mm/init.c b/arch/xtensa/mm/init.c
index ed3dd5f67b4a..e205a89a2097 100644
--- a/arch/xtensa/mm/init.c
+++ b/arch/xtensa/mm/init.c
@@ -26,7 +26,6 @@
#include <linux/nodemask.h>
#include <linux/mm.h>
#include <linux/of_fdt.h>
-#include <linux/of_reserved_mem.h>
#include <linux/dma-map-ops.h>
#include <asm/bootparam.h>
@@ -49,7 +48,6 @@ void __init bootmem_init(void)
memblock_reserve(0, PHYS_OFFSET ? PHYS_OFFSET : 1);
early_fdt_scan_reserved_mem();
- fdt_init_reserved_mem();
if (!memblock_phys_mem_size())
panic("No memory found!\n");
--
2.17.1
Call early_fdt_scan_reserved_mem() in place of
early_init_fdt_scan_reserved_mem() to carry out the first stage of the
reserved memory processing only.
The early_fdt_scan_reserved_mem() function is used to scan through the
DT and mark all the reserved memory regions as reserved or nomap as
needed, as well as allocate the memory required by the
dynamically-placed
reserved memory regions.
The second stage of the reserved memory processing is done by
fdt_init_reserved_mem(). This function is used to store the information
of the statically-placed reserved memory nodes in the reserved_mem
array as well as call the region specific initialization function on all
the stored reserved memory regions.
The call to fdt_init_reserved_mem() is placed after setup_arch_memory()
in preparation for the dynamic allocation of the reserved_mem array
using memblock. This is because memblock allocated memory is not
writable until after the page tables have been setup on the arm64
architecture.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/arm64/kernel/setup.c | 3 +++
arch/arm64/mm/init.c | 2 +-
2 files changed, 4 insertions(+), 1 deletion(-)
diff --git a/arch/arm64/kernel/setup.c b/arch/arm64/kernel/setup.c
index 42c690bb2d60..2a9e98104af7 100644
--- a/arch/arm64/kernel/setup.c
+++ b/arch/arm64/kernel/setup.c
@@ -27,6 +27,7 @@
#include <linux/proc_fs.h>
#include <linux/memblock.h>
#include <linux/of_fdt.h>
+#include <linux/of_reserved_mem.h>
#include <linux/efi.h>
#include <linux/psci.h>
#include <linux/sched/task.h>
@@ -346,6 +347,8 @@ void __init __no_sanitize_address setup_arch(char **cmdline_p)
paging_init();
+ fdt_init_reserved_mem();
+
acpi_table_upgrade();
/* Parse the ACPI tables for possible boot-time configuration */
diff --git a/arch/arm64/mm/init.c b/arch/arm64/mm/init.c
index 74c1db8ce271..0fe8587e550c 100644
--- a/arch/arm64/mm/init.c
+++ b/arch/arm64/mm/init.c
@@ -310,7 +310,7 @@ void __init arm64_memblock_init(void)
initrd_end = initrd_start + phys_initrd_size;
}
- early_init_fdt_scan_reserved_mem();
+ early_fdt_scan_reserved_mem();
high_memory = __va(memblock_end_of_DRAM() - 1) + 1;
}
--
2.17.1
Switch from using fdt_init_reserved_mem() to dt_init_reserved_mem() to
reflect the use of the unflatten devicetree APIs to process the
reserved memory regions.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/openrisc/kernel/setup.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/openrisc/kernel/setup.c b/arch/openrisc/kernel/setup.c
index 2c7059a0484b..a7249538a497 100644
--- a/arch/openrisc/kernel/setup.c
+++ b/arch/openrisc/kernel/setup.c
@@ -88,7 +88,7 @@ static void __init setup_memory(void)
early_init_fdt_reserve_self();
early_fdt_scan_reserved_mem();
- fdt_init_reserved_mem();
+ dt_init_reserved_mem();
memblock_dump_all();
}
--
2.17.1
Switch from using fdt_init_reserved_mem() to dt_init_reserved_mem() to
reflect the use of the unflatten devicetree APIs to process the
reserved memory regions.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/powerpc/kernel/setup-common.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/setup-common.c b/arch/powerpc/kernel/setup-common.c
index 96bd4c964943..fdb7cb8235d0 100644
--- a/arch/powerpc/kernel/setup-common.c
+++ b/arch/powerpc/kernel/setup-common.c
@@ -891,7 +891,7 @@ void __init setup_arch(char **cmdline_p)
/* Unflatten the device-tree passed by prom_init or kexec */
unflatten_device_tree();
- fdt_init_reserved_mem();
+ dt_init_reserved_mem();
/*
* Initialize cache line/block info from device-tree (on ppc64) or
--
2.17.1
Switch from using fdt_init_reserved_mem() to dt_init_reserved_mem() to
reflect the use of the unflatten devicetree APIs to process the
reserved memory regions.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/mips/kernel/setup.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/mips/kernel/setup.c b/arch/mips/kernel/setup.c
index eeafc3abcb96..8d2aaa96d3c3 100644
--- a/arch/mips/kernel/setup.c
+++ b/arch/mips/kernel/setup.c
@@ -671,7 +671,7 @@ static void __init arch_mem_init(char **cmdline_p)
mips_parse_crashkernel();
device_tree_init();
- fdt_init_reserved_mem();
+ dt_init_reserved_mem();
/*
* In order to reduce the possibility of kernel panic when failed to
* get IO TLB memory under CONFIG_SWIOTLB, it is better to allocate
--
2.17.1
Switch from using fdt_init_reserved_mem() to dt_init_reserved_mem() to
reflect the use of the unflatten devicetree APIs to process the
reserved memory regions.
Signed-off-by: Oreoluwa Babatunde <[email protected]>
---
arch/loongarch/kernel/setup.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/loongarch/kernel/setup.c b/arch/loongarch/kernel/setup.c
index 72b164d3ace0..fd1bf9a74c14 100644
--- a/arch/loongarch/kernel/setup.c
+++ b/arch/loongarch/kernel/setup.c
@@ -393,7 +393,7 @@ static void __init arch_mem_init(char **cmdline_p)
early_fdt_scan_reserved_mem();
- fdt_init_reserved_mem();
+ dt_init_reserved_mem();
/*
* In order to reduce the possibility of kernel panic when failed to
* get IO TLB memory under CONFIG_SWIOTLB, it is better to allocate
--
2.17.1
Hi Oreoluwa,
kernel test robot noticed the following build warnings:
[auto build test WARNING on robh/for-next]
[also build test WARNING on arm64/for-next/core vgupta-arc/for-curr powerpc/next powerpc/fixes jcmvbkbc-xtensa/xtensa-for-next linus/master v6.8-rc1 next-20240125]
[cannot apply to vgupta-arc/for-next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Oreoluwa-Babatunde/of-reserved_mem-Change-the-order-that-reserved_mem-regions-are-stored/20240127-081735
base: https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git for-next
patch link: https://lore.kernel.org/r/20240126235425.12233-31-quic_obabatun%40quicinc.com
patch subject: [PATCH 30/46] of: reserved_mem: Add code to use unflattened DT for reserved_mem nodes
config: i386-randconfig-061-20240127 (https://download.01.org/0day-ci/archive/20240128/[email protected]/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240128/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
sparse warnings: (new ones prefixed by >>)
>> drivers/of/of_reserved_mem.c:111:18: sparse: sparse: incompatible types for operation (<):
drivers/of/of_reserved_mem.c:111:18: sparse: struct device_node *[assigned] node
drivers/of/of_reserved_mem.c:111:18: sparse: int
vim +111 drivers/of/of_reserved_mem.c
98
99 /*
100 * Save the reserved_mem reg nodes in the reserved_mem array
101 */
102 static void __init dt_scan_reserved_mem_reg_nodes(void)
103 {
104 int t_len = (dt_root_addr_cells + dt_root_size_cells) * sizeof(__be32);
105 struct device_node *node, *child;
106 phys_addr_t base, size;
107 const __be32 *prop;
108 int len;
109
110 node = of_find_node_by_path("/reserved-memory");
> 111 if (node < 0) {
112 pr_err("Reserved memory: Did not find reserved-memory node\n");
113 return;
114 }
115
116 for_each_child_of_node(node, child) {
117 const char *uname;
118 struct reserved_mem *rmem;
119
120 if (!of_device_is_available(child))
121 continue;
122
123 prop = of_get_property(child, "reg", &len);
124 if (!prop) {
125 rmem = of_reserved_mem_lookup(child);
126 if (rmem)
127 rmem->dev_node = child;
128 continue;
129 }
130
131 uname = of_node_full_name(child);
132 if (len && len % t_len != 0) {
133 pr_err("Reserved memory: invalid reg property in '%s', skipping node.\n",
134 uname);
135 continue;
136 }
137
138 base = dt_mem_next_cell(dt_root_addr_cells, &prop);
139 size = dt_mem_next_cell(dt_root_size_cells, &prop);
140
141 if (size)
142 fdt_reserved_mem_save_node(child, uname, base, size);
143 }
144 }
145
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Hi Oreoluwa,
kernel test robot noticed the following build warnings:
[auto build test WARNING on robh/for-next]
[also build test WARNING on arm64/for-next/core vgupta-arc/for-curr powerpc/next powerpc/fixes jcmvbkbc-xtensa/xtensa-for-next linus/master v6.8-rc1 next-20240125]
[cannot apply to vgupta-arc/for-next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Oreoluwa-Babatunde/of-reserved_mem-Change-the-order-that-reserved_mem-regions-are-stored/20240127-081735
base: https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git for-next
patch link: https://lore.kernel.org/r/20240126235425.12233-31-quic_obabatun%40quicinc.com
patch subject: [PATCH 30/46] of: reserved_mem: Add code to use unflattened DT for reserved_mem nodes
config: arm-randconfig-r133-20240127 (https://download.01.org/0day-ci/archive/20240128/[email protected]/config)
compiler: arm-linux-gnueabi-gcc (GCC) 13.2.0
reproduce: (https://download.01.org/0day-ci/archive/20240128/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
sparse warnings: (new ones prefixed by >>)
>> kernel/dma/coherent.c:371:34: sparse: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned long node @@ got struct device_node *node @@
kernel/dma/coherent.c:371:34: sparse: expected unsigned long node
kernel/dma/coherent.c:371:34: sparse: got struct device_node *node
kernel/dma/coherent.c:378:33: sparse: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned long node @@ got struct device_node *node @@
kernel/dma/coherent.c:378:33: sparse: expected unsigned long node
kernel/dma/coherent.c:378:33: sparse: got struct device_node *node
vim +371 kernel/dma/coherent.c
7bfa5ab6fa1b18 drivers/base/dma-coherent.c Marek Szyprowski 2014-10-13 362
7bfa5ab6fa1b18 drivers/base/dma-coherent.c Marek Szyprowski 2014-10-13 363 static int __init rmem_dma_setup(struct reserved_mem *rmem)
7bfa5ab6fa1b18 drivers/base/dma-coherent.c Marek Szyprowski 2014-10-13 364 {
b81d457b174810 kernel/dma/coherent.c Oreoluwa Babatunde 2024-01-26 365 struct device_node *node = rmem->dev_node;
7bfa5ab6fa1b18 drivers/base/dma-coherent.c Marek Szyprowski 2014-10-13 366
b81d457b174810 kernel/dma/coherent.c Oreoluwa Babatunde 2024-01-26 367 if (of_get_property(node, "reusable", NULL))
7bfa5ab6fa1b18 drivers/base/dma-coherent.c Marek Szyprowski 2014-10-13 368 return -EINVAL;
7bfa5ab6fa1b18 drivers/base/dma-coherent.c Marek Szyprowski 2014-10-13 369
7bfa5ab6fa1b18 drivers/base/dma-coherent.c Marek Szyprowski 2014-10-13 370 #ifdef CONFIG_ARM
7bfa5ab6fa1b18 drivers/base/dma-coherent.c Marek Szyprowski 2014-10-13 @371 if (!of_get_flat_dt_prop(node, "no-map", NULL)) {
7bfa5ab6fa1b18 drivers/base/dma-coherent.c Marek Szyprowski 2014-10-13 372 pr_err("Reserved memory: regions without no-map are not yet supported\n");
7bfa5ab6fa1b18 drivers/base/dma-coherent.c Marek Szyprowski 2014-10-13 373 return -EINVAL;
7bfa5ab6fa1b18 drivers/base/dma-coherent.c Marek Szyprowski 2014-10-13 374 }
70d6aa0ecfed25 kernel/dma/coherent.c Christoph Hellwig 2021-06-24 375 #endif
93228b44c33a57 drivers/base/dma-coherent.c Vladimir Murzin 2017-06-26 376
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
Hi Oreoluwa,
kernel test robot noticed the following build warnings:
[auto build test WARNING on robh/for-next]
[also build test WARNING on arm64/for-next/core vgupta-arc/for-curr powerpc/next powerpc/fixes jcmvbkbc-xtensa/xtensa-for-next linus/master v6.8-rc2 next-20240129]
[cannot apply to vgupta-arc/for-next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Oreoluwa-Babatunde/of-reserved_mem-Change-the-order-that-reserved_mem-regions-are-stored/20240127-081735
base: https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git for-next
patch link: https://lore.kernel.org/r/20240126235425.12233-31-quic_obabatun%40quicinc.com
patch subject: [PATCH 30/46] of: reserved_mem: Add code to use unflattened DT for reserved_mem nodes
config: i386-randconfig-141-20240128 (https://download.01.org/0day-ci/archive/20240130/[email protected]/config)
compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
smatch warnings:
drivers/of/of_reserved_mem.c:111 dt_scan_reserved_mem_reg_nodes() warn: unsigned 'node' is never less than zero.
vim +/node +111 drivers/of/of_reserved_mem.c
98
99 /*
100 * Save the reserved_mem reg nodes in the reserved_mem array
101 */
102 static void __init dt_scan_reserved_mem_reg_nodes(void)
103 {
104 int t_len = (dt_root_addr_cells + dt_root_size_cells) * sizeof(__be32);
105 struct device_node *node, *child;
106 phys_addr_t base, size;
107 const __be32 *prop;
108 int len;
109
110 node = of_find_node_by_path("/reserved-memory");
> 111 if (node < 0) {
112 pr_err("Reserved memory: Did not find reserved-memory node\n");
113 return;
114 }
115
116 for_each_child_of_node(node, child) {
117 const char *uname;
118 struct reserved_mem *rmem;
119
120 if (!of_device_is_available(child))
121 continue;
122
123 prop = of_get_property(child, "reg", &len);
124 if (!prop) {
125 rmem = of_reserved_mem_lookup(child);
126 if (rmem)
127 rmem->dev_node = child;
128 continue;
129 }
130
131 uname = of_node_full_name(child);
132 if (len && len % t_len != 0) {
133 pr_err("Reserved memory: invalid reg property in '%s', skipping node.\n",
134 uname);
135 continue;
136 }
137
138 base = dt_mem_next_cell(dt_root_addr_cells, &prop);
139 size = dt_mem_next_cell(dt_root_size_cells, &prop);
140
141 if (size)
142 fdt_reserved_mem_save_node(child, uname, base, size);
143 }
144 }
145
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
On Fri, Jan 26, 2024 at 03:53:39PM -0800, Oreoluwa Babatunde wrote:
> The reserved_mem array is used to store data for the different
> reserved memory regions defined in the DT of a device. The array
> stores information such as region name, node, start-address, and size
> of the reserved memory regions.
>
> The array is currently statically allocated with a size of
> MAX_RESERVED_REGIONS(64). This means that any system that specifies a
> number of reserved memory regions greater than MAX_RESERVED_REGIONS(64)
> will not have enough space to store the information for all the regions.
>
> Therefore, this series extends the use of the static array for
> reserved_mem, and introduces a dynamically allocated array using
> memblock_alloc() based on the number of reserved memory regions
> specified in the DT.
>
> Some architectures such as arm64 require the page tables to be setup
> before memblock allocated memory is writable. Therefore, the dynamic
> allocation of the reserved_mem array will need to be done after the
> page tables have been setup on these architectures. In most cases that
> will be after paging_init().
>
> Reserved memory regions can be divided into 2 groups.
> i) Statically-placed reserved memory regions
> i.e. regions defined in the DT using the @reg property.
> ii) Dynamically-placed reserved memory regions.
> i.e. regions specified in the DT using the @alloc_ranges
> and @size properties.
>
> It is possible to call memblock_reserve() and memblock_mark_nomap() on
> the statically-placed reserved memory regions and not need to save them
> to the reserved_mem array until memory is allocated for it using
> memblock, which will be after the page tables have been setup.
> For the dynamically-placed reserved memory regions, it is not possible
> to wait to store its information because the starting address is
> allocated only at run time, and hence they need to be stored somewhere
> after they are allocated.
> Waiting until after the page tables have been setup to allocate memory
> for the dynamically-placed regions is also not an option because the
> allocations will come from memory that have already been added to the
> page tables, which is not good for memory that is supposed to be
> reserved and/or marked as nomap.
>
> Therefore, this series splits up the processing of the reserved memory
> regions into two stages, of which the first stage is carried out by
> early_init_fdt_scan_reserved_mem() and the second is carried out by
> fdt_init_reserved_mem().
>
> The early_init_fdt_scan_reserved_mem(), which is called before the page
> tables are setup is used to:
> 1. Call memblock_reserve() and memblock_mark_nomap() on all the
> statically-placed reserved memory regions as needed.
> 2. Allocate memory from memblock for the dynamically-placed reserved
> memory regions and store them in the static array for reserved_mem.
> memblock_reserve() and memblock_mark_nomap() are also called as
> needed on all the memory allocated for the dynamically-placed
> regions.
> 3. Count the total number of reserved memory regions found in the DT.
>
> fdt_init_reserved_mem(), which should be called after the page tables
> have been setup, is used to carry out the following:
> 1. Allocate memory for the reserved_mem array based on the number of
> reserved memory regions counted as mentioned above.
> 2. Copy all the information for the dynamically-placed reserved memory
> regions from the static array into the new allocated memory for the
> reserved_mem array.
> 3. Add the information for the statically-placed reserved memory into
> reserved_mem array.
> 4. Run the region specific init functions for each of the reserve memory
> regions saved in the reserved_mem array.
I don't see the need for fdt_init_reserved_mem() to be explicitly called
by arch code. I said this already, but that can be done at the same time
as unflattening the DT. The same conditions are needed for both: we need
to be able to allocate memory from memblock.
To put it another way, if fdt_init_reserved_mem() can be called "early",
then unflattening could be moved earlier as well. Though I don't think
we should optimize that. I'd rather see all arches call the DT functions
at the same stages.
> Once the above steps have been completed and the init process is done
> running, the original statically allocated reserved_mem array of size
> MAX_RESERVED_REGIONS(64) will be automatically freed back to buddy
> because it is no longer needed. This is done by marking the array as an
> "__initdata" object in Patch 0018.
>
> Note:
>
> - Per Architecture, this series is effectively only 10 patches. The
> code for each architecture is split up into separate patches to
> allow each architecture to be tested independently of changes from
> other architectures. Should this series be accepted, this should
> allow for each arcitecture change to be picked up independently as
> well.
Only if patches 1 and 2 are accepted in one cycle and the arch ones in
the next cycle. No need for that though, I can take the whole thing
(when it's ready).
>
> Patch 0001: Splits up the processing of the reserved memory regions
> between early_init_fdt_scan_reserved_mem and fdt_init_reserved_mem.
>
> Patch 0002: Introduces a copy of early_init_fdt_scan_reserved_mem()
> which is used to separate it from fdt_init_reserved_mem() so that the
> two functions can be called independently of each other.
>
> Patch 0003 - Patch 0016: Duplicated change for each architecture to
> call early_init_fdt_scan_reserved_mem() and fdt_init_reserved_mem()
> at their appropriate locations. Here fdt_init_reserved_mem() is called
> either before of after the page tables have been setup depending on
> the architecture requirements.
>
> Patch 0017: Deletes the early_init_fdt_scan_reserved_mem() function
> since all architectures are now using the copy introduced in
> Patch 0002.
>
> Patch 0018: Dynamically allocate memory for the reserved_mem array
> based on the total number of reserved memory regions specified in the
> DT.
>
> Patch 0019 - Patch 0029: Duplicated change for each architecture to
> move the fdt_init_reserved_mem() function call to below the
> unflatten_devicetree() function call. This is so that the unflatten
> devicetree APIs can be used to process the reserved memory regions.
>
> Patch 0030: Make code changes to start using the unflatten devicetree
> APIs to access the reserved memory regions defined in the DT.
>
> Patch 0031: Rename fdt_* functions as dt_* to refelct that the
> flattened devicetree (fdt) APIs have been replaced with the unflatten
> devicetree APIs.
>
> Patch 0032 - Patch 0045: Duplicated change for each architecture to
> switch from the use of fdt_init_reserved_mem() to
> dt_init_reserved_mem(), which is the same function but the later uses
> the unflatten devicetree APIs.
>
> Patch 0046: Delete the fdt_init_reserved_mem() function as all
> architectures have switched to using dt_init_reserved_mem() which was
> introduced in Patch 0031.
>
> - The limitation to this approach is that there is still a limit of
> 64 for dynamically-placed reserved memory regions. But from my current
> analysis, these types of reserved memory regions are generally less
> in number when compared to the statically-placed reserved memory
> regions.
>
> - I have looked through all architectures and placed the call to
> memblock_alloc() for the reserved_mem array at points where I
> believe memblock allocated memory are available to be written to.
> I currently only have access to an arm64 device and this is where I am
> testing the functionality of this series. Hence, I will need help from
> architecture maintainers to test this series on other architectures to
> ensure that the code is functioning properly on there.
>
> Previous patch revisions:
> 1. [RFC V1 Patchset]:
> https://lore.kernel.org/all/[email protected]/
>
> 2. [RFC V2 Patchset]:
> https://lore.kernel.org/all/[email protected]/
> - Extend changes to all other relevant architectures.
> - Add code to use unflatten devicetree APIs to process the reserved
> memory regions.
Dropping RFC does not make this v1. RFC is a state of the patches not a
version.
Rob
On Fri, Jan 26, 2024 at 03:53:53PM -0800, Oreoluwa Babatunde wrote:
> Call early_fdt_scan_reserved_mem() in place of
> early_init_fdt_scan_reserved_mem() to carry out the first stage of the
> reserved memory processing only.
>
> The early_fdt_scan_reserved_mem() function is used to scan through the
> DT and mark all the reserved memory regions as reserved or nomap as
> needed, as well as allocate the memory required by the
> dynamically-placed
> reserved memory regions.
>
> The second stage of the reserved memory processing is done by
> fdt_init_reserved_mem(). This function is used to store the information
> of the statically-placed reserved memory nodes in the reserved_mem
> array as well as call the region specific initialization function on all
> the stored reserved memory regions.
>
> The call to fdt_init_reserved_mem() is placed right after
> early_fdt_scan_reserved_mem() because memblock allocated memory should
> already be writable at this point.
>
> Signed-off-by: Oreoluwa Babatunde <[email protected]>
> ---
> arch/sh/boards/of-generic.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/arch/sh/boards/of-generic.c b/arch/sh/boards/of-generic.c
> index f7f3e618e85b..7bec409f077c 100644
> --- a/arch/sh/boards/of-generic.c
> +++ b/arch/sh/boards/of-generic.c
> @@ -8,6 +8,7 @@
> #include <linux/of.h>
> #include <linux/of_clk.h>
> #include <linux/of_fdt.h>
> +#include <linux/of_reserved_mem.h>
> #include <linux/clocksource.h>
> #include <linux/irqchip.h>
> #include <asm/machvec.h>
> @@ -110,7 +111,8 @@ static int noopi(void)
> static void __init sh_of_mem_reserve(void)
> {
> early_init_fdt_reserve_self();
> - early_init_fdt_scan_reserved_mem();
> + early_fdt_scan_reserved_mem();
> + fdt_init_reserved_mem();
Looking at the sh code, there's an existing problem with the order of
init. This is called from paging_init() and is done after unflattening
and copying the DT. That means the kernel could freely allocate memory
for the DT in a reserved region.
Rob
Hi Oreoluwa,
kernel test robot noticed the following build warnings:
[auto build test WARNING on robh/for-next]
[also build test WARNING on arm64/for-next/core vgupta-arc/for-curr powerpc/next powerpc/fixes jcmvbkbc-xtensa/xtensa-for-next linus/master v6.8-rc2 next-20240131]
[cannot apply to vgupta-arc/for-next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Oreoluwa-Babatunde/of-reserved_mem-Change-the-order-that-reserved_mem-regions-are-stored/20240127-081735
base: https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git for-next
patch link: https://lore.kernel.org/r/20240126235425.12233-31-quic_obabatun%40quicinc.com
patch subject: [PATCH 30/46] of: reserved_mem: Add code to use unflattened DT for reserved_mem nodes
config: i386-randconfig-141-20240128 (https://download.01.org/0day-ci/archive/20240201/[email protected]/config)
compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
smatch warnings:
drivers/of/of_reserved_mem.c:111 dt_scan_reserved_mem_reg_nodes() warn: unsigned 'node' is never less than zero.
vim +/node +111 drivers/of/of_reserved_mem.c
98
99 /*
100 * Save the reserved_mem reg nodes in the reserved_mem array
101 */
102 static void __init dt_scan_reserved_mem_reg_nodes(void)
103 {
104 int t_len = (dt_root_addr_cells + dt_root_size_cells) * sizeof(__be32);
105 struct device_node *node, *child;
106 phys_addr_t base, size;
107 const __be32 *prop;
108 int len;
109
110 node = of_find_node_by_path("/reserved-memory");
> 111 if (node < 0) {
112 pr_err("Reserved memory: Did not find reserved-memory node\n");
113 return;
114 }
115
116 for_each_child_of_node(node, child) {
117 const char *uname;
118 struct reserved_mem *rmem;
119
120 if (!of_device_is_available(child))
121 continue;
122
123 prop = of_get_property(child, "reg", &len);
124 if (!prop) {
125 rmem = of_reserved_mem_lookup(child);
126 if (rmem)
127 rmem->dev_node = child;
128 continue;
129 }
130
131 uname = of_node_full_name(child);
132 if (len && len % t_len != 0) {
133 pr_err("Reserved memory: invalid reg property in '%s', skipping node.\n",
134 uname);
135 continue;
136 }
137
138 base = dt_mem_next_cell(dt_root_addr_cells, &prop);
139 size = dt_mem_next_cell(dt_root_size_cells, &prop);
140
141 if (size)
142 fdt_reserved_mem_save_node(child, uname, base, size);
143 }
144 }
145
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
On 1/30/2024 4:07 PM, Rob Herring wrote:
> On Fri, Jan 26, 2024 at 03:53:39PM -0800, Oreoluwa Babatunde wrote:
>> The reserved_mem array is used to store data for the different
>> reserved memory regions defined in the DT of a device. The array
>> stores information such as region name, node, start-address, and size
>> of the reserved memory regions.
>>
>> The array is currently statically allocated with a size of
>> MAX_RESERVED_REGIONS(64). This means that any system that specifies a
>> number of reserved memory regions greater than MAX_RESERVED_REGIONS(64)
>> will not have enough space to store the information for all the regions.
>>
>> Therefore, this series extends the use of the static array for
>> reserved_mem, and introduces a dynamically allocated array using
>> memblock_alloc() based on the number of reserved memory regions
>> specified in the DT.
>>
>> Some architectures such as arm64 require the page tables to be setup
>> before memblock allocated memory is writable. Therefore, the dynamic
>> allocation of the reserved_mem array will need to be done after the
>> page tables have been setup on these architectures. In most cases that
>> will be after paging_init().
>>
>> Reserved memory regions can be divided into 2 groups.
>> i) Statically-placed reserved memory regions
>> i.e. regions defined in the DT using the @reg property.
>> ii) Dynamically-placed reserved memory regions.
>> i.e. regions specified in the DT using the @alloc_ranges
>> and @size properties.
>>
>> It is possible to call memblock_reserve() and memblock_mark_nomap() on
>> the statically-placed reserved memory regions and not need to save them
>> to the reserved_mem array until memory is allocated for it using
>> memblock, which will be after the page tables have been setup.
>> For the dynamically-placed reserved memory regions, it is not possible
>> to wait to store its information because the starting address is
>> allocated only at run time, and hence they need to be stored somewhere
>> after they are allocated.
>> Waiting until after the page tables have been setup to allocate memory
>> for the dynamically-placed regions is also not an option because the
>> allocations will come from memory that have already been added to the
>> page tables, which is not good for memory that is supposed to be
>> reserved and/or marked as nomap.
>>
>> Therefore, this series splits up the processing of the reserved memory
>> regions into two stages, of which the first stage is carried out by
>> early_init_fdt_scan_reserved_mem() and the second is carried out by
>> fdt_init_reserved_mem().
>>
>> The early_init_fdt_scan_reserved_mem(), which is called before the page
>> tables are setup is used to:
>> 1. Call memblock_reserve() and memblock_mark_nomap() on all the
>> statically-placed reserved memory regions as needed.
>> 2. Allocate memory from memblock for the dynamically-placed reserved
>> memory regions and store them in the static array for reserved_mem.
>> memblock_reserve() and memblock_mark_nomap() are also called as
>> needed on all the memory allocated for the dynamically-placed
>> regions.
>> 3. Count the total number of reserved memory regions found in the DT.
>>
>> fdt_init_reserved_mem(), which should be called after the page tables
>> have been setup, is used to carry out the following:
>> 1. Allocate memory for the reserved_mem array based on the number of
>> reserved memory regions counted as mentioned above.
>> 2. Copy all the information for the dynamically-placed reserved memory
>> regions from the static array into the new allocated memory for the
>> reserved_mem array.
>> 3. Add the information for the statically-placed reserved memory into
>> reserved_mem array.
>> 4. Run the region specific init functions for each of the reserve memory
>> regions saved in the reserved_mem array.
> I don't see the need for fdt_init_reserved_mem() to be explicitly called
> by arch code. I said this already, but that can be done at the same time
> as unflattening the DT. The same conditions are needed for both: we need
> to be able to allocate memory from memblock.
>
> To put it another way, if fdt_init_reserved_mem() can be called "early",
> then unflattening could be moved earlier as well. Though I don't think
> we should optimize that. I'd rather see all arches call the DT functions
> at the same stages.
Hi Rob,
The reason we moved fdt_init_reserved_mem() back into the arch specific code
was because we realized that there was no apparently obvious way to call
early_init_fdt_scan_reserved_mem() and fdt_init_reserved_mem() in the correct
order that will work for all archs if we placed fdt_init_reserved_mem() inside the
unflatten_devicetree() function.
early_init_fdt_scan_reserved_mem() needs to be
called first before fdt_init_reserved_mem(). But on some archs,
unflatten_devicetree() is called before early_init_fdt_scan_reserved_mem(), which
means that if we have fdt_init_reserved_mem() inside the unflatten_devicetree()
function, it will be called before early_init_fdt_scan_reserved_mem().
This is connected to your other comments on Patch 7 & Patch 14.
I agree, unflatten_devicetree() should NOT be getting called before we reserve
memory for the reserved memory regions because that could cause memory to be
allocated from regions that should be reserved.
Hence, resolving this issue should allow us to call fdt_init_reserved_mem() from
the unflatten_devicetree() function without it changing the order that we are
trying to have.
I will work on implementing this and send another revision.
>
>> Once the above steps have been completed and the init process is done
>> running, the original statically allocated reserved_mem array of size
>> MAX_RESERVED_REGIONS(64) will be automatically freed back to buddy
>> because it is no longer needed. This is done by marking the array as an
>> "__initdata" object in Patch 0018.
>>
>> Note:
>>
>> - Per Architecture, this series is effectively only 10 patches. The
>> code for each architecture is split up into separate patches to
>> allow each architecture to be tested independently of changes from
>> other architectures. Should this series be accepted, this should
>> allow for each arcitecture change to be picked up independently as
>> well.
> Only if patches 1 and 2 are accepted in one cycle and the arch ones in
> the next cycle. No need for that though, I can take the whole thing
> (when it's ready).
ack.
>
>> Patch 0001: Splits up the processing of the reserved memory regions
>> between early_init_fdt_scan_reserved_mem and fdt_init_reserved_mem.
>>
>> Patch 0002: Introduces a copy of early_init_fdt_scan_reserved_mem()
>> which is used to separate it from fdt_init_reserved_mem() so that the
>> two functions can be called independently of each other.
>>
>> Patch 0003 - Patch 0016: Duplicated change for each architecture to
>> call early_init_fdt_scan_reserved_mem() and fdt_init_reserved_mem()
>> at their appropriate locations. Here fdt_init_reserved_mem() is called
>> either before of after the page tables have been setup depending on
>> the architecture requirements.
>>
>> Patch 0017: Deletes the early_init_fdt_scan_reserved_mem() function
>> since all architectures are now using the copy introduced in
>> Patch 0002.
>>
>> Patch 0018: Dynamically allocate memory for the reserved_mem array
>> based on the total number of reserved memory regions specified in the
>> DT.
>>
>> Patch 0019 - Patch 0029: Duplicated change for each architecture to
>> move the fdt_init_reserved_mem() function call to below the
>> unflatten_devicetree() function call. This is so that the unflatten
>> devicetree APIs can be used to process the reserved memory regions.
>>
>> Patch 0030: Make code changes to start using the unflatten devicetree
>> APIs to access the reserved memory regions defined in the DT.
>>
>> Patch 0031: Rename fdt_* functions as dt_* to refelct that the
>> flattened devicetree (fdt) APIs have been replaced with the unflatten
>> devicetree APIs.
>>
>> Patch 0032 - Patch 0045: Duplicated change for each architecture to
>> switch from the use of fdt_init_reserved_mem() to
>> dt_init_reserved_mem(), which is the same function but the later uses
>> the unflatten devicetree APIs.
>>
>> Patch 0046: Delete the fdt_init_reserved_mem() function as all
>> architectures have switched to using dt_init_reserved_mem() which was
>> introduced in Patch 0031.
>>
>> - The limitation to this approach is that there is still a limit of
>> 64 for dynamically-placed reserved memory regions. But from my current
>> analysis, these types of reserved memory regions are generally less
>> in number when compared to the statically-placed reserved memory
>> regions.
>>
>> - I have looked through all architectures and placed the call to
>> memblock_alloc() for the reserved_mem array at points where I
>> believe memblock allocated memory are available to be written to.
>> I currently only have access to an arm64 device and this is where I am
>> testing the functionality of this series. Hence, I will need help from
>> architecture maintainers to test this series on other architectures to
>> ensure that the code is functioning properly on there.
>>
>> Previous patch revisions:
>> 1. [RFC V1 Patchset]:
>> https://lore.kernel.org/all/[email protected]/
>>
>> 2. [RFC V2 Patchset]:
>> https://lore.kernel.org/all/[email protected]/
>> - Extend changes to all other relevant architectures.
>> - Add code to use unflatten devicetree APIs to process the reserved
>> memory regions.
> Dropping RFC does not make this v1. RFC is a state of the patches not a
> version.
ack.
Thank you for your comments!
Oreoluwa
On 1/31/2024 7:41 AM, Rob Herring wrote:
> On Fri, Jan 26, 2024 at 03:53:53PM -0800, Oreoluwa Babatunde wrote:
>> Call early_fdt_scan_reserved_mem() in place of
>> early_init_fdt_scan_reserved_mem() to carry out the first stage of the
>> reserved memory processing only.
>>
>> The early_fdt_scan_reserved_mem() function is used to scan through the
>> DT and mark all the reserved memory regions as reserved or nomap as
>> needed, as well as allocate the memory required by the
>> dynamically-placed
>> reserved memory regions.
>>
>> The second stage of the reserved memory processing is done by
>> fdt_init_reserved_mem(). This function is used to store the information
>> of the statically-placed reserved memory nodes in the reserved_mem
>> array as well as call the region specific initialization function on all
>> the stored reserved memory regions.
>>
>> The call to fdt_init_reserved_mem() is placed right after
>> early_fdt_scan_reserved_mem() because memblock allocated memory should
>> already be writable at this point.
>>
>> Signed-off-by: Oreoluwa Babatunde <[email protected]>
>> ---
>> arch/sh/boards/of-generic.c | 4 +++-
>> 1 file changed, 3 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/sh/boards/of-generic.c b/arch/sh/boards/of-generic.c
>> index f7f3e618e85b..7bec409f077c 100644
>> --- a/arch/sh/boards/of-generic.c
>> +++ b/arch/sh/boards/of-generic.c
>> @@ -8,6 +8,7 @@
>> #include <linux/of.h>
>> #include <linux/of_clk.h>
>> #include <linux/of_fdt.h>
>> +#include <linux/of_reserved_mem.h>
>> #include <linux/clocksource.h>
>> #include <linux/irqchip.h>
>> #include <asm/machvec.h>
>> @@ -110,7 +111,8 @@ static int noopi(void)
>> static void __init sh_of_mem_reserve(void)
>> {
>> early_init_fdt_reserve_self();
>> - early_init_fdt_scan_reserved_mem();
>> + early_fdt_scan_reserved_mem();
>> + fdt_init_reserved_mem();
> Looking at the sh code, there's an existing problem with the order of
> init. This is called from paging_init() and is done after unflattening
> and copying the DT. That means the kernel could freely allocate memory
> for the DT in a reserved region.
>
> Rob
Hi Rob,
Yes I agree! I can try to restructure the code to address
this. I think we should be able to move the call to
early_init_fdt_scan_reserved_mem() higher in the init
sequence without having any issues.
Will try this out and see.
Regards,
Oreoluwa
On Thu, Feb 01, 2024 at 09:08:06AM -0800, Oreoluwa Babatunde wrote:
>
> On 1/30/2024 4:07 PM, Rob Herring wrote:
> > On Fri, Jan 26, 2024 at 03:53:39PM -0800, Oreoluwa Babatunde wrote:
> >> The reserved_mem array is used to store data for the different
> >> reserved memory regions defined in the DT of a device. The array
> >> stores information such as region name, node, start-address, and size
> >> of the reserved memory regions.
> >>
> >> The array is currently statically allocated with a size of
> >> MAX_RESERVED_REGIONS(64). This means that any system that specifies a
> >> number of reserved memory regions greater than MAX_RESERVED_REGIONS(64)
> >> will not have enough space to store the information for all the regions.
> >>
> >> Therefore, this series extends the use of the static array for
> >> reserved_mem, and introduces a dynamically allocated array using
> >> memblock_alloc() based on the number of reserved memory regions
> >> specified in the DT.
> >>
> >> Some architectures such as arm64 require the page tables to be setup
> >> before memblock allocated memory is writable. Therefore, the dynamic
> >> allocation of the reserved_mem array will need to be done after the
> >> page tables have been setup on these architectures. In most cases that
> >> will be after paging_init().
> >>
> >> Reserved memory regions can be divided into 2 groups.
> >> i) Statically-placed reserved memory regions
> >> i.e. regions defined in the DT using the @reg property.
> >> ii) Dynamically-placed reserved memory regions.
> >> i.e. regions specified in the DT using the @alloc_ranges
> >> and @size properties.
> >>
> >> It is possible to call memblock_reserve() and memblock_mark_nomap() on
> >> the statically-placed reserved memory regions and not need to save them
> >> to the reserved_mem array until memory is allocated for it using
> >> memblock, which will be after the page tables have been setup.
> >> For the dynamically-placed reserved memory regions, it is not possible
> >> to wait to store its information because the starting address is
> >> allocated only at run time, and hence they need to be stored somewhere
> >> after they are allocated.
> >> Waiting until after the page tables have been setup to allocate memory
> >> for the dynamically-placed regions is also not an option because the
> >> allocations will come from memory that have already been added to the
> >> page tables, which is not good for memory that is supposed to be
> >> reserved and/or marked as nomap.
> >>
> >> Therefore, this series splits up the processing of the reserved memory
> >> regions into two stages, of which the first stage is carried out by
> >> early_init_fdt_scan_reserved_mem() and the second is carried out by
> >> fdt_init_reserved_mem().
> >>
> >> The early_init_fdt_scan_reserved_mem(), which is called before the page
> >> tables are setup is used to:
> >> 1. Call memblock_reserve() and memblock_mark_nomap() on all the
> >> statically-placed reserved memory regions as needed.
> >> 2. Allocate memory from memblock for the dynamically-placed reserved
> >> memory regions and store them in the static array for reserved_mem.
> >> memblock_reserve() and memblock_mark_nomap() are also called as
> >> needed on all the memory allocated for the dynamically-placed
> >> regions.
> >> 3. Count the total number of reserved memory regions found in the DT.
> >>
> >> fdt_init_reserved_mem(), which should be called after the page tables
> >> have been setup, is used to carry out the following:
> >> 1. Allocate memory for the reserved_mem array based on the number of
> >> reserved memory regions counted as mentioned above.
> >> 2. Copy all the information for the dynamically-placed reserved memory
> >> regions from the static array into the new allocated memory for the
> >> reserved_mem array.
> >> 3. Add the information for the statically-placed reserved memory into
> >> reserved_mem array.
> >> 4. Run the region specific init functions for each of the reserve memory
> >> regions saved in the reserved_mem array.
> > I don't see the need for fdt_init_reserved_mem() to be explicitly called
> > by arch code. I said this already, but that can be done at the same time
> > as unflattening the DT. The same conditions are needed for both: we need
> > to be able to allocate memory from memblock.
> >
> > To put it another way, if fdt_init_reserved_mem() can be called "early",
> > then unflattening could be moved earlier as well. Though I don't think
> > we should optimize that. I'd rather see all arches call the DT functions
> > at the same stages.
> Hi Rob,
>
> The reason we moved fdt_init_reserved_mem() back into the arch specific code
> was because we realized that there was no apparently obvious way to call
> early_init_fdt_scan_reserved_mem() and fdt_init_reserved_mem() in the correct
> order that will work for all archs if we placed fdt_init_reserved_mem() inside the
> unflatten_devicetree() function.
>
> early_init_fdt_scan_reserved_mem() needs to be
> called first before fdt_init_reserved_mem(). But on some archs,
> unflatten_devicetree() is called before early_init_fdt_scan_reserved_mem(), which
> means that if we have fdt_init_reserved_mem() inside the unflatten_devicetree()
> function, it will be called before early_init_fdt_scan_reserved_mem().
>
> This is connected to your other comments on Patch 7 & Patch 14.
> I agree, unflatten_devicetree() should NOT be getting called before we reserve
> memory for the reserved memory regions because that could cause memory to be
> allocated from regions that should be reserved.
>
> Hence, resolving this issue should allow us to call fdt_init_reserved_mem() from
> the? unflatten_devicetree() function without it changing the order that we are
> trying to have.
There's one issue I've found which is unflatten_device_tree() isn't
called for ACPI case on arm64. Turns out we need /reserved-memory
handled in that case too. However, I think we're going to change
calling unflatten_device_tree() unconditionally for another reason[1].
[1] https://lore.kernel.org/all/[email protected]/
>
> I will work on implementing this and send another revision.
I think we should go with a simpler route that's just copy the an
initial array in initdata to a properly sized, allocated array like the
patch below. Of course it will need some arch fixes and a follow-on
patch to increase the initial array size.
8<--------------------------------------------------------------------
From: Rob Herring <[email protected]>
Date: Wed, 31 Jan 2024 16:26:23 -0600
Subject: [PATCH] of: reserved-mem: Re-allocate reserved_mem array to actual
size
In preparation to increase the static reserved_mem array size yet again,
copy the initial array to an allocated array sized based on the actual
size needed. Now increasing the the size of the static reserved_mem
array only eats up the initdata space. For platforms with reasonable
number of reserved regions, we have a net gain in free memory.
In order to do memblock allocations, fdt_init_reserved_mem() is moved a
bit later to unflatten_device_tree(). On some arches this is effectively
a nop.
Signed-off-by: Rob Herring <[email protected]>
---
RFC as this is compile tested only. This is an alternative to this
series[1].
[1] https://lore.kernel.org/all/[email protected]/
---
drivers/of/fdt.c | 4 ++--
drivers/of/of_reserved_mem.c | 18 +++++++++++++-----
2 files changed, 15 insertions(+), 7 deletions(-)
diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
index bf502ba8da95..14360f5191ae 100644
--- a/drivers/of/fdt.c
+++ b/drivers/of/fdt.c
@@ -645,8 +645,6 @@ void __init early_init_fdt_scan_reserved_mem(void)
break;
memblock_reserve(base, size);
}
-
- fdt_init_reserved_mem();
}
/**
@@ -1328,6 +1326,8 @@ bool __init early_init_dt_scan(void *params)
*/
void __init unflatten_device_tree(void)
{
+ fdt_init_reserved_mem();
+
__unflatten_device_tree(initial_boot_params, NULL, &of_root,
early_init_dt_alloc_memory_arch, false);
diff --git a/drivers/of/of_reserved_mem.c b/drivers/of/of_reserved_mem.c
index 7ec94cfcbddb..ae323d6b25ad 100644
--- a/drivers/of/of_reserved_mem.c
+++ b/drivers/of/of_reserved_mem.c
@@ -27,7 +27,8 @@
#include "of_private.h"
#define MAX_RESERVED_REGIONS 64
-static struct reserved_mem reserved_mem[MAX_RESERVED_REGIONS];
+static struct reserved_mem reserved_mem[MAX_RESERVED_REGIONS] __initdata;
+static struct reserved_mem *reserved_mem_p;
static int reserved_mem_count;
static int __init early_init_dt_alloc_reserved_memory_arch(phys_addr_t size,
@@ -354,6 +355,13 @@ void __init fdt_init_reserved_mem(void)
}
}
}
+
+ reserved_mem_p = memblock_alloc(sizeof(struct reserved_mem) * reserved_mem_count,
+ sizeof(struct reserved_mem));
+ if (WARN(!reserved_mem_p, "of: reserved-memory allocation failed, continuing with __initdata array!\n"))
+ reserved_mem_p = reserved_mem;
+ else
+ memcpy(reserved_mem_p, reserved_mem, sizeof(struct reserved_mem) * reserved_mem_count);
}
static inline struct reserved_mem *__find_rmem(struct device_node *node)
@@ -364,8 +372,8 @@ static inline struct reserved_mem *__find_rmem(struct device_node *node)
return NULL;
for (i = 0; i < reserved_mem_count; i++)
- if (reserved_mem[i].phandle == node->phandle)
- return &reserved_mem[i];
+ if (reserved_mem_p[i].phandle == node->phandle)
+ return &reserved_mem_p[i];
return NULL;
}
@@ -507,8 +515,8 @@ struct reserved_mem *of_reserved_mem_lookup(struct device_node *np)
name = kbasename(np->full_name);
for (i = 0; i < reserved_mem_count; i++)
- if (!strcmp(reserved_mem[i].name, name))
- return &reserved_mem[i];
+ if (!strcmp(reserved_mem_p[i].name, name))
+ return &reserved_mem_p[i];
return NULL;
}
--
2.43.0
On 2/1/2024 11:46 AM, Rob Herring wrote:
> On Thu, Feb 01, 2024 at 09:08:06AM -0800, Oreoluwa Babatunde wrote:
>> On 1/30/2024 4:07 PM, Rob Herring wrote:
>>> On Fri, Jan 26, 2024 at 03:53:39PM -0800, Oreoluwa Babatunde wrote:
>>>> The reserved_mem array is used to store data for the different
>>>> reserved memory regions defined in the DT of a device. The array
>>>> stores information such as region name, node, start-address, and size
>>>> of the reserved memory regions.
>>>>
>>>> The array is currently statically allocated with a size of
>>>> MAX_RESERVED_REGIONS(64). This means that any system that specifies a
>>>> number of reserved memory regions greater than MAX_RESERVED_REGIONS(64)
>>>> will not have enough space to store the information for all the regions.
>>>>
>>>> Therefore, this series extends the use of the static array for
>>>> reserved_mem, and introduces a dynamically allocated array using
>>>> memblock_alloc() based on the number of reserved memory regions
>>>> specified in the DT.
>>>>
>>>> Some architectures such as arm64 require the page tables to be setup
>>>> before memblock allocated memory is writable. Therefore, the dynamic
>>>> allocation of the reserved_mem array will need to be done after the
>>>> page tables have been setup on these architectures. In most cases that
>>>> will be after paging_init().
>>>>
>>>> Reserved memory regions can be divided into 2 groups.
>>>> i) Statically-placed reserved memory regions
>>>> i.e. regions defined in the DT using the @reg property.
>>>> ii) Dynamically-placed reserved memory regions.
>>>> i.e. regions specified in the DT using the @alloc_ranges
>>>> and @size properties.
>>>>
>>>> It is possible to call memblock_reserve() and memblock_mark_nomap() on
>>>> the statically-placed reserved memory regions and not need to save them
>>>> to the reserved_mem array until memory is allocated for it using
>>>> memblock, which will be after the page tables have been setup.
>>>> For the dynamically-placed reserved memory regions, it is not possible
>>>> to wait to store its information because the starting address is
>>>> allocated only at run time, and hence they need to be stored somewhere
>>>> after they are allocated.
>>>> Waiting until after the page tables have been setup to allocate memory
>>>> for the dynamically-placed regions is also not an option because the
>>>> allocations will come from memory that have already been added to the
>>>> page tables, which is not good for memory that is supposed to be
>>>> reserved and/or marked as nomap.
>>>>
>>>> Therefore, this series splits up the processing of the reserved memory
>>>> regions into two stages, of which the first stage is carried out by
>>>> early_init_fdt_scan_reserved_mem() and the second is carried out by
>>>> fdt_init_reserved_mem().
>>>>
>>>> The early_init_fdt_scan_reserved_mem(), which is called before the page
>>>> tables are setup is used to:
>>>> 1. Call memblock_reserve() and memblock_mark_nomap() on all the
>>>> statically-placed reserved memory regions as needed.
>>>> 2. Allocate memory from memblock for the dynamically-placed reserved
>>>> memory regions and store them in the static array for reserved_mem.
>>>> memblock_reserve() and memblock_mark_nomap() are also called as
>>>> needed on all the memory allocated for the dynamically-placed
>>>> regions.
>>>> 3. Count the total number of reserved memory regions found in the DT.
>>>>
>>>> fdt_init_reserved_mem(), which should be called after the page tables
>>>> have been setup, is used to carry out the following:
>>>> 1. Allocate memory for the reserved_mem array based on the number of
>>>> reserved memory regions counted as mentioned above.
>>>> 2. Copy all the information for the dynamically-placed reserved memory
>>>> regions from the static array into the new allocated memory for the
>>>> reserved_mem array.
>>>> 3. Add the information for the statically-placed reserved memory into
>>>> reserved_mem array.
>>>> 4. Run the region specific init functions for each of the reserve memory
>>>> regions saved in the reserved_mem array.
>>> I don't see the need for fdt_init_reserved_mem() to be explicitly called
>>> by arch code. I said this already, but that can be done at the same time
>>> as unflattening the DT. The same conditions are needed for both: we need
>>> to be able to allocate memory from memblock.
>>>
>>> To put it another way, if fdt_init_reserved_mem() can be called "early",
>>> then unflattening could be moved earlier as well. Though I don't think
>>> we should optimize that. I'd rather see all arches call the DT functions
>>> at the same stages.
>> Hi Rob,
>>
>> The reason we moved fdt_init_reserved_mem() back into the arch specific code
>> was because we realized that there was no apparently obvious way to call
>> early_init_fdt_scan_reserved_mem() and fdt_init_reserved_mem() in the correct
>> order that will work for all archs if we placed fdt_init_reserved_mem() inside the
>> unflatten_devicetree() function.
>>
>> early_init_fdt_scan_reserved_mem() needs to be
>> called first before fdt_init_reserved_mem(). But on some archs,
>> unflatten_devicetree() is called before early_init_fdt_scan_reserved_mem(), which
>> means that if we have fdt_init_reserved_mem() inside the unflatten_devicetree()
>> function, it will be called before early_init_fdt_scan_reserved_mem().
>>
>> This is connected to your other comments on Patch 7 & Patch 14.
>> I agree, unflatten_devicetree() should NOT be getting called before we reserve
>> memory for the reserved memory regions because that could cause memory to be
>> allocated from regions that should be reserved.
>>
>> Hence, resolving this issue should allow us to call fdt_init_reserved_mem() from
>> the unflatten_devicetree() function without it changing the order that we are
>> trying to have.
> There's one issue I've found which is unflatten_device_tree() isn't
> called for ACPI case on arm64. Turns out we need /reserved-memory
> handled in that case too. However, I think we're going to change
> calling unflatten_device_tree() unconditionally for another reason[1].
>
> [1] https://lore.kernel.org/all/[email protected]/
>
>> I will work on implementing this and send another revision.
> I think we should go with a simpler route that's just copy the an
> initial array in initdata to a properly sized, allocated array like the
> patch below. Of course it will need some arch fixes and a follow-on
> patch to increase the initial array size.
>
> 8<--------------------------------------------------------------------
> From: Rob Herring <[email protected]>
> Date: Wed, 31 Jan 2024 16:26:23 -0600
> Subject: [PATCH] of: reserved-mem: Re-allocate reserved_mem array to actual
> size
>
> In preparation to increase the static reserved_mem array size yet again,
> copy the initial array to an allocated array sized based on the actual
> size needed. Now increasing the the size of the static reserved_mem
> array only eats up the initdata space. For platforms with reasonable
> number of reserved regions, we have a net gain in free memory.
>
> In order to do memblock allocations, fdt_init_reserved_mem() is moved a
> bit later to unflatten_device_tree(). On some arches this is effectively
> a nop.
>
> Signed-off-by: Rob Herring <[email protected]>
> ---
> RFC as this is compile tested only. This is an alternative to this
> series[1].
>
> [1] https://lore.kernel.org/all/[email protected]/
> ---
> drivers/of/fdt.c | 4 ++--
> drivers/of/of_reserved_mem.c | 18 +++++++++++++-----
> 2 files changed, 15 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/of/fdt.c b/drivers/of/fdt.c
> index bf502ba8da95..14360f5191ae 100644
> --- a/drivers/of/fdt.c
> +++ b/drivers/of/fdt.c
> @@ -645,8 +645,6 @@ void __init early_init_fdt_scan_reserved_mem(void)
> break;
> memblock_reserve(base, size);
> }
> -
> - fdt_init_reserved_mem();
> }
>
> /**
> @@ -1328,6 +1326,8 @@ bool __init early_init_dt_scan(void *params)
> */
> void __init unflatten_device_tree(void)
> {
> + fdt_init_reserved_mem();
> +
> __unflatten_device_tree(initial_boot_params, NULL, &of_root,
> early_init_dt_alloc_memory_arch, false);
>
> diff --git a/drivers/of/of_reserved_mem.c b/drivers/of/of_reserved_mem.c
> index 7ec94cfcbddb..ae323d6b25ad 100644
> --- a/drivers/of/of_reserved_mem.c
> +++ b/drivers/of/of_reserved_mem.c
> @@ -27,7 +27,8 @@
> #include "of_private.h"
>
> #define MAX_RESERVED_REGIONS 64
> -static struct reserved_mem reserved_mem[MAX_RESERVED_REGIONS];
> +static struct reserved_mem reserved_mem[MAX_RESERVED_REGIONS] __initdata;
> +static struct reserved_mem *reserved_mem_p;
> static int reserved_mem_count;
>
> static int __init early_init_dt_alloc_reserved_memory_arch(phys_addr_t size,
> @@ -354,6 +355,13 @@ void __init fdt_init_reserved_mem(void)
> }
> }
> }
> +
> + reserved_mem_p = memblock_alloc(sizeof(struct reserved_mem) * reserved_mem_count,
> + sizeof(struct reserved_mem));
> + if (WARN(!reserved_mem_p, "of: reserved-memory allocation failed, continuing with __initdata array!\n"))
> + reserved_mem_p = reserved_mem;
> + else
> + memcpy(reserved_mem_p, reserved_mem, sizeof(struct reserved_mem) * reserved_mem_count);
> }
>
> static inline struct reserved_mem *__find_rmem(struct device_node *node)
> @@ -364,8 +372,8 @@ static inline struct reserved_mem *__find_rmem(struct device_node *node)
> return NULL;
>
> for (i = 0; i < reserved_mem_count; i++)
> - if (reserved_mem[i].phandle == node->phandle)
> - return &reserved_mem[i];
> + if (reserved_mem_p[i].phandle == node->phandle)
> + return &reserved_mem_p[i];
> return NULL;
> }
>
> @@ -507,8 +515,8 @@ struct reserved_mem *of_reserved_mem_lookup(struct device_node *np)
>
> name = kbasename(np->full_name);
> for (i = 0; i < reserved_mem_count; i++)
> - if (!strcmp(reserved_mem[i].name, name))
> - return &reserved_mem[i];
> + if (!strcmp(reserved_mem_p[i].name, name))
> + return &reserved_mem_p[i];
>
> return NULL;
> }
Hi Rob,
One thing that could come up with this is that memory
for the dynamically-placed reserved memory regions
won't be allocated until we call fdt_init_reserved_mem().
(i.e. reserved memory regions defined using @alloc-ranges
and @size properties)
Since fdt_init_reserved_mem() is now being called from
unflatten_device_tree(), the page tables would have been
setup on most architectures, which means we will be
allocating from memory that have already been mapped.
Could this be an issue for memory that is supposed to be
reserved? Especially for the regions that are specified as
no-map?
On Thu, Feb 01, 2024 at 01:10:18PM -0800, Oreoluwa Babatunde wrote:
>
> On 2/1/2024 11:46 AM, Rob Herring wrote:
> > On Thu, Feb 01, 2024 at 09:08:06AM -0800, Oreoluwa Babatunde wrote:
> >> On 1/30/2024 4:07 PM, Rob Herring wrote:
> >>> On Fri, Jan 26, 2024 at 03:53:39PM -0800, Oreoluwa Babatunde wrote:
> >>>> The reserved_mem array is used to store data for the different
> >>>> reserved memory regions defined in the DT of a device. The array
> >>>> stores information such as region name, node, start-address, and size
> >>>> of the reserved memory regions.
> >>>>
> >>>> The array is currently statically allocated with a size of
> >>>> MAX_RESERVED_REGIONS(64). This means that any system that specifies a
> >>>> number of reserved memory regions greater than MAX_RESERVED_REGIONS(64)
> >>>> will not have enough space to store the information for all the regions.
> >>>>
> >>>> Therefore, this series extends the use of the static array for
> >>>> reserved_mem, and introduces a dynamically allocated array using
> >>>> memblock_alloc() based on the number of reserved memory regions
> >>>> specified in the DT.
> >>>>
> >>>> Some architectures such as arm64 require the page tables to be setup
> >>>> before memblock allocated memory is writable. Therefore, the dynamic
> >>>> allocation of the reserved_mem array will need to be done after the
> >>>> page tables have been setup on these architectures. In most cases that
> >>>> will be after paging_init().
> >>>>
> >>>> Reserved memory regions can be divided into 2 groups.
> >>>> i) Statically-placed reserved memory regions
> >>>> i.e. regions defined in the DT using the @reg property.
> >>>> ii) Dynamically-placed reserved memory regions.
> >>>> i.e. regions specified in the DT using the @alloc_ranges
> >>>> and @size properties.
> >>>>
> >>>> It is possible to call memblock_reserve() and memblock_mark_nomap() on
> >>>> the statically-placed reserved memory regions and not need to save them
> >>>> to the reserved_mem array until memory is allocated for it using
> >>>> memblock, which will be after the page tables have been setup.
> >>>> For the dynamically-placed reserved memory regions, it is not possible
> >>>> to wait to store its information because the starting address is
> >>>> allocated only at run time, and hence they need to be stored somewhere
> >>>> after they are allocated.
> >>>> Waiting until after the page tables have been setup to allocate memory
> >>>> for the dynamically-placed regions is also not an option because the
> >>>> allocations will come from memory that have already been added to the
> >>>> page tables, which is not good for memory that is supposed to be
> >>>> reserved and/or marked as nomap.
> >>>>
> >>>> Therefore, this series splits up the processing of the reserved memory
> >>>> regions into two stages, of which the first stage is carried out by
> >>>> early_init_fdt_scan_reserved_mem() and the second is carried out by
> >>>> fdt_init_reserved_mem().
> >>>>
> >>>> The early_init_fdt_scan_reserved_mem(), which is called before the page
> >>>> tables are setup is used to:
> >>>> 1. Call memblock_reserve() and memblock_mark_nomap() on all the
> >>>> statically-placed reserved memory regions as needed.
> >>>> 2. Allocate memory from memblock for the dynamically-placed reserved
> >>>> memory regions and store them in the static array for reserved_mem.
> >>>> memblock_reserve() and memblock_mark_nomap() are also called as
> >>>> needed on all the memory allocated for the dynamically-placed
> >>>> regions.
> >>>> 3. Count the total number of reserved memory regions found in the DT.
> >>>>
> >>>> fdt_init_reserved_mem(), which should be called after the page tables
> >>>> have been setup, is used to carry out the following:
> >>>> 1. Allocate memory for the reserved_mem array based on the number of
> >>>> reserved memory regions counted as mentioned above.
> >>>> 2. Copy all the information for the dynamically-placed reserved memory
> >>>> regions from the static array into the new allocated memory for the
> >>>> reserved_mem array.
> >>>> 3. Add the information for the statically-placed reserved memory into
> >>>> reserved_mem array.
> >>>> 4. Run the region specific init functions for each of the reserve memory
> >>>> regions saved in the reserved_mem array.
> >>> I don't see the need for fdt_init_reserved_mem() to be explicitly called
> >>> by arch code. I said this already, but that can be done at the same time
> >>> as unflattening the DT. The same conditions are needed for both: we need
> >>> to be able to allocate memory from memblock.
> >>>
> >>> To put it another way, if fdt_init_reserved_mem() can be called "early",
> >>> then unflattening could be moved earlier as well. Though I don't think
> >>> we should optimize that. I'd rather see all arches call the DT functions
> >>> at the same stages.
> >> Hi Rob,
> >>
> >> The reason we moved fdt_init_reserved_mem() back into the arch specific code
> >> was because we realized that there was no apparently obvious way to call
> >> early_init_fdt_scan_reserved_mem() and fdt_init_reserved_mem() in the correct
> >> order that will work for all archs if we placed fdt_init_reserved_mem() inside the
> >> unflatten_devicetree() function.
> >>
> >> early_init_fdt_scan_reserved_mem() needs to be
> >> called first before fdt_init_reserved_mem(). But on some archs,
> >> unflatten_devicetree() is called before early_init_fdt_scan_reserved_mem(), which
> >> means that if we have fdt_init_reserved_mem() inside the unflatten_devicetree()
> >> function, it will be called before early_init_fdt_scan_reserved_mem().
> >>
> >> This is connected to your other comments on Patch 7 & Patch 14.
> >> I agree, unflatten_devicetree() should NOT be getting called before we reserve
> >> memory for the reserved memory regions because that could cause memory to be
> >> allocated from regions that should be reserved.
> >>
> >> Hence, resolving this issue should allow us to call fdt_init_reserved_mem() from
> >> the? unflatten_devicetree() function without it changing the order that we are
> >> trying to have.
> > There's one issue I've found which is unflatten_device_tree() isn't
> > called for ACPI case on arm64. Turns out we need /reserved-memory
> > handled in that case too. However, I think we're going to change
> > calling unflatten_device_tree() unconditionally for another reason[1].
> >
> > [1] https://lore.kernel.org/all/[email protected]/
> >
> >> I will work on implementing this and send another revision.
> > I think we should go with a simpler route that's just copy the an
> > initial array in initdata to a properly sized, allocated array like the
> > patch below. Of course it will need some arch fixes and a follow-on
> > patch to increase the initial array size.
> >
> > 8<--------------------------------------------------------------------
> > From: Rob Herring <[email protected]>
> > Date: Wed, 31 Jan 2024 16:26:23 -0600
> > Subject: [PATCH] of: reserved-mem: Re-allocate reserved_mem array to actual
> > size
> >
> > In preparation to increase the static reserved_mem array size yet again,
> > copy the initial array to an allocated array sized based on the actual
> > size needed. Now increasing the the size of the static reserved_mem
> > array only eats up the initdata space. For platforms with reasonable
> > number of reserved regions, we have a net gain in free memory.
> >
> > In order to do memblock allocations, fdt_init_reserved_mem() is moved a
> > bit later to unflatten_device_tree(). On some arches this is effectively
> > a nop.
[...]
> Hi Rob,
>
> One thing that could come up with this is that? memory
> for the dynamically-placed reserved memory regions
> won't be allocated until we call fdt_init_reserved_mem().
> (i.e. reserved memory regions defined using @alloc-ranges
> and @size properties)
>
> Since fdt_init_reserved_mem() is now being called from
> unflatten_device_tree(), the page tables would have been
> setup on most architectures, which means we will be
> allocating from memory that have already been mapped.
>
> Could this be an issue for memory that is supposed to be
> reserved?
I suppose if the alloc-ranges region is not much bigger than the size
and the kernel already made some allocation that landed in the region,
then the allocation could fail. Not much we can do other than alloc the
reserved regions as soon as possible. Are there cases where that's not
happening?
I suppose the kernel could try and avoid all alloc-ranges until they've
been allocated, but that would have to be best effort. I've seen
optimizations where it's desired to spread buffers across DRAM banks, so
you could have N alloc-ranges for N banks that covers all of memory.
There's also the issue that if you have more fixed regions than memblock
can handle (128) before it can reallocate its arrays, then the
page tables themselves could be allocated in reserved regions.
> Especially for the regions that are specified as
> no-map?
'no-map' is a hint, not a guarantee. Arm32 ignores it for regions
within the kernel's linear map (at least it used to). I don't think
anything changes here with it.
Rob
On 2/2/2024 7:29 AM, Rob Herring wrote:
> On Thu, Feb 01, 2024 at 01:10:18PM -0800, Oreoluwa Babatunde wrote:
>> On 2/1/2024 11:46 AM, Rob Herring wrote:
>>> On Thu, Feb 01, 2024 at 09:08:06AM -0800, Oreoluwa Babatunde wrote:
>>>> On 1/30/2024 4:07 PM, Rob Herring wrote:
>>>>> On Fri, Jan 26, 2024 at 03:53:39PM -0800, Oreoluwa Babatunde wrote:
>>>>>> The reserved_mem array is used to store data for the different
>>>>>> reserved memory regions defined in the DT of a device. The array
>>>>>> stores information such as region name, node, start-address, and size
>>>>>> of the reserved memory regions.
>>>>>>
>>>>>> The array is currently statically allocated with a size of
>>>>>> MAX_RESERVED_REGIONS(64). This means that any system that specifies a
>>>>>> number of reserved memory regions greater than MAX_RESERVED_REGIONS(64)
>>>>>> will not have enough space to store the information for all the regions.
>>>>>>
>>>>>> Therefore, this series extends the use of the static array for
>>>>>> reserved_mem, and introduces a dynamically allocated array using
>>>>>> memblock_alloc() based on the number of reserved memory regions
>>>>>> specified in the DT.
>>>>>>
>>>>>> Some architectures such as arm64 require the page tables to be setup
>>>>>> before memblock allocated memory is writable. Therefore, the dynamic
>>>>>> allocation of the reserved_mem array will need to be done after the
>>>>>> page tables have been setup on these architectures. In most cases that
>>>>>> will be after paging_init().
>>>>>>
>>>>>> Reserved memory regions can be divided into 2 groups.
>>>>>> i) Statically-placed reserved memory regions
>>>>>> i.e. regions defined in the DT using the @reg property.
>>>>>> ii) Dynamically-placed reserved memory regions.
>>>>>> i.e. regions specified in the DT using the @alloc_ranges
>>>>>> and @size properties.
>>>>>>
>>>>>> It is possible to call memblock_reserve() and memblock_mark_nomap() on
>>>>>> the statically-placed reserved memory regions and not need to save them
>>>>>> to the reserved_mem array until memory is allocated for it using
>>>>>> memblock, which will be after the page tables have been setup.
>>>>>> For the dynamically-placed reserved memory regions, it is not possible
>>>>>> to wait to store its information because the starting address is
>>>>>> allocated only at run time, and hence they need to be stored somewhere
>>>>>> after they are allocated.
>>>>>> Waiting until after the page tables have been setup to allocate memory
>>>>>> for the dynamically-placed regions is also not an option because the
>>>>>> allocations will come from memory that have already been added to the
>>>>>> page tables, which is not good for memory that is supposed to be
>>>>>> reserved and/or marked as nomap.
>>>>>>
>>>>>> Therefore, this series splits up the processing of the reserved memory
>>>>>> regions into two stages, of which the first stage is carried out by
>>>>>> early_init_fdt_scan_reserved_mem() and the second is carried out by
>>>>>> fdt_init_reserved_mem().
>>>>>>
>>>>>> The early_init_fdt_scan_reserved_mem(), which is called before the page
>>>>>> tables are setup is used to:
>>>>>> 1. Call memblock_reserve() and memblock_mark_nomap() on all the
>>>>>> statically-placed reserved memory regions as needed.
>>>>>> 2. Allocate memory from memblock for the dynamically-placed reserved
>>>>>> memory regions and store them in the static array for reserved_mem.
>>>>>> memblock_reserve() and memblock_mark_nomap() are also called as
>>>>>> needed on all the memory allocated for the dynamically-placed
>>>>>> regions.
>>>>>> 3. Count the total number of reserved memory regions found in the DT.
>>>>>>
>>>>>> fdt_init_reserved_mem(), which should be called after the page tables
>>>>>> have been setup, is used to carry out the following:
>>>>>> 1. Allocate memory for the reserved_mem array based on the number of
>>>>>> reserved memory regions counted as mentioned above.
>>>>>> 2. Copy all the information for the dynamically-placed reserved memory
>>>>>> regions from the static array into the new allocated memory for the
>>>>>> reserved_mem array.
>>>>>> 3. Add the information for the statically-placed reserved memory into
>>>>>> reserved_mem array.
>>>>>> 4. Run the region specific init functions for each of the reserve memory
>>>>>> regions saved in the reserved_mem array.
>>>>> I don't see the need for fdt_init_reserved_mem() to be explicitly called
>>>>> by arch code. I said this already, but that can be done at the same time
>>>>> as unflattening the DT. The same conditions are needed for both: we need
>>>>> to be able to allocate memory from memblock.
>>>>>
>>>>> To put it another way, if fdt_init_reserved_mem() can be called "early",
>>>>> then unflattening could be moved earlier as well. Though I don't think
>>>>> we should optimize that. I'd rather see all arches call the DT functions
>>>>> at the same stages.
>>>> Hi Rob,
>>>>
>>>> The reason we moved fdt_init_reserved_mem() back into the arch specific code
>>>> was because we realized that there was no apparently obvious way to call
>>>> early_init_fdt_scan_reserved_mem() and fdt_init_reserved_mem() in the correct
>>>> order that will work for all archs if we placed fdt_init_reserved_mem() inside the
>>>> unflatten_devicetree() function.
>>>>
>>>> early_init_fdt_scan_reserved_mem() needs to be
>>>> called first before fdt_init_reserved_mem(). But on some archs,
>>>> unflatten_devicetree() is called before early_init_fdt_scan_reserved_mem(), which
>>>> means that if we have fdt_init_reserved_mem() inside the unflatten_devicetree()
>>>> function, it will be called before early_init_fdt_scan_reserved_mem().
>>>>
>>>> This is connected to your other comments on Patch 7 & Patch 14.
>>>> I agree, unflatten_devicetree() should NOT be getting called before we reserve
>>>> memory for the reserved memory regions because that could cause memory to be
>>>> allocated from regions that should be reserved.
>>>>
>>>> Hence, resolving this issue should allow us to call fdt_init_reserved_mem() from
>>>> the unflatten_devicetree() function without it changing the order that we are
>>>> trying to have.
>>> There's one issue I've found which is unflatten_device_tree() isn't
>>> called for ACPI case on arm64. Turns out we need /reserved-memory
>>> handled in that case too. However, I think we're going to change
>>> calling unflatten_device_tree() unconditionally for another reason[1].
>>>
>>> [1] https://lore.kernel.org/all/[email protected]/
>>>
>>>> I will work on implementing this and send another revision.
>>> I think we should go with a simpler route that's just copy the an
>>> initial array in initdata to a properly sized, allocated array like the
>>> patch below. Of course it will need some arch fixes and a follow-on
>>> patch to increase the initial array size.
>>>
>>> 8<--------------------------------------------------------------------
>>> From: Rob Herring <[email protected]>
>>> Date: Wed, 31 Jan 2024 16:26:23 -0600
>>> Subject: [PATCH] of: reserved-mem: Re-allocate reserved_mem array to actual
>>> size
>>>
>>> In preparation to increase the static reserved_mem array size yet again,
>>> copy the initial array to an allocated array sized based on the actual
>>> size needed. Now increasing the the size of the static reserved_mem
>>> array only eats up the initdata space. For platforms with reasonable
>>> number of reserved regions, we have a net gain in free memory.
>>>
>>> In order to do memblock allocations, fdt_init_reserved_mem() is moved a
>>> bit later to unflatten_device_tree(). On some arches this is effectively
>>> a nop.
> [...]
>
>> Hi Rob,
>>
>> One thing that could come up with this is that memory
>> for the dynamically-placed reserved memory regions
>> won't be allocated until we call fdt_init_reserved_mem().
>> (i.e. reserved memory regions defined using @alloc-ranges
>> and @size properties)
>>
>> Since fdt_init_reserved_mem() is now being called from
>> unflatten_device_tree(), the page tables would have been
>> setup on most architectures, which means we will be
>> allocating from memory that have already been mapped.
>>
>> Could this be an issue for memory that is supposed to be
>> reserved?
> I suppose if the alloc-ranges region is not much bigger than the size
> and the kernel already made some allocation that landed in the region,
> then the allocation could fail. Not much we can do other than alloc the
> reserved regions as soon as possible. Are there cases where that's not
> happening?
Correct, the best thing we can do here is to make sure we allocate the
reserved memory regions as soon as possible to avoid other users from
allocating from these regions.
>
> I suppose the kernel could try and avoid all alloc-ranges until they've
> been allocated, but that would have to be best effort. I've seen
> optimizations where it's desired to spread buffers across DRAM banks, so
> you could have N alloc-ranges for N banks that covers all of memory.
ack
>
> There's also the issue that if you have more fixed regions than memblock
> can handle (128) before it can reallocate its arrays, then the
> page tables themselves could be allocated in reserved regions.
True, this is also a limitation on the side of memblock, and there is
not much else we can do on this front as well.
> [ . . . ]
> 'no-map' is a hint, not a guarantee. Arm32 ignores it for regions
> within the kernel's linear map (at least it used to). I don't think
> anything changes here with it.
Anyone adding the "no-map" property to a reserved region is specifying that the region must not be mapped into the kernel page tables so that that there is no access from other users, not even speculative accesses.
This can be seen in the description of no-map here: https://github.com/devicetree-org/dt-schema/blob/main/dtschema/schemas/reserved-memory/reserved-memory.yaml#L79
I'm not sure about the arm32 architecture taking no-map as a hint, but I ran some tests on the arm64 architecture which I am currently testing on, and when a region is marked as no-map, it is excluded from the kernel page mappings (which is the correct behavior from the description of no-map above). But if we wait until after the page tables are setup to initialize reserved memory regions that are marked as no-map, then those regions would already be part of the page tables. And this defeats the purpose of users specifying no-map for their reserved memory. Hence, I think it is important to prioritize marking all nomap reserved memory regions as nomap before the page tables are setup so that this functionality is preserved. And we are able to achieve this with the code re-ordering that is being done in [PATCH 01/46] of this series. Regards, Oreoluwa