For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
do not free the reserved memory for the page map, decrease the section
size can reduce the waste of reserved memory.
Signed-off-by: Wei Li <[email protected]>
Signed-off-by: Baopeng Feng <[email protected]>
Signed-off-by: Xia Qing <[email protected]>
---
arch/arm64/include/asm/sparsemem.h | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
index 1f43fcc79738..8963bd3def28 100644
--- a/arch/arm64/include/asm/sparsemem.h
+++ b/arch/arm64/include/asm/sparsemem.h
@@ -7,7 +7,7 @@
#ifdef CONFIG_SPARSEMEM
#define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
-#define SECTION_SIZE_BITS 30
+#define SECTION_SIZE_BITS 27
#endif
#endif
--
2.15.0
> -----Original Message-----
> From: liwei (CM)
> Sent: Friday, December 4, 2020 2:45 PM
> To: [email protected]; [email protected]; [email protected]; liwei (CM)
> <[email protected]>
> Cc: fengbaopeng <[email protected]>; [email protected];
> [email protected]; Song Bao Hua (Barry Song) <[email protected]>;
> [email protected]; [email protected]; butao
> <[email protected]>
> Subject: [PATCH] arm64: mm: decrease the section size to reduce the memory
> reserved for the page map
>
> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> do not free the reserved memory for the page map, decrease the section
> size can reduce the waste of reserved memory.
>
> Signed-off-by: Wei Li <[email protected]>
> Signed-off-by: Baopeng Feng <[email protected]>
> Signed-off-by: Xia Qing <[email protected]>
> ---
Reviewed-by: Barry Song <[email protected]>
When page size is 4K, for each 1GB memory, we need 16MB vmemmap;
For each 128MB memory, we need 2MB vmemmap.
So while we have memory hole like 928MB(1GB-64MB),if SECTION_SIZE_BITS
is 30, we waste 15MB; if SECTION_SIZE_BITS is 27, we waste 1MB only.
> arch/arm64/include/asm/sparsemem.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/sparsemem.h
> b/arch/arm64/include/asm/sparsemem.h
> index 1f43fcc79738..8963bd3def28 100644
> --- a/arch/arm64/include/asm/sparsemem.h
> +++ b/arch/arm64/include/asm/sparsemem.h
> @@ -7,7 +7,7 @@
>
> #ifdef CONFIG_SPARSEMEM
> #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
> -#define SECTION_SIZE_BITS 30
> +#define SECTION_SIZE_BITS 27
> #endif
>
> #endif
> --
> 2.15.0
Thanks
Barry
On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> do not free the reserved memory for the page map, decrease the section
> size can reduce the waste of reserved memory.
>
> Signed-off-by: Wei Li <[email protected]>
> Signed-off-by: Baopeng Feng <[email protected]>
> Signed-off-by: Xia Qing <[email protected]>
> ---
> arch/arm64/include/asm/sparsemem.h | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
> index 1f43fcc79738..8963bd3def28 100644
> --- a/arch/arm64/include/asm/sparsemem.h
> +++ b/arch/arm64/include/asm/sparsemem.h
> @@ -7,7 +7,7 @@
>
> #ifdef CONFIG_SPARSEMEM
> #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
> -#define SECTION_SIZE_BITS 30
> +#define SECTION_SIZE_BITS 27
We chose '30' to avoid running out of bits in the page flags. What changed?
With this patch, I can trigger:
./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds SECTION_SIZE
#error Allocator MAX_ORDER exceeds SECTION_SIZE
if I bump up NR_CPUS and NODES_SHIFT.
Will
On Fri, Dec 04, 2020 at 11:13:47AM +0000, Will Deacon wrote:
> On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
> > For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> > do not free the reserved memory for the page map, decrease the section
> > size can reduce the waste of reserved memory.
> >
> > Signed-off-by: Wei Li <[email protected]>
> > Signed-off-by: Baopeng Feng <[email protected]>
> > Signed-off-by: Xia Qing <[email protected]>
> > ---
> > arch/arm64/include/asm/sparsemem.h | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
> > index 1f43fcc79738..8963bd3def28 100644
> > --- a/arch/arm64/include/asm/sparsemem.h
> > +++ b/arch/arm64/include/asm/sparsemem.h
> > @@ -7,7 +7,7 @@
> >
> > #ifdef CONFIG_SPARSEMEM
> > #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
> > -#define SECTION_SIZE_BITS 30
> > +#define SECTION_SIZE_BITS 27
>
> We chose '30' to avoid running out of bits in the page flags. What changed?
I think that for 64-bit there are still plenty of free bits. I didn't
check now, but when I played with SPARSEMEM on m68k there were 8 bits
for section out of 32.
> With this patch, I can trigger:
>
> ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds SECTION_SIZE
> #error Allocator MAX_ORDER exceeds SECTION_SIZE
>
> if I bump up NR_CPUS and NODES_SHIFT.
I don't think it's related to NR_CPUS and NODES_SHIFT.
This seems rather 64K pages that cause this.
Not that is shouldn't be addressed.
> Will
--
Sincerely yours,
Mike.
> -----Original Message-----
> From: Mike Rapoport [mailto:[email protected]]
> Sent: Saturday, December 5, 2020 12:44 AM
> To: Will Deacon <[email protected]>
> Cc: liwei (CM) <[email protected]>; [email protected]; fengbaopeng
> <[email protected]>; [email protected]; [email protected];
> Song Bao Hua (Barry Song) <[email protected]>;
> [email protected]; [email protected]; butao
> <[email protected]>
> Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory
> reserved for the page map
>
> On Fri, Dec 04, 2020 at 11:13:47AM +0000, Will Deacon wrote:
> > On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
> > > For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> > > do not free the reserved memory for the page map, decrease the section
> > > size can reduce the waste of reserved memory.
> > >
> > > Signed-off-by: Wei Li <[email protected]>
> > > Signed-off-by: Baopeng Feng <[email protected]>
> > > Signed-off-by: Xia Qing <[email protected]>
> > > ---
> > > arch/arm64/include/asm/sparsemem.h | 2 +-
> > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > >
> > > diff --git a/arch/arm64/include/asm/sparsemem.h
> b/arch/arm64/include/asm/sparsemem.h
> > > index 1f43fcc79738..8963bd3def28 100644
> > > --- a/arch/arm64/include/asm/sparsemem.h
> > > +++ b/arch/arm64/include/asm/sparsemem.h
> > > @@ -7,7 +7,7 @@
> > >
> > > #ifdef CONFIG_SPARSEMEM
> > > #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
> > > -#define SECTION_SIZE_BITS 30
> > > +#define SECTION_SIZE_BITS 27
> >
> > We chose '30' to avoid running out of bits in the page flags. What changed?
>
> I think that for 64-bit there are still plenty of free bits. I didn't
> check now, but when I played with SPARSEMEM on m68k there were 8 bits
> for section out of 32.
>
> > With this patch, I can trigger:
> >
> > ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds
> SECTION_SIZE
> > #error Allocator MAX_ORDER exceeds SECTION_SIZE
> >
> > if I bump up NR_CPUS and NODES_SHIFT.
>
> I don't think it's related to NR_CPUS and NODES_SHIFT.
> This seems rather 64K pages that cause this.
>
> Not that is shouldn't be addressed.
Right now, only 4K PAGES will define ARM64_SWAPPER_USES_SECTION_MAPS.
Other cases will use vmemmap_populate_basepages().
The original patch should be only addressing the issue in 4K pages:
https://lore.kernel.org/lkml/[email protected]/
would we do something like the below?
#ifdef CONFIG_ARM64_4K_PAGE
#define SECTION_SIZE_BITS 27
#else
#define SECTION_SIZE_BITS 30
#endif
>
> > Will
>
> --
> Sincerely yours,
> Mike.
Thanks
Barry
On 12/7/20 7:10 AM, Song Bao Hua (Barry Song) wrote:
>
>
>> -----Original Message-----
>> From: Mike Rapoport [mailto:[email protected]]
>> Sent: Saturday, December 5, 2020 12:44 AM
>> To: Will Deacon <[email protected]>
>> Cc: liwei (CM) <[email protected]>; [email protected]; fengbaopeng
>> <[email protected]>; [email protected]; [email protected];
>> Song Bao Hua (Barry Song) <[email protected]>;
>> [email protected]; [email protected]; butao
>> <[email protected]>
>> Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory
>> reserved for the page map
>>
>> On Fri, Dec 04, 2020 at 11:13:47AM +0000, Will Deacon wrote:
>>> On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
>>>> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
>>>> do not free the reserved memory for the page map, decrease the section
>>>> size can reduce the waste of reserved memory.
>>>>
>>>> Signed-off-by: Wei Li <[email protected]>
>>>> Signed-off-by: Baopeng Feng <[email protected]>
>>>> Signed-off-by: Xia Qing <[email protected]>
>>>> ---
>>>> arch/arm64/include/asm/sparsemem.h | 2 +-
>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/arch/arm64/include/asm/sparsemem.h
>> b/arch/arm64/include/asm/sparsemem.h
>>>> index 1f43fcc79738..8963bd3def28 100644
>>>> --- a/arch/arm64/include/asm/sparsemem.h
>>>> +++ b/arch/arm64/include/asm/sparsemem.h
>>>> @@ -7,7 +7,7 @@
>>>>
>>>> #ifdef CONFIG_SPARSEMEM
>>>> #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
>>>> -#define SECTION_SIZE_BITS 30
>>>> +#define SECTION_SIZE_BITS 27
>>>
>>> We chose '30' to avoid running out of bits in the page flags. What changed?
>>
>> I think that for 64-bit there are still plenty of free bits. I didn't
>> check now, but when I played with SPARSEMEM on m68k there were 8 bits
>> for section out of 32.
>>
>>> With this patch, I can trigger:
>>>
>>> ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds
>> SECTION_SIZE
>>> #error Allocator MAX_ORDER exceeds SECTION_SIZE
>>>
>>> if I bump up NR_CPUS and NODES_SHIFT.
>>
>> I don't think it's related to NR_CPUS and NODES_SHIFT.
>> This seems rather 64K pages that cause this.
>>
>> Not that is shouldn't be addressed.
>
> Right now, only 4K PAGES will define ARM64_SWAPPER_USES_SECTION_MAPS.
> Other cases will use vmemmap_populate_basepages().
> The original patch should be only addressing the issue in 4K pages:
> https://lore.kernel.org/lkml/[email protected]/
>
> would we do something like the below?
> #ifdef CONFIG_ARM64_4K_PAGE
> #define SECTION_SIZE_BITS 27
> #else
> #define SECTION_SIZE_BITS 30
> #endif
This is bit arbitrary. Probably 27 can be further reduced for 4K page size.
Instead, we should make SECTION_SIZE_BITS explicitly depend upon MAX_ORDER.
IOW section size should be the same as the highest order page in the buddy.
CONFIG_FORCE_MAX_ZONEORDER is always defined on arm64. A quick test shows
SECTION_SIZE_BITS would be 22 on 4K pages and 29 for 64K pages. As a fall
back SECTION_SIZE_BITS can still be 30 in case CONFIG_FORCE_MAX_ZONEORDER
is not defined.
--- a/arch/arm64/include/asm/sparsemem.h
+++ b/arch/arm64/include/asm/sparsemem.h
@@ -7,7 +7,7 @@
#ifdef CONFIG_SPARSEMEM
#define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
-#define SECTION_SIZE_BITS 30
+#define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT)
#endif
#endif
A similar approach exists on ia64 platform as well.
> -----Original Message-----
> From: Anshuman Khandual [mailto:[email protected]]
> Sent: Monday, December 7, 2020 8:31 PM
> To: Song Bao Hua (Barry Song) <[email protected]>; Mike Rapoport
> <[email protected]>; Will Deacon <[email protected]>
> Cc: [email protected]; [email protected];
> [email protected]; [email protected]; liwei (CM)
> <[email protected]>; butao <[email protected]>;
> [email protected]; fengbaopeng
> <[email protected]>
> Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory
> reserved for the page map
>
>
>
> On 12/7/20 7:10 AM, Song Bao Hua (Barry Song) wrote:
> >
> >
> >> -----Original Message-----
> >> From: Mike Rapoport [mailto:[email protected]]
> >> Sent: Saturday, December 5, 2020 12:44 AM
> >> To: Will Deacon <[email protected]>
> >> Cc: liwei (CM) <[email protected]>; [email protected]; fengbaopeng
> >> <[email protected]>; [email protected];
> [email protected];
> >> Song Bao Hua (Barry Song) <[email protected]>;
> >> [email protected]; [email protected]; butao
> >> <[email protected]>
> >> Subject: Re: [PATCH] arm64: mm: decrease the section size to reduce the memory
> >> reserved for the page map
> >>
> >> On Fri, Dec 04, 2020 at 11:13:47AM +0000, Will Deacon wrote:
> >>> On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
> >>>> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> >>>> do not free the reserved memory for the page map, decrease the section
> >>>> size can reduce the waste of reserved memory.
> >>>>
> >>>> Signed-off-by: Wei Li <[email protected]>
> >>>> Signed-off-by: Baopeng Feng <[email protected]>
> >>>> Signed-off-by: Xia Qing <[email protected]>
> >>>> ---
> >>>> arch/arm64/include/asm/sparsemem.h | 2 +-
> >>>> 1 file changed, 1 insertion(+), 1 deletion(-)
> >>>>
> >>>> diff --git a/arch/arm64/include/asm/sparsemem.h
> >> b/arch/arm64/include/asm/sparsemem.h
> >>>> index 1f43fcc79738..8963bd3def28 100644
> >>>> --- a/arch/arm64/include/asm/sparsemem.h
> >>>> +++ b/arch/arm64/include/asm/sparsemem.h
> >>>> @@ -7,7 +7,7 @@
> >>>>
> >>>> #ifdef CONFIG_SPARSEMEM
> >>>> #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
> >>>> -#define SECTION_SIZE_BITS 30
> >>>> +#define SECTION_SIZE_BITS 27
> >>>
> >>> We chose '30' to avoid running out of bits in the page flags. What changed?
> >>
> >> I think that for 64-bit there are still plenty of free bits. I didn't
> >> check now, but when I played with SPARSEMEM on m68k there were 8 bits
> >> for section out of 32.
> >>
> >>> With this patch, I can trigger:
> >>>
> >>> ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds
> >> SECTION_SIZE
> >>> #error Allocator MAX_ORDER exceeds SECTION_SIZE
> >>>
> >>> if I bump up NR_CPUS and NODES_SHIFT.
> >>
> >> I don't think it's related to NR_CPUS and NODES_SHIFT.
> >> This seems rather 64K pages that cause this.
> >>
> >> Not that is shouldn't be addressed.
> >
> > Right now, only 4K PAGES will define ARM64_SWAPPER_USES_SECTION_MAPS.
> > Other cases will use vmemmap_populate_basepages().
> > The original patch should be only addressing the issue in 4K pages:
> > https://lore.kernel.org/lkml/[email protected]/
> >
> > would we do something like the below?
> > #ifdef CONFIG_ARM64_4K_PAGE
> > #define SECTION_SIZE_BITS 27
> > #else
> > #define SECTION_SIZE_BITS 30
> > #endif
>
> This is bit arbitrary. Probably 27 can be further reduced for 4K page size.
> Instead, we should make SECTION_SIZE_BITS explicitly depend upon MAX_ORDER.
> IOW section size should be the same as the highest order page in the buddy.
> CONFIG_FORCE_MAX_ZONEORDER is always defined on arm64. A quick test shows
> SECTION_SIZE_BITS would be 22 on 4K pages and 29 for 64K pages. As a fall
> back SECTION_SIZE_BITS can still be 30 in case CONFIG_FORCE_MAX_ZONEORDER
> is not defined.
This will break the pmd mapping for vmemmap. As for each 128M(2^27), we can
have 2MB pmd mapping.
>
> --- a/arch/arm64/include/asm/sparsemem.h
> +++ b/arch/arm64/include/asm/sparsemem.h
> @@ -7,7 +7,7 @@
>
> #ifdef CONFIG_SPARSEMEM
> #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
> -#define SECTION_SIZE_BITS 30
> +#define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT)
> #endif
>
> #endif
>
> A similar approach exists on ia64 platform as well.
Thanks
Barry
(+ Marc)
On Fri, 4 Dec 2020 at 12:14, Will Deacon <[email protected]> wrote:
>
> On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
> > For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> > do not free the reserved memory for the page map, decrease the section
> > size can reduce the waste of reserved memory.
> >
> > Signed-off-by: Wei Li <[email protected]>
> > Signed-off-by: Baopeng Feng <[email protected]>
> > Signed-off-by: Xia Qing <[email protected]>
> > ---
> > arch/arm64/include/asm/sparsemem.h | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
> > index 1f43fcc79738..8963bd3def28 100644
> > --- a/arch/arm64/include/asm/sparsemem.h
> > +++ b/arch/arm64/include/asm/sparsemem.h
> > @@ -7,7 +7,7 @@
> >
> > #ifdef CONFIG_SPARSEMEM
> > #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
> > -#define SECTION_SIZE_BITS 30
> > +#define SECTION_SIZE_BITS 27
>
> We chose '30' to avoid running out of bits in the page flags. What changed?
>
> With this patch, I can trigger:
>
> ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds SECTION_SIZE
> #error Allocator MAX_ORDER exceeds SECTION_SIZE
>
> if I bump up NR_CPUS and NODES_SHIFT.
>
Does this mean we will run into problems with the GICv3 ITS LPI tables
again if we are forced to reduce MAX_ORDER to fit inside
SECTION_SIZE_BITS?
On 2020-12-07 09:09, Ard Biesheuvel wrote:
> (+ Marc)
>
> On Fri, 4 Dec 2020 at 12:14, Will Deacon <[email protected]> wrote:
>>
>> On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
>> > For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
>> > do not free the reserved memory for the page map, decrease the section
>> > size can reduce the waste of reserved memory.
>> >
>> > Signed-off-by: Wei Li <[email protected]>
>> > Signed-off-by: Baopeng Feng <[email protected]>
>> > Signed-off-by: Xia Qing <[email protected]>
>> > ---
>> > arch/arm64/include/asm/sparsemem.h | 2 +-
>> > 1 file changed, 1 insertion(+), 1 deletion(-)
>> >
>> > diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
>> > index 1f43fcc79738..8963bd3def28 100644
>> > --- a/arch/arm64/include/asm/sparsemem.h
>> > +++ b/arch/arm64/include/asm/sparsemem.h
>> > @@ -7,7 +7,7 @@
>> >
>> > #ifdef CONFIG_SPARSEMEM
>> > #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
>> > -#define SECTION_SIZE_BITS 30
>> > +#define SECTION_SIZE_BITS 27
>>
>> We chose '30' to avoid running out of bits in the page flags. What
>> changed?
>>
>> With this patch, I can trigger:
>>
>> ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds
>> SECTION_SIZE
>> #error Allocator MAX_ORDER exceeds SECTION_SIZE
>>
>> if I bump up NR_CPUS and NODES_SHIFT.
>>
>
> Does this mean we will run into problems with the GICv3 ITS LPI tables
> again if we are forced to reduce MAX_ORDER to fit inside
> SECTION_SIZE_BITS?
Most probably. We are already massively constraint on platforms
such as TX1, and dividing the max allocatable range by 8 isn't
going to make it work any better...
M.
--
Jazz is not dead. It just smells funny...
On Mon, Dec 07, 2020 at 09:35:06AM +0000, Marc Zyngier wrote:
> On 2020-12-07 09:09, Ard Biesheuvel wrote:
> > (+ Marc)
> >
> > On Fri, 4 Dec 2020 at 12:14, Will Deacon <[email protected]> wrote:
> > >
> > > On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
> > > > For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> > > > do not free the reserved memory for the page map, decrease the section
> > > > size can reduce the waste of reserved memory.
> > > >
> > > > Signed-off-by: Wei Li <[email protected]>
> > > > Signed-off-by: Baopeng Feng <[email protected]>
> > > > Signed-off-by: Xia Qing <[email protected]>
> > > > ---
> > > > arch/arm64/include/asm/sparsemem.h | 2 +-
> > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > >
> > > > diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
> > > > index 1f43fcc79738..8963bd3def28 100644
> > > > --- a/arch/arm64/include/asm/sparsemem.h
> > > > +++ b/arch/arm64/include/asm/sparsemem.h
> > > > @@ -7,7 +7,7 @@
> > > >
> > > > #ifdef CONFIG_SPARSEMEM
> > > > #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
> > > > -#define SECTION_SIZE_BITS 30
> > > > +#define SECTION_SIZE_BITS 27
> > >
> > > We chose '30' to avoid running out of bits in the page flags. What
> > > changed?
> > >
> > > With this patch, I can trigger:
> > >
> > > ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds
> > > SECTION_SIZE
> > > #error Allocator MAX_ORDER exceeds SECTION_SIZE
> > >
> > > if I bump up NR_CPUS and NODES_SHIFT.
> > >
> >
> > Does this mean we will run into problems with the GICv3 ITS LPI tables
> > again if we are forced to reduce MAX_ORDER to fit inside
> > SECTION_SIZE_BITS?
>
> Most probably. We are already massively constraint on platforms
> such as TX1, and dividing the max allocatable range by 8 isn't
> going to make it work any better...
I don't think MAX_ORDER should shrink. Even if SECTION_SIZE_BITS is
reduced it should accomodate the existing MAX_ORDER.
My two pennies.
> M.
> --
> Jazz is not dead. It just smells funny...
--
Sincerely yours,
Mike.
On Mon, 7 Dec 2020 at 10:42, Mike Rapoport <[email protected]> wrote:
>
> On Mon, Dec 07, 2020 at 09:35:06AM +0000, Marc Zyngier wrote:
> > On 2020-12-07 09:09, Ard Biesheuvel wrote:
> > > (+ Marc)
> > >
> > > On Fri, 4 Dec 2020 at 12:14, Will Deacon <[email protected]> wrote:
> > > >
> > > > On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
> > > > > For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> > > > > do not free the reserved memory for the page map, decrease the section
> > > > > size can reduce the waste of reserved memory.
> > > > >
> > > > > Signed-off-by: Wei Li <[email protected]>
> > > > > Signed-off-by: Baopeng Feng <[email protected]>
> > > > > Signed-off-by: Xia Qing <[email protected]>
> > > > > ---
> > > > > arch/arm64/include/asm/sparsemem.h | 2 +-
> > > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > > >
> > > > > diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
> > > > > index 1f43fcc79738..8963bd3def28 100644
> > > > > --- a/arch/arm64/include/asm/sparsemem.h
> > > > > +++ b/arch/arm64/include/asm/sparsemem.h
> > > > > @@ -7,7 +7,7 @@
> > > > >
> > > > > #ifdef CONFIG_SPARSEMEM
> > > > > #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
> > > > > -#define SECTION_SIZE_BITS 30
> > > > > +#define SECTION_SIZE_BITS 27
> > > >
> > > > We chose '30' to avoid running out of bits in the page flags. What
> > > > changed?
> > > >
> > > > With this patch, I can trigger:
> > > >
> > > > ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds
> > > > SECTION_SIZE
> > > > #error Allocator MAX_ORDER exceeds SECTION_SIZE
> > > >
> > > > if I bump up NR_CPUS and NODES_SHIFT.
> > > >
> > >
> > > Does this mean we will run into problems with the GICv3 ITS LPI tables
> > > again if we are forced to reduce MAX_ORDER to fit inside
> > > SECTION_SIZE_BITS?
> >
> > Most probably. We are already massively constraint on platforms
> > such as TX1, and dividing the max allocatable range by 8 isn't
> > going to make it work any better...
>
> I don't think MAX_ORDER should shrink. Even if SECTION_SIZE_BITS is
> reduced it should accomodate the existing MAX_ORDER.
>
> My two pennies.
>
But include/linux/mmzone.h:1170 has this:
#if (MAX_ORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS
#error Allocator MAX_ORDER exceeds SECTION_SIZE
#endif
and Will managed to trigger it after applying this patch.
On Mon, Dec 07, 2020 at 10:49:26AM +0100, Ard Biesheuvel wrote:
> On Mon, 7 Dec 2020 at 10:42, Mike Rapoport <[email protected]> wrote:
> >
> > On Mon, Dec 07, 2020 at 09:35:06AM +0000, Marc Zyngier wrote:
> > > On 2020-12-07 09:09, Ard Biesheuvel wrote:
> > > > (+ Marc)
> > > >
> > > > On Fri, 4 Dec 2020 at 12:14, Will Deacon <[email protected]> wrote:
> > > > >
> > > > > On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
> > > > > > For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
> > > > > > do not free the reserved memory for the page map, decrease the section
> > > > > > size can reduce the waste of reserved memory.
> > > > > >
> > > > > > Signed-off-by: Wei Li <[email protected]>
> > > > > > Signed-off-by: Baopeng Feng <[email protected]>
> > > > > > Signed-off-by: Xia Qing <[email protected]>
> > > > > > ---
> > > > > > arch/arm64/include/asm/sparsemem.h | 2 +-
> > > > > > 1 file changed, 1 insertion(+), 1 deletion(-)
> > > > > >
> > > > > > diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
> > > > > > index 1f43fcc79738..8963bd3def28 100644
> > > > > > --- a/arch/arm64/include/asm/sparsemem.h
> > > > > > +++ b/arch/arm64/include/asm/sparsemem.h
> > > > > > @@ -7,7 +7,7 @@
> > > > > >
> > > > > > #ifdef CONFIG_SPARSEMEM
> > > > > > #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
> > > > > > -#define SECTION_SIZE_BITS 30
> > > > > > +#define SECTION_SIZE_BITS 27
> > > > >
> > > > > We chose '30' to avoid running out of bits in the page flags. What
> > > > > changed?
> > > > >
> > > > > With this patch, I can trigger:
> > > > >
> > > > > ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds
> > > > > SECTION_SIZE
> > > > > #error Allocator MAX_ORDER exceeds SECTION_SIZE
> > > > >
> > > > > if I bump up NR_CPUS and NODES_SHIFT.
> > > > >
> > > >
> > > > Does this mean we will run into problems with the GICv3 ITS LPI tables
> > > > again if we are forced to reduce MAX_ORDER to fit inside
> > > > SECTION_SIZE_BITS?
> > >
> > > Most probably. We are already massively constraint on platforms
> > > such as TX1, and dividing the max allocatable range by 8 isn't
> > > going to make it work any better...
> >
> > I don't think MAX_ORDER should shrink. Even if SECTION_SIZE_BITS is
> > reduced it should accomodate the existing MAX_ORDER.
> >
> > My two pennies.
> >
>
> But include/linux/mmzone.h:1170 has this:
>
> #if (MAX_ORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS
> #error Allocator MAX_ORDER exceeds SECTION_SIZE
> #endif
>
> and Will managed to trigger it after applying this patch.
Right, because with 64K pages section size of 27 bits is not enough to
accomodate MAX_ORDER (2^13 pages of 64K).
Which means that definition of SECTION_SIZE_BITS should take MAX_ORDER
into account either statically with
#ifdef ARM64_4K_PAGES
#define SECTION_SIZE_BITS <a number>
#elif ARM64_16K_PAGES
#define SECTION_SIZE_BITS <a larger number>
#elif ARM64_64K_PAGES
#define SECTION_SIZE_BITS <even larger number>
#else
#error "and what is the page size?"
#endif
or dynamically, like e.g. ia64 does:
#ifdef CONFIG_FORCE_MAX_ZONEORDER
#if ((CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS)
#undef SECTION_SIZE_BITS
#define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT)
#endif
--
Sincerely yours,
Mike.
On 12/7/20 3:34 PM, Mike Rapoport wrote:
> On Mon, Dec 07, 2020 at 10:49:26AM +0100, Ard Biesheuvel wrote:
>> On Mon, 7 Dec 2020 at 10:42, Mike Rapoport <[email protected]> wrote:
>>>
>>> On Mon, Dec 07, 2020 at 09:35:06AM +0000, Marc Zyngier wrote:
>>>> On 2020-12-07 09:09, Ard Biesheuvel wrote:
>>>>> (+ Marc)
>>>>>
>>>>> On Fri, 4 Dec 2020 at 12:14, Will Deacon <[email protected]> wrote:
>>>>>>
>>>>>> On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
>>>>>>> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
>>>>>>> do not free the reserved memory for the page map, decrease the section
>>>>>>> size can reduce the waste of reserved memory.
>>>>>>>
>>>>>>> Signed-off-by: Wei Li <[email protected]>
>>>>>>> Signed-off-by: Baopeng Feng <[email protected]>
>>>>>>> Signed-off-by: Xia Qing <[email protected]>
>>>>>>> ---
>>>>>>> arch/arm64/include/asm/sparsemem.h | 2 +-
>>>>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>>
>>>>>>> diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
>>>>>>> index 1f43fcc79738..8963bd3def28 100644
>>>>>>> --- a/arch/arm64/include/asm/sparsemem.h
>>>>>>> +++ b/arch/arm64/include/asm/sparsemem.h
>>>>>>> @@ -7,7 +7,7 @@
>>>>>>>
>>>>>>> #ifdef CONFIG_SPARSEMEM
>>>>>>> #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
>>>>>>> -#define SECTION_SIZE_BITS 30
>>>>>>> +#define SECTION_SIZE_BITS 27
>>>>>>
>>>>>> We chose '30' to avoid running out of bits in the page flags. What
>>>>>> changed?
>>>>>>
>>>>>> With this patch, I can trigger:
>>>>>>
>>>>>> ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds
>>>>>> SECTION_SIZE
>>>>>> #error Allocator MAX_ORDER exceeds SECTION_SIZE
>>>>>>
>>>>>> if I bump up NR_CPUS and NODES_SHIFT.
>>>>>>
>>>>>
>>>>> Does this mean we will run into problems with the GICv3 ITS LPI tables
>>>>> again if we are forced to reduce MAX_ORDER to fit inside
>>>>> SECTION_SIZE_BITS?
>>>>
>>>> Most probably. We are already massively constraint on platforms
>>>> such as TX1, and dividing the max allocatable range by 8 isn't
>>>> going to make it work any better...
>>>
>>> I don't think MAX_ORDER should shrink. Even if SECTION_SIZE_BITS is
>>> reduced it should accomodate the existing MAX_ORDER.
>>>
>>> My two pennies.
>>>
>>
>> But include/linux/mmzone.h:1170 has this:
>>
>> #if (MAX_ORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS
>> #error Allocator MAX_ORDER exceeds SECTION_SIZE
>> #endif
>>
>> and Will managed to trigger it after applying this patch.
>
> Right, because with 64K pages section size of 27 bits is not enough to
> accomodate MAX_ORDER (2^13 pages of 64K).
>
> Which means that definition of SECTION_SIZE_BITS should take MAX_ORDER
> into account either statically with
>
> #ifdef ARM64_4K_PAGES
> #define SECTION_SIZE_BITS <a number>
> #elif ARM64_16K_PAGES
> #define SECTION_SIZE_BITS <a larger number>
> #elif ARM64_64K_PAGES
> #define SECTION_SIZE_BITS <even larger number>
> #else
> #error "and what is the page size?"
> #endif
>
> or dynamically, like e.g. ia64 does:
>
> #ifdef CONFIG_FORCE_MAX_ZONEORDER
> #if ((CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS)
> #undef SECTION_SIZE_BITS
> #define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT)
> #endif
I had proposed the same on the other thread here. But with this the
SECTION_SIZE_BITS becomes 22 in case of 4K page size reducing to an
extent where PMD based vmemmap mapping could not be created. Though
have not looked into much details yet.
Using CONFIG_FORCE_MAX_ZONEORDER seems to the right thing to do. But
if that does not reasonably work for 4K pages, we might have to hard
code it as 27 to have huge page vmemmap mappings.
(+ saberlily + jiapeng)
On 2020/12/7 18:39, Anshuman Khandual wrote:
>
>
> On 12/7/20 3:34 PM, Mike Rapoport wrote:
>> On Mon, Dec 07, 2020 at 10:49:26AM +0100, Ard Biesheuvel wrote:
>>> On Mon, 7 Dec 2020 at 10:42, Mike Rapoport <[email protected]> wrote:
>>>>
>>>> On Mon, Dec 07, 2020 at 09:35:06AM +0000, Marc Zyngier wrote:
>>>>> On 2020-12-07 09:09, Ard Biesheuvel wrote:
>>>>>> (+ Marc)
>>>>>>
>>>>>> On Fri, 4 Dec 2020 at 12:14, Will Deacon <[email protected]> wrote:
>>>>>>>
>>>>>>> On Fri, Dec 04, 2020 at 09:44:43AM +0800, Wei Li wrote:
>>>>>>>> For the memory hole, sparse memory model that define SPARSEMEM_VMEMMAP
>>>>>>>> do not free the reserved memory for the page map, decrease the section
>>>>>>>> size can reduce the waste of reserved memory.
>>>>>>>>
>>>>>>>> Signed-off-by: Wei Li <[email protected]>
>>>>>>>> Signed-off-by: Baopeng Feng <[email protected]>
>>>>>>>> Signed-off-by: Xia Qing <[email protected]>
>>>>>>>> ---
>>>>>>>> arch/arm64/include/asm/sparsemem.h | 2 +-
>>>>>>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>>>>>>>
>>>>>>>> diff --git a/arch/arm64/include/asm/sparsemem.h b/arch/arm64/include/asm/sparsemem.h
>>>>>>>> index 1f43fcc79738..8963bd3def28 100644
>>>>>>>> --- a/arch/arm64/include/asm/sparsemem.h
>>>>>>>> +++ b/arch/arm64/include/asm/sparsemem.h
>>>>>>>> @@ -7,7 +7,7 @@
>>>>>>>>
>>>>>>>> #ifdef CONFIG_SPARSEMEM
>>>>>>>> #define MAX_PHYSMEM_BITS CONFIG_ARM64_PA_BITS
>>>>>>>> -#define SECTION_SIZE_BITS 30
>>>>>>>> +#define SECTION_SIZE_BITS 27
>>>>>>>
>>>>>>> We chose '30' to avoid running out of bits in the page flags. What
>>>>>>> changed?
>>>>>>>
>>>>>>> With this patch, I can trigger:
>>>>>>>
>>>>>>> ./include/linux/mmzone.h:1170:2: error: Allocator MAX_ORDER exceeds
>>>>>>> SECTION_SIZE
>>>>>>> #error Allocator MAX_ORDER exceeds SECTION_SIZE
>>>>>>>
>>>>>>> if I bump up NR_CPUS and NODES_SHIFT.
>>>>>>>
>>>>>>
>>>>>> Does this mean we will run into problems with the GICv3 ITS LPI tables
>>>>>> again if we are forced to reduce MAX_ORDER to fit inside
>>>>>> SECTION_SIZE_BITS?
>>>>>
>>>>> Most probably. We are already massively constraint on platforms
>>>>> such as TX1, and dividing the max allocatable range by 8 isn't
>>>>> going to make it work any better...
>>>>
>>>> I don't think MAX_ORDER should shrink. Even if SECTION_SIZE_BITS is
>>>> reduced it should accomodate the existing MAX_ORDER.
>>>>
>>>> My two pennies.
>>>>
>>>
>>> But include/linux/mmzone.h:1170 has this:
>>>
>>> #if (MAX_ORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS
>>> #error Allocator MAX_ORDER exceeds SECTION_SIZE
>>> #endif
>>>
>>> and Will managed to trigger it after applying this patch.
>>
>> Right, because with 64K pages section size of 27 bits is not enough to
>> accomodate MAX_ORDER (2^13 pages of 64K).
>>
>> Which means that definition of SECTION_SIZE_BITS should take MAX_ORDER
>> into account either statically with
>>
>> #ifdef ARM64_4K_PAGES
>> #define SECTION_SIZE_BITS <a number>
>> #elif ARM64_16K_PAGES
>> #define SECTION_SIZE_BITS <a larger number>
>> #elif ARM64_64K_PAGES
>> #define SECTION_SIZE_BITS <even larger number>
>> #else
>> #error "and what is the page size?"
>> #endif
>>
>> or dynamically, like e.g. ia64 does:
>>
>> #ifdef CONFIG_FORCE_MAX_ZONEORDER
>> #if ((CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT) > SECTION_SIZE_BITS)
>> #undef SECTION_SIZE_BITS
>> #define SECTION_SIZE_BITS (CONFIG_FORCE_MAX_ZONEORDER - 1 + PAGE_SHIFT)
>> #endif
>
> I had proposed the same on the other thread here. But with this the
> SECTION_SIZE_BITS becomes 22 in case of 4K page size reducing to an
> extent where PMD based vmemmap mapping could not be created. Though
> have not looked into much details yet.
>
> Using CONFIG_FORCE_MAX_ZONEORDER seems to the right thing to do. But
> if that does not reasonably work for 4K pages, we might have to hard
> code it as 27 to have huge page vmemmap mappings.
> .
>