2014-04-29 04:59:34

by Jungseok Lee

[permalink] [raw]
Subject: [PATCH v4 4/7] arm64: Add a description on 48-bit address space with 4KB pages

This patch adds memory layout and translation lookup information
about 48-bit address space with 4K pages. The description is based
on 4 levels of translation tables.

Cc: Catalin Marinas <[email protected]>
Cc: Steve Capper <[email protected]>
Signed-off-by: Jungseok Lee <[email protected]>
Reviewed-by: Sungjinn Chung <[email protected]>
---
Documentation/arm64/memory.txt | 59 ++++++++++++++++++++++++++++++++++------
1 file changed, 51 insertions(+), 8 deletions(-)

diff --git a/Documentation/arm64/memory.txt b/Documentation/arm64/memory.txt
index d50fa61..8142709 100644
--- a/Documentation/arm64/memory.txt
+++ b/Documentation/arm64/memory.txt
@@ -8,10 +8,11 @@ This document describes the virtual memory layout used by the AArch64
Linux kernel. The architecture allows up to 4 levels of translation
tables with a 4KB page size and up to 3 levels with a 64KB page size.

-AArch64 Linux uses 3 levels of translation tables with the 4KB page
-configuration, allowing 39-bit (512GB) virtual addresses for both user
-and kernel. With 64KB pages, only 2 levels of translation tables are
-used but the memory layout is the same.
+AArch64 Linux uses 3 levels and 4 levels of translation tables with
+the 4KB page configuration, allowing 39-bit (512GB) and 48-bit (256TB)
+virtual addresses, respectively, for both user and kernel. With 64KB
+pages, only 2 levels of translation tables are used but the memory layout
+is the same.

User addresses have bits 63:39 set to 0 while the kernel addresses have
the same bits set to 1. TTBRx selection is given by bit 63 of the
@@ -21,7 +22,7 @@ The swapper_pgd_dir address is written to TTBR1 and never written to
TTBR0.


-AArch64 Linux memory layout with 4KB pages:
+AArch64 Linux memory layout with 4KB pages + 3 levels:

Start End Size Use
-----------------------------------------------------------------------
@@ -48,7 +49,34 @@ ffffffbffc000000 ffffffbfffffffff 64MB modules
ffffffc000000000 ffffffffffffffff 256GB kernel logical memory map


-AArch64 Linux memory layout with 64KB pages:
+AArch64 Linux memory layout with 4KB pages + 4 levels:
+
+Start End Size Use
+-----------------------------------------------------------------------
+0000000000000000 0000ffffffffffff 256TB user
+
+ffff000000000000 ffff7bfffffeffff ~124TB vmalloc
+
+ffff7bffffff0000 ffff7bffffffffff 64KB [guard page]
+
+ffff7c0000000000 ffff7dffffffffff 2TB vmemmap
+
+ffff7e0000000000 ffff7ffffbbfffff ~2TB [guard, future vmmemap]
+
+ffff7ffffa000000 ffff7ffffaffffff 16MB PCI I/O space
+
+ffff7ffffb000000 ffff7ffffbbfffff 12MB [guard]
+
+ffff7ffffbc00000 ffff7ffffbdfffff 2MB earlyprintk device
+
+ffff7ffffbe00000 ffff7ffffbffffff 2MB [guard]
+
+ffff7ffffc000000 ffff7fffffffffff 64MB modules
+
+ffff800000000000 ffffffffffffffff 128TB kernel logical memory map
+
+
+AArch64 Linux memory layout with 64KB pages + 2 levels:

Start End Size Use
-----------------------------------------------------------------------
@@ -75,7 +103,7 @@ fffffdfffc000000 fffffdffffffffff 64MB modules
fffffe0000000000 ffffffffffffffff 2TB kernel logical memory map


-Translation table lookup with 4KB pages:
+Translation table lookup with 4KB pages + 3 levels:

+--------+--------+--------+--------+--------+--------+--------+--------+
|63 56|55 48|47 40|39 32|31 24|23 16|15 8|7 0|
@@ -90,7 +118,22 @@ Translation table lookup with 4KB pages:
+-------------------------------------------------> [63] TTBR0/1


-Translation table lookup with 64KB pages:
+Translation table lookup with 4KB pages + 4 levels:
+
++--------+--------+--------+--------+--------+--------+--------+--------+
+|63 56|55 48|47 40|39 32|31 24|23 16|15 8|7 0|
++--------+--------+--------+--------+--------+--------+--------+--------+
+ | | | | | |
+ | | | | | v
+ | | | | | [11:0] in-page offset
+ | | | | +-> [20:12] L3 index
+ | | | +-----------> [29:21] L2 index
+ | | +---------------------> [38:30] L1 index
+ | +-------------------------------> [47:39] L0 index
+ +-------------------------------------------------> [63] TTBR0/1
+
+
+Translation table lookup with 64KB pages + 2 levels:

+--------+--------+--------+--------+--------+--------+--------+--------+
|63 56|55 48|47 40|39 32|31 24|23 16|15 8|7 0|
--
1.7.10.4


2014-04-29 14:48:36

by Catalin Marinas

[permalink] [raw]
Subject: Re: [PATCH v4 4/7] arm64: Add a description on 48-bit address space with 4KB pages

On Tue, Apr 29, 2014 at 05:59:27AM +0100, Jungseok Lee wrote:
> --- a/Documentation/arm64/memory.txt
> +++ b/Documentation/arm64/memory.txt
> @@ -8,10 +8,11 @@ This document describes the virtual memory layout used by the AArch64
> Linux kernel. The architecture allows up to 4 levels of translation
> tables with a 4KB page size and up to 3 levels with a 64KB page size.
>
> -AArch64 Linux uses 3 levels of translation tables with the 4KB page
> -configuration, allowing 39-bit (512GB) virtual addresses for both user
> -and kernel. With 64KB pages, only 2 levels of translation tables are
> -used but the memory layout is the same.
> +AArch64 Linux uses 3 levels and 4 levels of translation tables with
> +the 4KB page configuration, allowing 39-bit (512GB) and 48-bit (256TB)
> +virtual addresses, respectively, for both user and kernel. With 64KB
> +pages, only 2 levels of translation tables are used but the memory layout
> +is the same.

Any reason why we couldn't use 48-bit address space with 64K pages
(implying 3 levels)?

> -AArch64 Linux memory layout with 64KB pages:
> +AArch64 Linux memory layout with 4KB pages + 4 levels:
> +
> +Start End Size Use
> +-----------------------------------------------------------------------
> +0000000000000000 0000ffffffffffff 256TB user
> +
> +ffff000000000000 ffff7bfffffeffff ~124TB vmalloc

BTW, maybe as a separate patch we should change the "end" to be
exclusive. It becomes harder to modify (I've been through this a few
times already ;)) and even follow the changes.

--
Catalin

2014-04-30 06:41:47

by Jungseok Lee

[permalink] [raw]
Subject: Re: [PATCH v4 4/7] arm64: Add a description on 48-bit address space with 4KB pages

On Tuesday, April 29, 2014 11:48 PM, Catalin Marinas wrote:
> On Tue, Apr 29, 2014 at 05:59:27AM +0100, Jungseok Lee wrote:
> > --- a/Documentation/arm64/memory.txt
> > +++ b/Documentation/arm64/memory.txt
> > @@ -8,10 +8,11 @@ This document describes the virtual memory layout
> > used by the AArch64 Linux kernel. The architecture allows up to 4
> > levels of translation tables with a 4KB page size and up to 3 levels with a 64KB page size.
> >
> > -AArch64 Linux uses 3 levels of translation tables with the 4KB page
> > -configuration, allowing 39-bit (512GB) virtual addresses for both
> > user -and kernel. With 64KB pages, only 2 levels of translation tables
> > are -used but the memory layout is the same.
> > +AArch64 Linux uses 3 levels and 4 levels of translation tables with
> > +the 4KB page configuration, allowing 39-bit (512GB) and 48-bit
> > +(256TB) virtual addresses, respectively, for both user and kernel.
> > +With 64KB pages, only 2 levels of translation tables are used but the
> > +memory layout is the same.
>
> Any reason why we couldn't use 48-bit address space with 64K pages (implying 3 levels)?

No technical reason.
Since 64K+3levels is not implemented in this set, I didn't add it.

Should 64K+3levels be prepared in this patchset?

> > -AArch64 Linux memory layout with 64KB pages:
> > +AArch64 Linux memory layout with 4KB pages + 4 levels:
> > +
> > +Start End Size Use
> > +-----------------------------------------------------------------------
> > +0000000000000000 0000ffffffffffff 256TB user
> > +
> > +ffff000000000000 ffff7bfffffeffff ~124TB vmalloc
>
> BTW, maybe as a separate patch we should change the "end" to be exclusive. It becomes harder to modify
> (I've been through this a few times already ;)) and even follow the changes.

Does "exclusive" mean that 0000ffffffffffff is changed to 0001000000000000?
Or Does it mean that "End" column is dropped?

If you are okay, I will make it as a separate patch.

Best Regards
Jungseok Lee

2014-04-30 13:13:02

by Catalin Marinas

[permalink] [raw]
Subject: Re: [PATCH v4 4/7] arm64: Add a description on 48-bit address space with 4KB pages

On Wed, Apr 30, 2014 at 07:41:40AM +0100, Jungseok Lee wrote:
> On Tuesday, April 29, 2014 11:48 PM, Catalin Marinas wrote:
> > On Tue, Apr 29, 2014 at 05:59:27AM +0100, Jungseok Lee wrote:
> > > --- a/Documentation/arm64/memory.txt
> > > +++ b/Documentation/arm64/memory.txt
> > > @@ -8,10 +8,11 @@ This document describes the virtual memory layout
> > > used by the AArch64 Linux kernel. The architecture allows up to 4
> > > levels of translation tables with a 4KB page size and up to 3 levels with a 64KB page size.
> > >
> > > -AArch64 Linux uses 3 levels of translation tables with the 4KB page
> > > -configuration, allowing 39-bit (512GB) virtual addresses for both
> > > user -and kernel. With 64KB pages, only 2 levels of translation tables
> > > are -used but the memory layout is the same.
> > > +AArch64 Linux uses 3 levels and 4 levels of translation tables with
> > > +the 4KB page configuration, allowing 39-bit (512GB) and 48-bit
> > > +(256TB) virtual addresses, respectively, for both user and kernel.
> > > +With 64KB pages, only 2 levels of translation tables are used but the
> > > +memory layout is the same.
> >
> > Any reason why we couldn't use 48-bit address space with 64K pages (implying 3 levels)?
>
> No technical reason.
> Since 64K+3levels is not implemented in this set, I didn't add it.
>
> Should 64K+3levels be prepared in this patchset?
>
> > > -AArch64 Linux memory layout with 64KB pages:
> > > +AArch64 Linux memory layout with 4KB pages + 4 levels:
> > > +
> > > +Start End Size Use
> > > +-----------------------------------------------------------------------
> > > +0000000000000000 0000ffffffffffff 256TB user
> > > +
> > > +ffff000000000000 ffff7bfffffeffff ~124TB vmalloc
> >
> > BTW, maybe as a separate patch we should change the "end" to be exclusive. It becomes harder to modify
> > (I've been through this a few times already ;)) and even follow the changes.
>
> Does "exclusive" mean that 0000ffffffffffff is changed to 0001000000000000?
> Or Does it mean that "End" column is dropped?

Not dropped but changed to 0001.... (the kernel already prints the
memory layout in a similar way).

--
Catalin

2014-05-01 00:53:12

by Jungseok Lee

[permalink] [raw]
Subject: Re: [PATCH v4 4/7] arm64: Add a description on 48-bit address space with 4KB pages

On Wednesday, April 30, 2014 10:12 PM, Catalin Marinas wrote:
> On Wed, Apr 30, 2014 at 07:41:40AM +0100, Jungseok Lee wrote:
> > On Tuesday, April 29, 2014 11:48 PM, Catalin Marinas wrote:
> > > On Tue, Apr 29, 2014 at 05:59:27AM +0100, Jungseok Lee wrote:
> > > > --- a/Documentation/arm64/memory.txt
> > > > +++ b/Documentation/arm64/memory.txt
> > > > @@ -8,10 +8,11 @@ This document describes the virtual memory
> > > > layout used by the AArch64 Linux kernel. The architecture allows
> > > > up to 4 levels of translation tables with a 4KB page size and up to 3 levels with a 64KB page
> size.
> > > >
> > > > -AArch64 Linux uses 3 levels of translation tables with the 4KB
> > > > page -configuration, allowing 39-bit (512GB) virtual addresses for
> > > > both user -and kernel. With 64KB pages, only 2 levels of
> > > > translation tables are -used but the memory layout is the same.
> > > > +AArch64 Linux uses 3 levels and 4 levels of translation tables
> > > > +with the 4KB page configuration, allowing 39-bit (512GB) and
> > > > +48-bit
> > > > +(256TB) virtual addresses, respectively, for both user and kernel.
> > > > +With 64KB pages, only 2 levels of translation tables are used but
> > > > +the memory layout is the same.
> > >
> > > Any reason why we couldn't use 48-bit address space with 64K pages (implying 3 levels)?
> >
> > No technical reason.
> > Since 64K+3levels is not implemented in this set, I didn't add it.
> >
> > Should 64K+3levels be prepared in this patchset?
> >
> > > > -AArch64 Linux memory layout with 64KB pages:
> > > > +AArch64 Linux memory layout with 4KB pages + 4 levels:
> > > > +
> > > > +Start End Size Use
> > > > +-----------------------------------------------------------------------
> > > > +0000000000000000 0000ffffffffffff 256TB user
> > > > +
> > > > +ffff000000000000 ffff7bfffffeffff ~124TB vmalloc
> > >
> > > BTW, maybe as a separate patch we should change the "end" to be
> > > exclusive. It becomes harder to modify (I've been through this a few times already ;)) and even
> follow the changes.
> >
> > Does "exclusive" mean that 0000ffffffffffff is changed to 0001000000000000?
> > Or Does it mean that "End" column is dropped?
>
> Not dropped but changed to 0001.... (the kernel already prints the memory layout in a similar way).

I see. I will make it as a separate patch.

Best Regards
Jungseok Lee