2014-12-04 15:46:35

by Alexander Graf

[permalink] [raw]
Subject: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

With binutils 2.25 the default alignment for 32bit arm sections changed to
have everything 64k aligned. Armv7 binaries built with this binutils version
run successfully on an arm64 system.

Since effectively there is now the chance to run armv7 code on arm64 even
with 64k page size, it doesn't make sense to block people from enabling
CONFIG_COMPAT on those configurations.

Signed-off-by: Alexander Graf <[email protected]>
---
arch/arm64/Kconfig | 1 -
1 file changed, 1 deletion(-)

diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 9532f8d..3cf4f238 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"

config COMPAT
bool "Kernel support for 32-bit EL0"
- depends on !ARM64_64K_PAGES
select COMPAT_BINFMT_ELF
select HAVE_UID16
select OLD_SIGSUSPEND3
--
1.7.12.4


2014-12-04 18:18:30

by Laura Abbott

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On 12/4/2014 7:46 AM, Alexander Graf wrote:
> With binutils 2.25 the default alignment for 32bit arm sections changed to
> have everything 64k aligned. Armv7 binaries built with this binutils version
> run successfully on an arm64 system.
>
> Since effectively there is now the chance to run armv7 code on arm64 even
> with 64k page size, it doesn't make sense to block people from enabling
> CONFIG_COMPAT on those configurations.
>

It's good to see this restriction dropped. Can we put a note somewhere
in the actual Kconfig about the binutils requirement so the requirements
are more obvious?

Thanks,
Laura

--
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
hosted by The Linux Foundation

2014-12-04 18:20:51

by Will Deacon

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On Thu, Dec 04, 2014 at 03:46:33PM +0000, Alexander Graf wrote:
> With binutils 2.25 the default alignment for 32bit arm sections changed to
> have everything 64k aligned. Armv7 binaries built with this binutils version
> run successfully on an arm64 system.
>
> Since effectively there is now the chance to run armv7 code on arm64 even
> with 64k page size, it doesn't make sense to block people from enabling
> CONFIG_COMPAT on those configurations.

Is there a distro available that is built with a recent enough binutils for
this? I'd really like to run our regression tests to check that page-size
assumptions don't exist for things like shm.

Will

>
> Signed-off-by: Alexander Graf <[email protected]>
> ---
> arch/arm64/Kconfig | 1 -
> 1 file changed, 1 deletion(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 9532f8d..3cf4f238 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"
>
> config COMPAT
> bool "Kernel support for 32-bit EL0"
> - depends on !ARM64_64K_PAGES
> select COMPAT_BINFMT_ELF
> select HAVE_UID16
> select OLD_SIGSUSPEND3
> --
> 1.7.12.4
>
>

2014-12-04 21:15:16

by Olof Johansson

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On Thu, Dec 4, 2014 at 7:46 AM, Alexander Graf <[email protected]> wrote:
> With binutils 2.25 the default alignment for 32bit arm sections changed to
> have everything 64k aligned. Armv7 binaries built with this binutils version
> run successfully on an arm64 system.
>
> Since effectively there is now the chance to run armv7 code on arm64 even
> with 64k page size, it doesn't make sense to block people from enabling
> CONFIG_COMPAT on those configurations.
>
> Signed-off-by: Alexander Graf <[email protected]>
> ---
> arch/arm64/Kconfig | 1 -
> 1 file changed, 1 deletion(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 9532f8d..3cf4f238 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"
>
> config COMPAT
> bool "Kernel support for 32-bit EL0"
> - depends on !ARM64_64K_PAGES
> select COMPAT_BINFMT_ELF
> select HAVE_UID16
> select OLD_SIGSUSPEND3

This is hardly "compat". Sure, it's great to have a new binutils that
has larger alignment, but practically not a single existing binary
will work today if someone tries to do this.

So, it seems very premature to take this off. At the very least
document it like Will requested, and make it depend on !ARM_64K_PAGES
|| EXPERT.


-Olof

2014-12-04 23:38:05

by Alexander Graf

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size



On 04.12.14 19:20, Will Deacon wrote:
> On Thu, Dec 04, 2014 at 03:46:33PM +0000, Alexander Graf wrote:
>> With binutils 2.25 the default alignment for 32bit arm sections changed to
>> have everything 64k aligned. Armv7 binaries built with this binutils version
>> run successfully on an arm64 system.
>>
>> Since effectively there is now the chance to run armv7 code on arm64 even
>> with 64k page size, it doesn't make sense to block people from enabling
>> CONFIG_COMPAT on those configurations.
>
> Is there a distro available that is built with a recent enough binutils for
> this? I'd really like to run our regression tests to check that page-size
> assumptions don't exist for things like shm.

I don't have a full distribution for you yet, but we'll start to switch
to the 2.25 branch soon.

Michael, what are you plans on when a first armv7 Factory build based on
2.25 would be available?


Alex

2014-12-04 23:41:36

by Alexander Graf

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size



On 04.12.14 22:15, Olof Johansson wrote:
> On Thu, Dec 4, 2014 at 7:46 AM, Alexander Graf <[email protected]> wrote:
>> With binutils 2.25 the default alignment for 32bit arm sections changed to
>> have everything 64k aligned. Armv7 binaries built with this binutils version
>> run successfully on an arm64 system.
>>
>> Since effectively there is now the chance to run armv7 code on arm64 even
>> with 64k page size, it doesn't make sense to block people from enabling
>> CONFIG_COMPAT on those configurations.
>>
>> Signed-off-by: Alexander Graf <[email protected]>
>> ---
>> arch/arm64/Kconfig | 1 -
>> 1 file changed, 1 deletion(-)
>>
>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>> index 9532f8d..3cf4f238 100644
>> --- a/arch/arm64/Kconfig
>> +++ b/arch/arm64/Kconfig
>> @@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"
>>
>> config COMPAT
>> bool "Kernel support for 32-bit EL0"
>> - depends on !ARM64_64K_PAGES
>> select COMPAT_BINFMT_ELF
>> select HAVE_UID16
>> select OLD_SIGSUSPEND3
>
> This is hardly "compat". Sure, it's great to have a new binutils that
> has larger alignment, but practically not a single existing binary
> will work today if someone tries to do this.

Yes, but IMHO that's an implementation detail. The same applies for
32bit PPC binaries if you use 4k aligned segments. If your applications
are not aligned for your page size, you can't run them. The only
platform that managed nevertheless FWIW was IA64 ;).

> So, it seems very premature to take this off. At the very least
> document it like Will requested, and make it depend on !ARM_64K_PAGES
> || EXPERT.

Sure, that certainly works for me (though you probably mean Laura).
Completely restricting users from enabling it just because our current
toolchain doesn't generate compatible binaries sounds a bit extreme to me.


Alex

2014-12-04 23:48:57

by Olof Johansson

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On Thu, Dec 4, 2014 at 3:41 PM, Alexander Graf <[email protected]> wrote:
>
>
> On 04.12.14 22:15, Olof Johansson wrote:
>> On Thu, Dec 4, 2014 at 7:46 AM, Alexander Graf <[email protected]> wrote:
>>> With binutils 2.25 the default alignment for 32bit arm sections changed to
>>> have everything 64k aligned. Armv7 binaries built with this binutils version
>>> run successfully on an arm64 system.
>>>
>>> Since effectively there is now the chance to run armv7 code on arm64 even
>>> with 64k page size, it doesn't make sense to block people from enabling
>>> CONFIG_COMPAT on those configurations.
>>>
>>> Signed-off-by: Alexander Graf <[email protected]>
>>> ---
>>> arch/arm64/Kconfig | 1 -
>>> 1 file changed, 1 deletion(-)
>>>
>>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>>> index 9532f8d..3cf4f238 100644
>>> --- a/arch/arm64/Kconfig
>>> +++ b/arch/arm64/Kconfig
>>> @@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"
>>>
>>> config COMPAT
>>> bool "Kernel support for 32-bit EL0"
>>> - depends on !ARM64_64K_PAGES
>>> select COMPAT_BINFMT_ELF
>>> select HAVE_UID16
>>> select OLD_SIGSUSPEND3
>>
>> This is hardly "compat". Sure, it's great to have a new binutils that
>> has larger alignment, but practically not a single existing binary
>> will work today if someone tries to do this.
>
> Yes, but IMHO that's an implementation detail. The same applies for
> 32bit PPC binaries if you use 4k aligned segments. If your applications
> are not aligned for your page size, you can't run them. The only
> platform that managed nevertheless FWIW was IA64 ;).

Yes, but there the binutils change happened early enough that by the
time the kernel change went in, all major distros had binaries that
were compatible.

>> So, it seems very premature to take this off. At the very least
>> document it like Will requested, and make it depend on !ARM_64K_PAGES
>> || EXPERT.
>
> Sure, that certainly works for me (though you probably mean Laura).

Yeah, sorry Laura. :)

> Completely restricting users from enabling it just because our current
> toolchain doesn't generate compatible binaries sounds a bit extreme to me.

Perhaps.

It's in many ways a timing thing too, if ARM had patched binutils a
couple of years ago it might have been fine to just go with it without
EXPERT now. :)


-Olof

2014-12-05 10:40:11

by Arnd Bergmann

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On Thursday 04 December 2014 15:48:50 Olof Johansson wrote:
> On Thu, Dec 4, 2014 at 3:41 PM, Alexander Graf <[email protected]> wrote:
> > On 04.12.14 22:15, Olof Johansson wrote:
> >> On Thu, Dec 4, 2014 at 7:46 AM, Alexander Graf <[email protected]> wrote:
> >>> With binutils 2.25 the default alignment for 32bit arm sections changed to
> >>> have everything 64k aligned. Armv7 binaries built with this binutils version
> >>> run successfully on an arm64 system.
> >>>
> >>> Since effectively there is now the chance to run armv7 code on arm64 even
> >>> with 64k page size, it doesn't make sense to block people from enabling
> >>> CONFIG_COMPAT on those configurations.
> >>>
> >>> Signed-off-by: Alexander Graf <[email protected]>
> >>> ---
> >>> arch/arm64/Kconfig | 1 -
> >>> 1 file changed, 1 deletion(-)
> >>>
> >>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> >>> index 9532f8d..3cf4f238 100644
> >>> --- a/arch/arm64/Kconfig
> >>> +++ b/arch/arm64/Kconfig
> >>> @@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"
> >>>
> >>> config COMPAT
> >>> bool "Kernel support for 32-bit EL0"
> >>> - depends on !ARM64_64K_PAGES
> >>> select COMPAT_BINFMT_ELF
> >>> select HAVE_UID16
> >>> select OLD_SIGSUSPEND3
> >>
> >> This is hardly "compat". Sure, it's great to have a new binutils that
> >> has larger alignment, but practically not a single existing binary
> >> will work today if someone tries to do this.
> >
> > Yes, but IMHO that's an implementation detail. The same applies for
> > 32bit PPC binaries if you use 4k aligned segments. If your applications
> > are not aligned for your page size, you can't run them. The only
> > platform that managed nevertheless FWIW was IA64 ;).
>
> Yes, but there the binutils change happened early enough that by the
> time the kernel change went in, all major distros had binaries that
> were compatible.

What is the exact symptom you see when running an unaligned user
space binary on 64k-pages? Do we at least print a helpful error
message somewhere or does it just crash?

Should we add support for 64k-pages in the arm32 kernel as well now?

Arnd

2014-12-05 11:35:38

by Will Deacon

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On Fri, Dec 05, 2014 at 11:14:55AM +0000, Catalin Marinas wrote:
> On Thu, Dec 04, 2014 at 09:15:12PM +0000, Olof Johansson wrote:
> > On Thu, Dec 4, 2014 at 7:46 AM, Alexander Graf <[email protected]> wrote:
> > > With binutils 2.25 the default alignment for 32bit arm sections changed to
> > > have everything 64k aligned. Armv7 binaries built with this binutils version
> > > run successfully on an arm64 system.
> > >
> > > Since effectively there is now the chance to run armv7 code on arm64 even
> > > with 64k page size, it doesn't make sense to block people from enabling
> > > CONFIG_COMPAT on those configurations.
> > >
> > > Signed-off-by: Alexander Graf <[email protected]>
> > > ---
> > > arch/arm64/Kconfig | 1 -
> > > 1 file changed, 1 deletion(-)
> > >
> > > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > > index 9532f8d..3cf4f238 100644
> > > --- a/arch/arm64/Kconfig
> > > +++ b/arch/arm64/Kconfig
> > > @@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"
> > >
> > > config COMPAT
> > > bool "Kernel support for 32-bit EL0"
> > > - depends on !ARM64_64K_PAGES
> > > select COMPAT_BINFMT_ELF
> > > select HAVE_UID16
> > > select OLD_SIGSUSPEND3
> >
> > This is hardly "compat". Sure, it's great to have a new binutils that
> > has larger alignment, but practically not a single existing binary
> > will work today if someone tries to do this.
> >
> > So, it seems very premature to take this off. At the very least
> > document it like Will requested, and make it depend on !ARM_64K_PAGES
> > || EXPERT.
>
> That would work for me. We need to be clear that most existing 32-bit
> binaries will fail.

I'd still like to run some regression tests to make sure we don't back
ourselves into corners with things like SHMLBA, which would now be less
than a page size.

Will

2014-12-05 12:31:32

by Catalin Marinas

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On Fri, Dec 05, 2014 at 12:24:49PM +0000, Arnd Bergmann wrote:
> On Friday 05 December 2014 11:05:07 Catalin Marinas wrote:
> >
> > > Should we add support for 64k-pages in the arm32 kernel as well now?
> >
> > 32-bit LPAE doesn't support 64K pages but IIRC the classic MMU does
> > (though I have to check whether it was optional). But it's not feasible
> > to enable this in a 32-bit environment because of the memory wasted with
> > large pages (on average 32KB per mapped file, possibly even more if you
> > have lots of small files).
>
> That sounds like a tradeoff that some users might be willing to make.
>
> I remember that with ps3-linux, a lot of people wanted to enable 64k-pages
> because the Cell processor has such a small number of TLB entries (256 or
> 1024), but Linux only has around 200MB of memory available. I believe
> there are arm32 machines with more RAM and fewer TLBs.

I've heard that the WD MyCloud NAS (Cortex-A9 based) moved to 64K pages
and they rebuilt the filesystem (which annoyed some people that were
previously installing Debian packages directly, they now need
recompilation).

--
Catalin

2014-12-05 12:06:14

by Alexander Graf

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size



On 05.12.14 11:39, Arnd Bergmann wrote:
> On Thursday 04 December 2014 15:48:50 Olof Johansson wrote:
>> On Thu, Dec 4, 2014 at 3:41 PM, Alexander Graf <[email protected]> wrote:
>>> On 04.12.14 22:15, Olof Johansson wrote:
>>>> On Thu, Dec 4, 2014 at 7:46 AM, Alexander Graf <[email protected]> wrote:
>>>>> With binutils 2.25 the default alignment for 32bit arm sections changed to
>>>>> have everything 64k aligned. Armv7 binaries built with this binutils version
>>>>> run successfully on an arm64 system.
>>>>>
>>>>> Since effectively there is now the chance to run armv7 code on arm64 even
>>>>> with 64k page size, it doesn't make sense to block people from enabling
>>>>> CONFIG_COMPAT on those configurations.
>>>>>
>>>>> Signed-off-by: Alexander Graf <[email protected]>
>>>>> ---
>>>>> arch/arm64/Kconfig | 1 -
>>>>> 1 file changed, 1 deletion(-)
>>>>>
>>>>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>>>>> index 9532f8d..3cf4f238 100644
>>>>> --- a/arch/arm64/Kconfig
>>>>> +++ b/arch/arm64/Kconfig
>>>>> @@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"
>>>>>
>>>>> config COMPAT
>>>>> bool "Kernel support for 32-bit EL0"
>>>>> - depends on !ARM64_64K_PAGES
>>>>> select COMPAT_BINFMT_ELF
>>>>> select HAVE_UID16
>>>>> select OLD_SIGSUSPEND3
>>>>
>>>> This is hardly "compat". Sure, it's great to have a new binutils that
>>>> has larger alignment, but practically not a single existing binary
>>>> will work today if someone tries to do this.
>>>
>>> Yes, but IMHO that's an implementation detail. The same applies for
>>> 32bit PPC binaries if you use 4k aligned segments. If your applications
>>> are not aligned for your page size, you can't run them. The only
>>> platform that managed nevertheless FWIW was IA64 ;).
>>
>> Yes, but there the binutils change happened early enough that by the
>> time the kernel change went in, all major distros had binaries that
>> were compatible.
>
> What is the exact symptom you see when running an unaligned user
> space binary on 64k-pages? Do we at least print a helpful error
> message somewhere or does it just crash?

It simply doesn't start:

init-4.2# /hello.binutils-2.23
init-4.2# echo $?
139
init-4.2# /hello.binutils-2.25





Hello world!





init-4.2# echo $?
0
init-4.2#


I'm not sure how to give the user an actually helpful error output here
though. The only real handle we have for executing a binary is to return
an error code.


Alex

2014-12-05 12:25:22

by Arnd Bergmann

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On Friday 05 December 2014 11:05:07 Catalin Marinas wrote:
>
> > Should we add support for 64k-pages in the arm32 kernel as well now?
>
> 32-bit LPAE doesn't support 64K pages but IIRC the classic MMU does
> (though I have to check whether it was optional). But it's not feasible
> to enable this in a 32-bit environment because of the memory wasted with
> large pages (on average 32KB per mapped file, possibly even more if you
> have lots of small files).

That sounds like a tradeoff that some users might be willing to make.

I remember that with ps3-linux, a lot of people wanted to enable 64k-pages
because the Cell processor has such a small number of TLB entries (256 or
1024), but Linux only has around 200MB of memory available. I believe
there are arm32 machines with more RAM and fewer TLBs.

Arnd

2014-12-05 11:15:06

by Catalin Marinas

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On Thu, Dec 04, 2014 at 09:15:12PM +0000, Olof Johansson wrote:
> On Thu, Dec 4, 2014 at 7:46 AM, Alexander Graf <[email protected]> wrote:
> > With binutils 2.25 the default alignment for 32bit arm sections changed to
> > have everything 64k aligned. Armv7 binaries built with this binutils version
> > run successfully on an arm64 system.
> >
> > Since effectively there is now the chance to run armv7 code on arm64 even
> > with 64k page size, it doesn't make sense to block people from enabling
> > CONFIG_COMPAT on those configurations.
> >
> > Signed-off-by: Alexander Graf <[email protected]>
> > ---
> > arch/arm64/Kconfig | 1 -
> > 1 file changed, 1 deletion(-)
> >
> > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > index 9532f8d..3cf4f238 100644
> > --- a/arch/arm64/Kconfig
> > +++ b/arch/arm64/Kconfig
> > @@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"
> >
> > config COMPAT
> > bool "Kernel support for 32-bit EL0"
> > - depends on !ARM64_64K_PAGES
> > select COMPAT_BINFMT_ELF
> > select HAVE_UID16
> > select OLD_SIGSUSPEND3
>
> This is hardly "compat". Sure, it's great to have a new binutils that
> has larger alignment, but practically not a single existing binary
> will work today if someone tries to do this.
>
> So, it seems very premature to take this off. At the very least
> document it like Will requested, and make it depend on !ARM_64K_PAGES
> || EXPERT.

That would work for me. We need to be clear that most existing 32-bit
binaries will fail.

--
Catalin

2014-12-05 11:05:22

by Catalin Marinas

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On Fri, Dec 05, 2014 at 10:39:40AM +0000, Arnd Bergmann wrote:
> On Thursday 04 December 2014 15:48:50 Olof Johansson wrote:
> > On Thu, Dec 4, 2014 at 3:41 PM, Alexander Graf <[email protected]> wrote:
> > > On 04.12.14 22:15, Olof Johansson wrote:
> > >> On Thu, Dec 4, 2014 at 7:46 AM, Alexander Graf <[email protected]> wrote:
> > >>> With binutils 2.25 the default alignment for 32bit arm sections changed to
> > >>> have everything 64k aligned. Armv7 binaries built with this binutils version
> > >>> run successfully on an arm64 system.
> > >>>
> > >>> Since effectively there is now the chance to run armv7 code on arm64 even
> > >>> with 64k page size, it doesn't make sense to block people from enabling
> > >>> CONFIG_COMPAT on those configurations.
> > >>>
> > >>> Signed-off-by: Alexander Graf <[email protected]>
> > >>> ---
> > >>> arch/arm64/Kconfig | 1 -
> > >>> 1 file changed, 1 deletion(-)
> > >>>
> > >>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > >>> index 9532f8d..3cf4f238 100644
> > >>> --- a/arch/arm64/Kconfig
> > >>> +++ b/arch/arm64/Kconfig
> > >>> @@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"
> > >>>
> > >>> config COMPAT
> > >>> bool "Kernel support for 32-bit EL0"
> > >>> - depends on !ARM64_64K_PAGES
> > >>> select COMPAT_BINFMT_ELF
> > >>> select HAVE_UID16
> > >>> select OLD_SIGSUSPEND3
> > >>
> > >> This is hardly "compat". Sure, it's great to have a new binutils that
> > >> has larger alignment, but practically not a single existing binary
> > >> will work today if someone tries to do this.
> > >
> > > Yes, but IMHO that's an implementation detail. The same applies for
> > > 32bit PPC binaries if you use 4k aligned segments. If your applications
> > > are not aligned for your page size, you can't run them. The only
> > > platform that managed nevertheless FWIW was IA64 ;).
> >
> > Yes, but there the binutils change happened early enough that by the
> > time the kernel change went in, all major distros had binaries that
> > were compatible.
>
> What is the exact symptom you see when running an unaligned user
> space binary on 64k-pages? Do we at least print a helpful error
> message somewhere or does it just crash?

The application doesn't even start because it cannot map page 0. It
looks like most 32-bit arm binaries are linked to be loaded at 32K.

> Should we add support for 64k-pages in the arm32 kernel as well now?

32-bit LPAE doesn't support 64K pages but IIRC the classic MMU does
(though I have to check whether it was optional). But it's not feasible
to enable this in a 32-bit environment because of the memory wasted with
large pages (on average 32KB per mapped file, possibly even more if you
have lots of small files).

--
Catalin

2014-12-05 16:35:17

by Liviu Dudau

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On Thu, Dec 04, 2014 at 03:46:33PM +0000, Alexander Graf wrote:
> With binutils 2.25 the default alignment for 32bit arm sections changed to
> have everything 64k aligned. Armv7 binaries built with this binutils version
> run successfully on an arm64 system.
>
> Since effectively there is now the chance to run armv7 code on arm64 even
> with 64k page size, it doesn't make sense to block people from enabling
> CONFIG_COMPAT on those configurations.
>
> Signed-off-by: Alexander Graf <[email protected]>
> ---
> arch/arm64/Kconfig | 1 -
> 1 file changed, 1 deletion(-)
>
> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> index 9532f8d..3cf4f238 100644
> --- a/arch/arm64/Kconfig
> +++ b/arch/arm64/Kconfig
> @@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"
>
> config COMPAT
> bool "Kernel support for 32-bit EL0"
> - depends on !ARM64_64K_PAGES
> select COMPAT_BINFMT_ELF
> select HAVE_UID16
> select OLD_SIGSUSPEND3
> --
> 1.7.12.4
>

Hi Alexander,

For what is worth, today's linux-next (20141204) does not boot on arm64
if I select CONFIG_ARM64_64K_PAGES. ext4 filesystem fails to mount with:

hub 1-1:1.0: USB hub found
hub 1-1:1.0: 4 ports detected
usb 1-1.2: new high-speed USB device number 3 using ehci-platform
usb-storage 1-1.2:1.0: USB Mass Storage device detected
scsi host0: usb-storage 1-1.2:1.0
atkbd serio0: keyboard reset failed on 1c060000.kmi
Waiting for root device /dev/sda3...
usb 1-1.1: new low-speed USB device number 4 using ehci-platform
input: USB Keyboard as /devices/platform/7ffc0000.ehci/usb1/1-1/1-1.1/1-1.1:1.0/0003:0DC6:3910.0001/input/input2
hid-generic 0003:0DC6:3910.0001: input: USB HID v1.00 Keyboard [USB Keyboard] on usb-7ffc0000.ehci-1.1/input0
input: USB Keyboard as /devices/platform/7ffc0000.ehci/usb1/1-1/1-1.1/1-1.1:1.1/0003:0DC6:3910.0002/input/input3
hid-generic 0003:0DC6:3910.0002: input: USB HID v1.00 Mouse [USB Keyboard] on usb-7ffc0000.ehci-1.1/input1
atkbd serio1: keyboard reset failed on 1c070000.kmi
scsi 0:0:0:0: Direct-Access Toshiba StorE HDD 0000 PQ: 0 ANSI: 4
sd 0:0:0:0: [sda] 976773168 512-byte logical blocks: (500 GB/465 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 38 00 00 00
sd 0:0:0:0: [sda] No Caching mode page found
sd 0:0:0:0: [sda] Assuming drive cache: write through
sda: sda1 sda2 sda3
sd 0:0:0:0: [sda] Attached SCSI disk
EXT3-fs (sda3): error: couldn't mount because of unsupported optional features (240)
EXT2-fs (sda3): error: couldn't mount because of unsupported optional features (240)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 128 failed (15930!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 129 failed (54075!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 130 failed (42043!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 131 failed (18746!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 132 failed (19003!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 133 failed (42810!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 134 failed (53306!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 135 failed (15675!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 136 failed (54840!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 137 failed (15161!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 138 failed (19513!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 139 failed (41272!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 140 failed (41529!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 141 failed (20280!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 142 failed (14392!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 143 failed (54585!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 144 failed (44605!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 145 failed (17212!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 146 failed (13372!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 147 failed (55613!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 148 failed (55868!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 149 failed (14141!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 150 failed (16445!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 151 failed (44348!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 152 failed (17983!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 153 failed (43838!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 154 failed (56382!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 155 failed (12607!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 156 failed (12862!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 157 failed (57151!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 158 failed (43071!=0)
EXT4-fs (sda3): ext4_check_descriptors: Checksum for group 159 failed (17726!=0)
EXT4-fs (sda3): ext4_check_descriptors: Inode bitmap for group 160 not in group (block 89328390)!
EXT4-fs (sda3): group descriptors corrupted!
List of all partitions:
b300 1931264 mmcblk0 driver: mmcblk
b301 1930240 mmcblk0p1 00000000-01
0800 488386584 sda driver: sd
0801 125829120 sda1 00000000-01
0802 5242880 sda2 00000000-02
0803 357313560 sda3 00000000-03
No filesystem could mount root, tried: ext3 ext2 ext4 vfat fuseblk
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(8,3)
CPU: 1 PID: 1 Comm: swapper/0 Not tainted 3.18.0-rc7-next-20141204+ #1


Same filesystem (no fsck inbetween) but with a 4K pages kernel boots fine.

Best regards,
Liviu

--
====================
| I would like to |
| fix the world, |
| but they're not |
| giving me the |
\ source code! /
---------------
¯\_(ツ)_/¯

2014-12-06 17:24:05

by Alexander Graf

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size



On 04.12.14 19:20, Will Deacon wrote:
> On Thu, Dec 04, 2014 at 03:46:33PM +0000, Alexander Graf wrote:
>> With binutils 2.25 the default alignment for 32bit arm sections changed to
>> have everything 64k aligned. Armv7 binaries built with this binutils version
>> run successfully on an arm64 system.
>>
>> Since effectively there is now the chance to run armv7 code on arm64 even
>> with 64k page size, it doesn't make sense to block people from enabling
>> CONFIG_COMPAT on those configurations.
>
> Is there a distro available that is built with a recent enough binutils for
> this? I'd really like to run our regression tests to check that page-size
> assumptions don't exist for things like shm.

So how much of a distro do you need? I could probably assemble a simple
very minimalistic rootfs with only bash if that helps.


Alex

2014-12-08 10:10:31

by Will Deacon

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On Sat, Dec 06, 2014 at 05:23:59PM +0000, Alexander Graf wrote:
> On 04.12.14 19:20, Will Deacon wrote:
> > On Thu, Dec 04, 2014 at 03:46:33PM +0000, Alexander Graf wrote:
> >> With binutils 2.25 the default alignment for 32bit arm sections changed to
> >> have everything 64k aligned. Armv7 binaries built with this binutils version
> >> run successfully on an arm64 system.
> >>
> >> Since effectively there is now the chance to run armv7 code on arm64 even
> >> with 64k page size, it doesn't make sense to block people from enabling
> >> CONFIG_COMPAT on those configurations.
> >
> > Is there a distro available that is built with a recent enough binutils for
> > this? I'd really like to run our regression tests to check that page-size
> > assumptions don't exist for things like shm.
>
> So how much of a distro do you need? I could probably assemble a simple
> very minimalistic rootfs with only bash if that helps.

I'd like to run LTP, so I'd probably need slightly more than that but I
certainly don't need the whole world.

Will

2014-12-08 10:47:05

by Alexander Graf

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size



On 08.12.14 11:10, Will Deacon wrote:
> On Sat, Dec 06, 2014 at 05:23:59PM +0000, Alexander Graf wrote:
>> On 04.12.14 19:20, Will Deacon wrote:
>>> On Thu, Dec 04, 2014 at 03:46:33PM +0000, Alexander Graf wrote:
>>>> With binutils 2.25 the default alignment for 32bit arm sections changed to
>>>> have everything 64k aligned. Armv7 binaries built with this binutils version
>>>> run successfully on an arm64 system.
>>>>
>>>> Since effectively there is now the chance to run armv7 code on arm64 even
>>>> with 64k page size, it doesn't make sense to block people from enabling
>>>> CONFIG_COMPAT on those configurations.
>>>
>>> Is there a distro available that is built with a recent enough binutils for
>>> this? I'd really like to run our regression tests to check that page-size
>>> assumptions don't exist for things like shm.
>>
>> So how much of a distro do you need? I could probably assemble a simple
>> very minimalistic rootfs with only bash if that helps.
>
> I'd like to run LTP, so I'd probably need slightly more than that but I
> certainly don't need the whole world.

Let me check with a few folks what we can easily assemble :).


Alex

2014-12-08 13:47:13

by Michael Matz

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

Hi,

On Fri, 5 Dec 2014, Alexander Graf wrote:

> I don't have a full distribution for you yet, but we'll start to switch
> to the 2.25 branch soon.
>
> Michael, what are you plans on when a first armv7 Factory build based on
> 2.25 would be available?

2.25 isn't released yet, but I just submitted the pre-release version to
Factory, so the answer would be "as soon as it's checked in" :)


Ciao,
Michael.

2015-02-18 13:40:56

by Christopher Covington

[permalink] [raw]
Subject: Re: [PATCH] arm64: Enable CONFIG_COMPAT also for 64k page size

On 12/05/2014 06:05 AM, Catalin Marinas wrote:
> On Fri, Dec 05, 2014 at 10:39:40AM +0000, Arnd Bergmann wrote:
>> On Thursday 04 December 2014 15:48:50 Olof Johansson wrote:
>>> On Thu, Dec 4, 2014 at 3:41 PM, Alexander Graf <[email protected]> wrote:
>>>> On 04.12.14 22:15, Olof Johansson wrote:
>>>>> On Thu, Dec 4, 2014 at 7:46 AM, Alexander Graf <[email protected]> wrote:
>>>>>> With binutils 2.25 the default alignment for 32bit arm sections changed to
>>>>>> have everything 64k aligned. Armv7 binaries built with this binutils version
>>>>>> run successfully on an arm64 system.
>>>>>>
>>>>>> Since effectively there is now the chance to run armv7 code on arm64 even
>>>>>> with 64k page size, it doesn't make sense to block people from enabling
>>>>>> CONFIG_COMPAT on those configurations.
>>>>>>
>>>>>> Signed-off-by: Alexander Graf <[email protected]>
>>>>>> ---
>>>>>> arch/arm64/Kconfig | 1 -
>>>>>> 1 file changed, 1 deletion(-)
>>>>>>
>>>>>> diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
>>>>>> index 9532f8d..3cf4f238 100644
>>>>>> --- a/arch/arm64/Kconfig
>>>>>> +++ b/arch/arm64/Kconfig
>>>>>> @@ -409,7 +409,6 @@ source "fs/Kconfig.binfmt"
>>>>>>
>>>>>> config COMPAT
>>>>>> bool "Kernel support for 32-bit EL0"
>>>>>> - depends on !ARM64_64K_PAGES
>>>>>> select COMPAT_BINFMT_ELF
>>>>>> select HAVE_UID16
>>>>>> select OLD_SIGSUSPEND3
>>>>>
>>>>> This is hardly "compat". Sure, it's great to have a new binutils that
>>>>> has larger alignment, but practically not a single existing binary
>>>>> will work today if someone tries to do this.
>>>>
>>>> Yes, but IMHO that's an implementation detail. The same applies for
>>>> 32bit PPC binaries if you use 4k aligned segments. If your applications
>>>> are not aligned for your page size, you can't run them. The only
>>>> platform that managed nevertheless FWIW was IA64 ;).
>>>
>>> Yes, but there the binutils change happened early enough that by the
>>> time the kernel change went in, all major distros had binaries that
>>> were compatible.
>>
>> What is the exact symptom you see when running an unaligned user
>> space binary on 64k-pages? Do we at least print a helpful error
>> message somewhere or does it just crash?
>
> The application doesn't even start because it cannot map page 0. It
> looks like most 32-bit arm binaries are linked to be loaded at 32K.

I looked at this briefly and the first problem I narrowed in on was underflow
in the calculation of the off variable in elf_map().

http://lxr.free-electrons.com/source/fs/binfmt_elf.c?v=3.18#L341

Chris

--
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project