Subject: Re: [PATCH RFC 0/2] KVM: s390: avoid having to enable vm.alloc_pgste
To: Martin Schwidefsky <schwidefsky@de.ibm.com>,
        Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: David Hildenbrand <david@redhat.com>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, Thomas Huth <thuth@redhat.com>
References: <20170529163202.13077-1-david@redhat.com>
 <20170601124651.3e7969ab@mschwideX1> <20170602070210.GA4221@osiris>
 <20170602114647.35e6d30f@mschwideX1>
From: Christian Borntraeger <borntraeger@de.ibm.com>
Date: Fri, 2 Jun 2017 12:19:19 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.1.0
MIME-Version: 1.0
In-Reply-To: <20170602114647.35e6d30f@mschwideX1>
Content-Type: text/plain; charset=utf-8
Content-Language: en-IE
Content-Transfer-Encoding: 7bit
Message-Id: <2d45c9d5-2108-7408-e7cd-44b9f6d03b0f@de.ibm.com>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2818
Lines: 73

On 06/02/2017 11:46 AM, Martin Schwidefsky wrote:
> On Fri, 2 Jun 2017 09:02:10 +0200
> Heiko Carstens <heiko.carstens@de.ibm.com> wrote:
> 
>> On Thu, Jun 01, 2017 at 12:46:51PM +0200, Martin Schwidefsky wrote:
>>>> Unfortunately, converting all page tables to 4k pgste page tables is
>>>> not possible without provoking various race conditions.  
>>>
>>> That is one approach we tried and was found to be buggy. The point is that
>>> you are not allowed to reallocate a page table while a VMA exists that is
>>> in the address range of that page table.
>>>
>>> Another approach we tried is to use an ELF flag on the qemu executable.
>>> That does not work either because fs/exec.c allocates and populates the
>>> new mm struct for the argument pages before fs/binfmt_elf.c comes into
>>> play.  
>>
>> How about if you would fail the system call within arch_check_elf() if you
>> detect that the binary requires pgstes (as indicated by elf flags) and then
>> restart the system call?
>>
>> That is: arch_check_elf() e.g. would set a thread flag that future mm's
>> should be allocated with pgstes. Then do_execve() would cleanup everything
>> and return to entry.S. Upon return to userspace we detect this condition
>> and simply restart the system call, similar to signals vs -ERESTARTSYS.
>>
>> That would make do_execve() cleanup everything and upon reentering it would
>> allocate an mm with the pgste flag set.
>>
>> Maybe this is a bit over-simplified, but might work.
> 
> This is not over-simplified at all, that does work:


Nice. Next question is how to integrate that elf flag into the qemu
build environment.

> --
> diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> index 69a77eecaec1..7bd182676ddd 100644
> --- a/arch/s390/Kconfig
> +++ b/arch/s390/Kconfig
> @@ -64,6 +64,7 @@ config ARCH_SUPPORTS_UPROBES
> 
>  config S390
>  	def_bool y
> +	select ARCH_BINFMT_ELF_STATE
>  	select ARCH_HAS_DEVMEM_IS_ALLOWED
>  	select ARCH_HAS_ELF_RANDOMIZE
>  	select ARCH_HAS_GCOV_PROFILE_ALL
> diff --git a/arch/s390/include/asm/elf.h b/arch/s390/include/asm/elf.h
> index e8f623041769..79911231f9e6 100644
> --- a/arch/s390/include/asm/elf.h
> +++ b/arch/s390/include/asm/elf.h
> @@ -151,6 +151,28 @@ extern unsigned int vdso_enabled;
>  	 && (x)->e_ident[EI_CLASS] == ELF_CLASS)
>  #define compat_start_thread	start_thread31
> 
> +struct arch_elf_state {
> +};
> +
> +#define INIT_ARCH_ELF_STATE { }
> +
> +#define arch_elf_pt_proc(ehdr, phdr, elf, interp, state) (0)
> +#define arch_check_elf(ehdr, interp, interp_ehdr, state)	\
> +({								\
> +	struct elf64_hdr *hdr = (void*) ehdr;			\
> +	int _rc = 0;						\
> +	if (hdr->e_ident[EI_CLASS] == ELFCLASS64 &&		\
> +	    (hdr->e_flags & 0x00000002) &&			\

shall we define some EF_S390_PGSTE flags in elf.h for this number?
What is 0x00000001 by the way?