Subject: Re: [PATCH RFC 0/2] KVM: s390: avoid having to enable vm.alloc_pgste
To: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>,
        David Hildenbrand <david@redhat.com>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, Thomas Huth <thuth@redhat.com>
References: <20170529163202.13077-1-david@redhat.com>
 <20170601124651.3e7969ab@mschwideX1> <20170602070210.GA4221@osiris>
 <20170602114647.35e6d30f@mschwideX1>
 <2d45c9d5-2108-7408-e7cd-44b9f6d03b0f@de.ibm.com>
 <20170602125345.5ac9e12e@mschwideX1>
From: Christian Borntraeger <borntraeger@de.ibm.com>
Date: Fri, 2 Jun 2017 15:20:33 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.1.0
MIME-Version: 1.0
In-Reply-To: <20170602125345.5ac9e12e@mschwideX1>
Content-Type: text/plain; charset=utf-8
Content-Language: en-IE
Content-Transfer-Encoding: 7bit
Message-Id: <63b5c762-fbbb-b6dc-0f2d-6837dac5bb04@de.ibm.com>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3188
Lines: 85

On 06/02/2017 12:53 PM, Martin Schwidefsky wrote:
> On Fri, 2 Jun 2017 12:19:19 +0200
> Christian Borntraeger <borntraeger@de.ibm.com> wrote:
> 
>> On 06/02/2017 11:46 AM, Martin Schwidefsky wrote:
>>> On Fri, 2 Jun 2017 09:02:10 +0200
>>> Heiko Carstens <heiko.carstens@de.ibm.com> wrote:
>>>   
>>>> On Thu, Jun 01, 2017 at 12:46:51PM +0200, Martin Schwidefsky wrote:  
>>>>>> Unfortunately, converting all page tables to 4k pgste page tables is
>>>>>> not possible without provoking various race conditions.    
>>>>>
>>>>> That is one approach we tried and was found to be buggy. The point is that
>>>>> you are not allowed to reallocate a page table while a VMA exists that is
>>>>> in the address range of that page table.
>>>>>
>>>>> Another approach we tried is to use an ELF flag on the qemu executable.
>>>>> That does not work either because fs/exec.c allocates and populates the
>>>>> new mm struct for the argument pages before fs/binfmt_elf.c comes into
>>>>> play.    
>>>>
>>>> How about if you would fail the system call within arch_check_elf() if you
>>>> detect that the binary requires pgstes (as indicated by elf flags) and then
>>>> restart the system call?
>>>>
>>>> That is: arch_check_elf() e.g. would set a thread flag that future mm's
>>>> should be allocated with pgstes. Then do_execve() would cleanup everything
>>>> and return to entry.S. Upon return to userspace we detect this condition
>>>> and simply restart the system call, similar to signals vs -ERESTARTSYS.
>>>>
>>>> That would make do_execve() cleanup everything and upon reentering it would
>>>> allocate an mm with the pgste flag set.
>>>>
>>>> Maybe this is a bit over-simplified, but might work.  
>>>
>>> This is not over-simplified at all, that does work:  
>>
>>
>> Nice. Next question is how to integrate that elf flag into the qemu
>> build environment.
> 
> So far I use a small C program to set the flag:
> 
> #include <elf.h>
> #include <fcntl.h>
> #include <stdio.h>
> #include <stdlib.h>
> 
> #define ERREXIT(...)                    \
> do {                                    \
>         fprintf(stderr, __VA_ARGS__);   \
>         exit(-1);                       \
> } while (0)
> 
> int main(int argc, char *argv[])
> {
>         Elf64_Ehdr ehdr;
>         int fd, rc;
> 
>         if (argc != 2)
>                 ERREXIT("Usage: %s <elf-file>\n", argv[0]);
> 
>         fd = open(argv[1], O_RDWR);
>         if (fd < 0)
>                 ERREXIT("Unable to open file %s\n", argv[1]);
> 
>         if (pread(fd, &ehdr, sizeof(ehdr), 0) != sizeof(ehdr) ||
>             memcmp(&ehdr.e_ident, ELFMAG, SELFMAG) != 0 ||
>                 ehdr.e_ident[EI_CLASS] != ELFCLASS64 ||
>             ehdr.e_machine != EM_S390)
>                 ERREXIT("Invalid ELF file %s\n", argv[1]);
> 
>         ehdr.e_flags |= 0x00000002;
> 
>         if (pwrite(fd, &ehdr, sizeof(ehdr), 0) != sizeof(ehdr))
>                 ERREXIT("Write to of file %s failed\n", argv[1]);
> 
>         close(fd);
>         return 0;
> }
> 

Thanks for that. I assume this is mostly for testing and we want to have 
toolchain support for that. Otherwise things like  build-id (the sha variant) 
might be wrong, no?