Date: Fri, 2 Jun 2017 12:53:45 +0200
From: Martin Schwidefsky <schwidefsky@de.ibm.com>
To: Christian Borntraeger <borntraeger@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>,
        David Hildenbrand <david@redhat.com>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, Thomas Huth <thuth@redhat.com>
Subject: Re: [PATCH RFC 0/2] KVM: s390: avoid having to enable
 vm.alloc_pgste
In-Reply-To: <2d45c9d5-2108-7408-e7cd-44b9f6d03b0f@de.ibm.com>
References: <20170529163202.13077-1-david@redhat.com>
        <20170601124651.3e7969ab@mschwideX1>
        <20170602070210.GA4221@osiris>
        <20170602114647.35e6d30f@mschwideX1>
        <2d45c9d5-2108-7408-e7cd-44b9f6d03b0f@de.ibm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Message-Id: <20170602125345.5ac9e12e@mschwideX1>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 4321
Lines: 125

On Fri, 2 Jun 2017 12:19:19 +0200
Christian Borntraeger <borntraeger@de.ibm.com> wrote:

> On 06/02/2017 11:46 AM, Martin Schwidefsky wrote:
> > On Fri, 2 Jun 2017 09:02:10 +0200
> > Heiko Carstens <heiko.carstens@de.ibm.com> wrote:
> >   
> >> On Thu, Jun 01, 2017 at 12:46:51PM +0200, Martin Schwidefsky wrote:  
> >>>> Unfortunately, converting all page tables to 4k pgste page tables is
> >>>> not possible without provoking various race conditions.    
> >>>
> >>> That is one approach we tried and was found to be buggy. The point is that
> >>> you are not allowed to reallocate a page table while a VMA exists that is
> >>> in the address range of that page table.
> >>>
> >>> Another approach we tried is to use an ELF flag on the qemu executable.
> >>> That does not work either because fs/exec.c allocates and populates the
> >>> new mm struct for the argument pages before fs/binfmt_elf.c comes into
> >>> play.    
> >>
> >> How about if you would fail the system call within arch_check_elf() if you
> >> detect that the binary requires pgstes (as indicated by elf flags) and then
> >> restart the system call?
> >>
> >> That is: arch_check_elf() e.g. would set a thread flag that future mm's
> >> should be allocated with pgstes. Then do_execve() would cleanup everything
> >> and return to entry.S. Upon return to userspace we detect this condition
> >> and simply restart the system call, similar to signals vs -ERESTARTSYS.
> >>
> >> That would make do_execve() cleanup everything and upon reentering it would
> >> allocate an mm with the pgste flag set.
> >>
> >> Maybe this is a bit over-simplified, but might work.  
> > 
> > This is not over-simplified at all, that does work:  
> 
> 
> Nice. Next question is how to integrate that elf flag into the qemu
> build environment.

So far I use a small C program to set the flag:

#include <elf.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>

#define ERREXIT(...)                    \
do {                                    \
        fprintf(stderr, __VA_ARGS__);   \
        exit(-1);                       \
} while (0)

int main(int argc, char *argv[])
{
        Elf64_Ehdr ehdr;
        int fd, rc;

        if (argc != 2)
                ERREXIT("Usage: %s <elf-file>\n", argv[0]);

        fd = open(argv[1], O_RDWR);
        if (fd < 0)
                ERREXIT("Unable to open file %s\n", argv[1]);

        if (pread(fd, &ehdr, sizeof(ehdr), 0) != sizeof(ehdr) ||
            memcmp(&ehdr.e_ident, ELFMAG, SELFMAG) != 0 ||
                ehdr.e_ident[EI_CLASS] != ELFCLASS64 ||
            ehdr.e_machine != EM_S390)
                ERREXIT("Invalid ELF file %s\n", argv[1]);

        ehdr.e_flags |= 0x00000002;

        if (pwrite(fd, &ehdr, sizeof(ehdr), 0) != sizeof(ehdr))
                ERREXIT("Write to of file %s failed\n", argv[1]);

        close(fd);
        return 0;
}
 
> > --
> > diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
> > index 69a77eecaec1..7bd182676ddd 100644
> > --- a/arch/s390/Kconfig
> > +++ b/arch/s390/Kconfig
> > @@ -64,6 +64,7 @@ config ARCH_SUPPORTS_UPROBES
> > 
> >  config S390
> >  	def_bool y
> > +	select ARCH_BINFMT_ELF_STATE
> >  	select ARCH_HAS_DEVMEM_IS_ALLOWED
> >  	select ARCH_HAS_ELF_RANDOMIZE
> >  	select ARCH_HAS_GCOV_PROFILE_ALL
> > diff --git a/arch/s390/include/asm/elf.h b/arch/s390/include/asm/elf.h
> > index e8f623041769..79911231f9e6 100644
> > --- a/arch/s390/include/asm/elf.h
> > +++ b/arch/s390/include/asm/elf.h
> > @@ -151,6 +151,28 @@ extern unsigned int vdso_enabled;
> >  	 && (x)->e_ident[EI_CLASS] == ELF_CLASS)
> >  #define compat_start_thread	start_thread31
> > 
> > +struct arch_elf_state {
> > +};
> > +
> > +#define INIT_ARCH_ELF_STATE { }
> > +
> > +#define arch_elf_pt_proc(ehdr, phdr, elf, interp, state) (0)
> > +#define arch_check_elf(ehdr, interp, interp_ehdr, state)	\
> > +({								\
> > +	struct elf64_hdr *hdr = (void*) ehdr;			\
> > +	int _rc = 0;						\
> > +	if (hdr->e_ident[EI_CLASS] == ELFCLASS64 &&		\
> > +	    (hdr->e_flags & 0x00000002) &&			\  
> 
> shall we define some EF_S390_PGSTE flags in elf.h for this number?
> What is 0x00000001 by the way?

Yes, we have to give the bit a nice name, like EF_S390_REQUEST_PGSTE
0x00000001 is EF_S390_HIGH_GPRS

-- 
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.