Subject: Re: [PATCH RFC 0/2] KVM: s390: avoid having to enable vm.alloc_pgste
To: Martin Schwidefsky <schwidefsky@de.ibm.com>
Cc: Heiko Carstens <heiko.carstens@de.ibm.com>,
        David Hildenbrand <david@redhat.com>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, Thomas Huth <thuth@redhat.com>
References: <20170529163202.13077-1-david@redhat.com>
 <20170601124651.3e7969ab@mschwideX1> <20170602070210.GA4221@osiris>
 <ab3ff23c-eb32-7e66-6c29-44738a3e6957@de.ibm.com>
 <20170602091638.5583cfb2@mschwideX1>
From: Christian Borntraeger <borntraeger@de.ibm.com>
Date: Fri, 2 Jun 2017 09:18:58 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.1.0
MIME-Version: 1.0
In-Reply-To: <20170602091638.5583cfb2@mschwideX1>
Content-Type: text/plain; charset=utf-8
Content-Language: en-IE
Content-Transfer-Encoding: 7bit
Message-Id: <787556bd-2c20-48e8-edd4-f4c5c5a5eaaf@de.ibm.com>
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2389
Lines: 48

On 06/02/2017 09:16 AM, Martin Schwidefsky wrote:
> On Fri, 2 Jun 2017 09:13:03 +0200
> Christian Borntraeger <borntraeger@de.ibm.com> wrote:
> 
>> On 06/02/2017 09:02 AM, Heiko Carstens wrote:
>>> On Thu, Jun 01, 2017 at 12:46:51PM +0200, Martin Schwidefsky wrote:  
>>>>> Unfortunately, converting all page tables to 4k pgste page tables is
>>>>> not possible without provoking various race conditions.  
>>>>
>>>> That is one approach we tried and was found to be buggy. The point is that
>>>> you are not allowed to reallocate a page table while a VMA exists that is
>>>> in the address range of that page table.
>>>>
>>>> Another approach we tried is to use an ELF flag on the qemu executable.
>>>> That does not work either because fs/exec.c allocates and populates the
>>>> new mm struct for the argument pages before fs/binfmt_elf.c comes into
>>>> play.  
>>>
>>> How about if you would fail the system call within arch_check_elf() if you
>>> detect that the binary requires pgstes (as indicated by elf flags) and then
>>> restart the system call?
>>>
>>> That is: arch_check_elf() e.g. would set a thread flag that future mm's
>>> should be allocated with pgstes. Then do_execve() would cleanup everything
>>> and return to entry.S. Upon return to userspace we detect this condition
>>> and simply restart the system call, similar to signals vs -ERESTARTSYS.
>>>
>>> That would make do_execve() cleanup everything and upon reentering it would
>>> allocate an mm with the pgste flag set.
>>>
>>> Maybe this is a bit over-simplified, but might work.
>>>
>>> At least I also don't like the next "hack", that is specifically designed
>>> to only work with how QEMU is currently implemented. It might break with
>>> future QEMU changes or the next user space implementation that drives the
>>> kvm interface, but is doing everything differently.
>>> Let's look for a "clean" solution that will always work. We had too many
>>> hacks for this problem and *all* of them were broken.  
>>
>>
>> The more I think about it, dropping 2k page tables and always allocate a full
>> page would simplify pgalloc. As far I can see this would also get rid of
>> the &mm->context.pgtable_lock.
> 
> And it would waste twice the amount of memory for page tables. NAK.

Yes and we spend the same amount of memory TODAY, because every distro on the
planet that uses KVM has sysctl.allocate_pgste set.