Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Mon, 13 Apr 2020 09:47:14 -0600
From:   Alex Williamson <alex.williamson@redhat.com>
To:     Dave Hansen <dave.hansen@intel.com>
Cc:     Alexander Duyck <alexander.h.duyck@linux.intel.com>,
        Mel Gorman <mgorman@techsingularity.net>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org,
        Andrea Arcangeli <aarcange@redhat.com>,
        Dan Williams <dan.j.williams@intel.com>,
        David Hildenbrand <david@redhat.com>,
        Michal Hocko <mhocko@kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        "Raj, Ashok" <ashok.raj@intel.com>
Subject: Re: [RFC PATCH 0/4] mm: Add PG_zero support
Message-ID: <20200413094714.432b5841@w520.home>
In-Reply-To: <7a064e81-6bc1-b3e7-5f82-292ffa392058@intel.com>
References: <20200412090728.GA19572@open-light-1.localdomain>
        <f9a3be0b-fe8d-ca25-f0df-4b9fd1f0fed5@intel.com>
        <20200413084915.1bae0007@w520.home>
        <7a064e81-6bc1-b3e7-5f82-292ffa392058@intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, 13 Apr 2020 08:14:32 -0700
Dave Hansen <dave.hansen@intel.com> wrote:

> On 4/13/20 7:49 AM, Alex Williamson wrote:
> >> VFIO's unconditional page pinning is the real problem here IMNHO.  They
> >> don't *really* need to pin the memory.  We just don't have good
> >> paravirtualized IOMMU support or want to pay the runtime cost for
> >> pin/unpin operations.  You *could* totally have speedy VM startup if
> >> only the pages being accessed or having DMA performed to them were
> >> allocated.  But, the hacks that are in place mean that everything must
> >> be pinned.  
> > Maybe in an SEV or Secure Boot environment we can assume the VM guest
> > OS uses the IOMMU exclusively for DMA, but otherwise the IOMMU is
> > optional (at least for x86, other archs do require IOMMU support
> > afaik).  Therefore, how would we know which pages to pin when there are
> > only limited configs where we might be able to lean on the vIOMMU to
> > this extent?  Thanks,  
> 
> You can delay pinning until the device is actually used.  That should be
> late enough for the host to figure out whether a paravirtualized IOMMU
> is in place.

So the guest enables the bus master bit in the command register and at
that point we'd stall the VM for an indeterminate length of time while
we potentially pin all memory, and hope that both the user and the host
has the resources to account and allocate that memory, otherwise the
VM suddenly crashes?  All of this potentially taking place in the
pre-boot environment to support option ROMs as well.  A delay starting
the VM seems a lot more predictable.  Thanks,

Alex