Subject: Re: [PATCH 00/11] KVM: x86: track guest page access
To: Paolo Bonzini <pbonzini@redhat.com>
References: <1448907973-36066-1-git-send-email-guangrong.xiao@linux.intel.com>
 <565D73BA.8020002@redhat.com>
Cc: gleb@kernel.org, mtosatti@redhat.com, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, Andrea Arcangeli <aarcange@redhat.com>
From: Xiao Guangrong <guangrong.xiao@linux.intel.com>
Message-ID: <565DD218.4080902@linux.intel.com>
Date: Wed, 2 Dec 2015 01:00:08 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.3.0
MIME-Version: 1.0
In-Reply-To: <565D73BA.8020002@redhat.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 3483
Lines: 78


On 12/01/2015 06:17 PM, Paolo Bonzini wrote:
>
>
> On 30/11/2015 19:26, Xiao Guangrong wrote:
>> This patchset introduces the feature which allows us to track page
>> access in guest. Currently, only write access tracking is implemented
>> in this version.
>>
>> Four APIs are introduces:
>> - kvm_page_track_add_page(kvm, gfn, mode), single guest page @gfn is
>>    added into the track pool of the guest instance represented by @kvm,
>>    @mode specifies which kind of access on the @gfn is tracked
>>
>> - kvm_page_track_remove_page(kvm, gfn, mode), is the opposed operation
>>    of kvm_page_track_add_page() which removes @gfn from the tracking pool.
>>    gfn is no tracked after its last user is gone
>>
>> - kvm_page_track_register_notifier(kvm, n), register a notifier so that
>>    the event triggered by page tracking will be received, at that time,
>>    the callback of n->track_write() will be called
>>
>> - kvm_page_track_unregister_notifier(kvm, n), does the opposed operation
>>    of kvm_page_track_register_notifier(), which unlinks the notifier and
>>    stops receiving the tracked event
>>
>> The first user of page track is non-leaf shadow page tables as they are
>> always write protected. It also gains performance improvement because
>> page track speeds up page fault handler for the tracked pages. The
>> performance result of kernel building is as followings:
>>
>>     before           after
>> real 461.63       real 455.48
>> user 4529.55      user 4557.88
>> sys 1995.39       sys 1922.57
>
> For KVM-GT, as far as I know Andrea Arcangeli is working on extending
> userfaultfd to tracking write faults only.  Perhaps KVM-GT can do
> something similar, where KVM gets the write tracking functionality for
> free through the MMU notifiers.  Any thoughts on this?

Userfaultfd is excellent and has the ability to notify write event indeed,
however, it is not suitable for the use case of shadow page.

For the performance, shadow GPU is performance critical and requires
frequently being switched, it is not good to handle it in userspace. And
windows guest has many GPU tables and updates it frequently, that means,
we need to write protect huge number of pages which are single page based,
I am afraid userfaultfd can not handle this case efficiently.

For the functionality, userfaultfd can not fill the need of shadow page
because:
- the page is keeping readonly, userfaultfd can not fix the fault and let
   the vcpu progress (write access causes writeable gup).

- the access need to be emulated, however, userfaultfd/kernel does not have
   the ability to emulate the access as the access is trigged by guest, the
   instruction info is stored in VMCS so that only KVM can emulate it.

- shadow page needs to be notified after the emulation is finished as it
   should know the new data written to the page to update its page hierarchy.
   (some hardwares lack the 'retry' ability so the shadow page table need to
    reflect the table in guest at any time).

>
> Applying your technique to non-leaf shadow pages actually makes this
> series quite interesting. :)  Shadow paging is still in use for nested
> EPT, so it's always a good idea to speed it up.

Yes. Very glad to see you like it. :)


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/