Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Fri, 4 Dec 2020 09:30:02 -0800
From:   Sean Christopherson <seanjc@google.com>
To:     David Woodhouse <dwmw2@infradead.org>
Cc:     Ankur Arora <ankur.a.arora@oracle.com>,
        Joao Martins <joao.m.martins@oracle.com>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Radim =?utf-8?B?S3LEjW3DocWZ?= <rkrcmar@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        "H. Peter Anvin" <hpa@zytor.com>, x86@kernel.org,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Subject: Re: [PATCH RFC 03/39] KVM: x86/xen: register shared_info page
Message-ID: <X8pyGiVDuBGJmazJ@google.com>
References: <20190220201609.28290-1-joao.m.martins@oracle.com>
 <20190220201609.28290-4-joao.m.martins@oracle.com>
 <b647bed6c75f8743b8afea251a88f00a5feaee29.camel@infradead.org>
 <2d4df59d-f945-32dc-6999-a6f711e972ea@oracle.com>
 <896dc984-fa71-8f2f-d12b-458294f5f706@oracle.com>
 <58db65203b9464f6f225f4ef97c45af3c72cf068.camel@infradead.org>
 <6ea92fe2-4067-d0e0-b716-16d39a7a6065@oracle.com>
 <8c92b2f3a8e8829ec85d22091b2fe84794f12f78.camel@infradead.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <8c92b2f3a8e8829ec85d22091b2fe84794f12f78.camel@infradead.org>
Precedence: bulk

On Thu, Dec 03, 2020, David Woodhouse wrote:
> On Wed, 2020-12-02 at 12:32 -0800, Ankur Arora wrote:
> > > On IRC, Paolo told me that permanent pinning causes problems for memory
> > > hotplug, and pointed me at the trick we do with an MMU notifier and
> > > kvm_vcpu_reload_apic_access_page().
> > 
> > Okay that answers my question. Thanks for clearing that up.
> > 
> > Not sure of a good place to document this but it would be good to
> > have this written down somewhere. Maybe kvm_map_gfn()?
> 
> Trying not to get too distracted by polishing this part, so I can
> continue with making more things actually work. But I took a quick look
> at the reload_apic_access_page() thing.
> 
> AFAICT it works because the access is only from *within* the vCPU, in
> guest mode.
> 
> So all the notifier has to do is kick all CPUs, which happens when it
> calls kvm_make_all_cpus_request(). Thus we are guaranteed that all CPUs
> are *out* of guest mode by the time...
> 
>     ...er... maybe not by the time the notifier returns, because all 
>     we've done is *send* the IPI and we don't know the other CPUs have 
>     actually stopped running the guest yet? 
> 
>     Maybe there's some explanation of why the actual TLB shootdown 
>     truly *will* occur before the page goes away, and some ordering 
>     rules which mean our reschedule IPI will happen first? Something 
>     like that ideally would have been in a comment in in MMU notifier.

KVM_REQ_APIC_PAGE_RELOAD is tagged with KVM_REQUEST_WAIT, which means that
kvm_kick_many_cpus() and thus smp_call_function_many() will have @wait=true,
i.e. the sender will wait for the SMP function call to finish on the target CPUs.