Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Eiichi Tsukata <eiichi.tsukata@nutanix.com>
To:     Paolo Bonzini <pbonzini@redhat.com>
CC:     "sean.j.christopherson@intel.com" <sean.j.christopherson@intel.com>,
        "vkuznets@redhat.com" <vkuznets@redhat.com>,
        "wanpengli@tencent.com" <wanpengli@tencent.com>,
        "jmattson@google.com" <jmattson@google.com>,
        "joro@8bytes.org" <joro@8bytes.org>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "mingo@redhat.com" <mingo@redhat.com>,
        "bp@alien8.de" <bp@alien8.de>, "x86@kernel.org" <x86@kernel.org>,
        "hpa@zytor.com" <hpa@zytor.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Felipe Franciosi <felipe@nutanix.com>,
        "rkrcmar@redhat.com" <rkrcmar@redhat.com>,
        Andrea Arcangeli <aarcange@redhat.com>
Subject: Re: [RFC PATCH] KVM: x86: Fix APIC page invalidation race
Thread-Topic: [RFC PATCH] KVM: x86: Fix APIC page invalidation race
Thread-Index: AQHWO7rKzyOSDfpNlUO9aScE6IXasajOtaaAgADGnYCAAJQaAIAAPg4A
Date:   Tue, 9 Jun 2020 13:36:13 +0000
Message-ID: <D9A42B2E-7BB1-4E70-98CD-40D295D9DB1A@nutanix.com>
References: <20200606042627.61070-1-eiichi.tsukata@nutanix.com>
 <0d9b3313-5d4c-9ef3-63e4-ba08ddbbe7a1@redhat.com>
 <7B9024C7-98D0-4940-91AE-40BCDE555C8F@nutanix.com>
 <6d2d2faf-116f-8c71-fda2-3fc052952dee@redhat.com>
In-Reply-To: <6d2d2faf-116f-8c71-fda2-3fc052952dee@redhat.com>
Accept-Language: ja-JP, en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <614DDFD98C7DE24685B5D905EF6E36D3@namprd02.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 7e919419-a64a-4c7c-13df-08d80c7a144a
X-MS-Exchange-CrossTenant-originalarrivaltime: 09 Jun 2020 13:36:13.7045
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: bb047546-786f-4de1-bd75-24e5b6f79043
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: jHkkKJdpfTl+RzGqrFTA6V1XIxZFb6IXl/qL90jAIrTZc/bE7pRgV27wqMfClFF92xz41d8MmqXw2StJ5CxqowHxCOZtUlsJgzKHvNA8mlo=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR02MB6820
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.216,18.0.687
 definitions=2020-06-09_06:2020-06-09,2020-06-09 signatures=0
X-Proofpoint-Spam-Reason: safe
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On Jun 9, 2020, at 18:54, Paolo Bonzini <pbonzini@redhat.com> wrote:
>=20
>=20
> No need to resend, the patch is good.  Here is my take on the commit mess=
age:

Thank you Paolo! Your commit message is much clearer.
I really appreciate your great job.

Best

Eiichi

>=20
>    Commit b1394e745b94 ("KVM: x86: fix APIC page invalidation") tried
>    to fix inappropriate APIC page invalidation by re-introducing arch
>    specific kvm_arch_mmu_notifier_invalidate_range() and calling it from
>    kvm_mmu_notifier_invalidate_range_start. However, the patch left a
>    possible race where the VMCS APIC address cache is updated *before*
>    it is unmapped:
>=20
>      (Invalidator) kvm_mmu_notifier_invalidate_range_start()
>      (Invalidator) kvm_make_all_cpus_request(kvm, KVM_REQ_APIC_PAGE_RELOA=
D)
>      (KVM VCPU) vcpu_enter_guest()
>      (KVM VCPU) kvm_vcpu_reload_apic_access_page()
>      (Invalidator) actually unmap page
>=20
>    Because of the above race, there can be a mismatch between the
>    host physical address stored in the APIC_ACCESS_PAGE VMCS field and
>    the host physical address stored in the EPT entry for the APIC GPA
>    (0xfee0000).  When this happens, the processor will not trap APIC
>    accesses, and will instead show the raw contents of the APIC-access pa=
ge.
>    Because Windows OS periodically checks for unexpected modifications to
>    the LAPIC register, this will show up as a BSOD crash with BugCheck
>    CRITICAL_STRUCTURE_CORRUPTION (109) we are currently seeing in
>    https://urldefense.proofpoint.com/v2/url?u=3Dhttps-3A__bugzilla.redhat=
.com_show-5Fbug.cgi-3Fid-3D1751017&d=3DDwIDaQ&c=3Ds883GpUCOChKOHiocYtGcg&r=
=3Ddy01Dr4Ly8mhvnUdx1pZhhT1bkq4h9z5aVWu3paoZtk&m=3Dyo6pYK4bnrvDLz_RwGyvwtJE=
Y6oWkg-9XGlbPBq1B2g&s=3D7tKkV92x4mCNLQ7miJIanMVqokYNJdjrcK4Rqqb_h7s&e=3D .
>=20
>    The root cause of the issue is that kvm_arch_mmu_notifier_invalidate_r=
ange()
>    cannot guarantee that no additional references are taken to the pages =
in
>    the range before kvm_mmu_notifier_invalidate_range_end().  Fortunately=
,
>    this case is supported by the MMU notifier API, as documented in
>    include/linux/mmu_notifier.h:
>=20
>             * If the subsystem
>             * can't guarantee that no additional references are taken to
>             * the pages in the range, it has to implement the
>             * invalidate_range() notifier to remove any references taken
>             * after invalidate_range_start().
>=20
>    The fix therefore is to reload the APIC-access page field in the VMCS
>    from kvm_mmu_notifier_invalidate_range() instead of ..._range_start().
>=20
> Thanks,
>=20
> Paolo