Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20200622145953.41931-1-pbonzini@redhat.com>
In-Reply-To: <20200622145953.41931-1-pbonzini@redhat.com>
From:   Wanpeng Li <kernellwp@gmail.com>
Date:   Tue, 23 Jun 2020 08:26:59 +0800
Message-ID: <CANRm+CyvtURKCMmpPvm-LPL6aqR5Qp2teL9pD7L8ufGWOM0LeQ@mail.gmail.com>
Subject: Re: [PATCH] KVM: LAPIC: ensure APIC map is up to date on concurrent
 update requests
To:     Paolo Bonzini <pbonzini@redhat.com>
Cc:     LKML <linux-kernel@vger.kernel.org>, kvm <kvm@vger.kernel.org>,
        Igor Mammedov <imammedo@redhat.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, 22 Jun 2020 at 23:01, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> The following race can cause lost map update events:
>
>          cpu1                            cpu2
>
>                                 apic_map_dirty = true
>   ------------------------------------------------------------
>                                 kvm_recalculate_apic_map:
>                                      pass check
>                                          mutex_lock(&kvm->arch.apic_map_lock);
>                                          if (!kvm->arch.apic_map_dirty)
>                                      and in process of updating map
>   -------------------------------------------------------------
>     other calls to
>        apic_map_dirty = true         might be too late for affected cpu
>   -------------------------------------------------------------
>                                      apic_map_dirty = false
>   -------------------------------------------------------------
>     kvm_recalculate_apic_map:
>     bail out on
>       if (!kvm->arch.apic_map_dirty)
>
> To fix it, record the beginning of an update of the APIC map in
> apic_map_dirty.  If another APIC map change switches apic_map_dirty
> back to DIRTY, kvm_recalculate_apic_map should not make it CLEAN and
> let the other caller go through the slow path.
>
> Reported-by: Igor Mammedov <imammedo@redhat.com>
> ---
>  arch/x86/include/asm/kvm_host.h |  2 +-
>  arch/x86/kvm/lapic.c            | 45 +++++++++++++++++++--------------
>  2 files changed, 27 insertions(+), 20 deletions(-)
>
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 1da5858501ca..d814032a81e7 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -943,7 +943,7 @@ struct kvm_arch {
>         atomic_t vapics_in_nmi_mode;
>         struct mutex apic_map_lock;
>         struct kvm_apic_map *apic_map;
> -       bool apic_map_dirty;
> +       atomic_t apic_map_dirty;
>
>         bool apic_access_page_done;
>         unsigned long apicv_inhibit_reasons;
> diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
> index 34a7e0533dad..ef98f2fd3bbd 100644
> --- a/arch/x86/kvm/lapic.c
> +++ b/arch/x86/kvm/lapic.c
> @@ -169,6 +169,18 @@ static void kvm_apic_map_free(struct rcu_head *rcu)
>         kvfree(map);
>  }
>
> +/*
> + * CLEAN -> DIRTY and UPDATE_IN_PROGRESS -> DIRTY changes happen without a lock.
> + *
> + * DIRTY -> UPDATE_IN_PROGRESS and UPDATE_IN_PROGRESS -> CLEAN happen with
> + * apic_map_lock_held.
> + */
> +enum {
> +       CLEAN,
> +       UPDATE_IN_PROGRESS,
> +       DIRTY
> +};

Great! Thanks for the fix.

    Wanpeng