Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210914200639.3305617-1-pgonda@google.com> <YUEVQDEvLbdJF+sj@google.com>
In-Reply-To: <YUEVQDEvLbdJF+sj@google.com>
From:   Peter Gonda <pgonda@google.com>
Date:   Tue, 14 Sep 2021 16:39:38 -0600
Message-ID: <CAMkAt6rSsKuzE__pAodiJR9wFU-B3942+kdkQG-3M+jxhVco2w@mail.gmail.com>
Subject: Re: [PATCH] KVM: SEV: Acquire vcpu mutex when updating VMSA
To:     Sean Christopherson <seanjc@google.com>
Cc:     kvm list <kvm@vger.kernel.org>, Marc Orr <marcorr@google.com>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Brijesh Singh <brijesh.singh@amd.com>, stable@vger.kernel.org,
        linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, Sep 14, 2021 at 3:34 PM Sean Christopherson <seanjc@google.com> wro=
te:
>
> On Tue, Sep 14, 2021, Peter Gonda wrote:
> > Adds mutex guard to the VMSA updating code. Also adds a check to skip a
> > vCPU if it has already been LAUNCH_UPDATE_VMSA'd which should allow
> > userspace to retry this ioctl until all the vCPUs can be successfully
> > LAUNCH_UPDATE_VMSA'd. Because this operation cannot be undone we cannot
> > unwind if one vCPU fails.
> >
> > Fixes: ad73109ae7ec ("KVM: SVM: Provide support to launch and run an SE=
V-ES guest")
> >
> > Signed-off-by: Peter Gonda <pgonda@google.com>
> > Cc: Marc Orr <marcorr@google.com>
> > Cc: Paolo Bonzini <pbonzini@redhat.com>
> > Cc: Sean Christopherson <seanjc@google.com>
> > Cc: Brijesh Singh <brijesh.singh@amd.com>
> > Cc: kvm@vger.kernel.org
> > Cc: stable@vger.kernel.org
> > Cc: linux-kernel@vger.kernel.org
> > ---
> >  arch/x86/kvm/svm/sev.c | 24 +++++++++++++++++++-----
> >  1 file changed, 19 insertions(+), 5 deletions(-)
> >
> > diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c
> > index 75e0b21ad07c..9a2ebd0328ca 100644
> > --- a/arch/x86/kvm/svm/sev.c
> > +++ b/arch/x86/kvm/svm/sev.c
> > @@ -598,22 +598,29 @@ static int sev_es_sync_vmsa(struct vcpu_svm *svm)
> >  static int sev_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd =
*argp)
> >  {
> >       struct kvm_sev_info *sev =3D &to_kvm_svm(kvm)->sev_info;
> > -     struct sev_data_launch_update_vmsa vmsa;
> > +     struct sev_data_launch_update_vmsa vmsa =3D {0};
> >       struct kvm_vcpu *vcpu;
> >       int i, ret;
> >
> >       if (!sev_es_guest(kvm))
> >               return -ENOTTY;
> >
> > -     vmsa.reserved =3D 0;
> > -
>
> Zeroing all of 'vmsa' is an unrelated chagne and belongs in a separate pa=
tch.  I
> would even go so far as to say it's unnecessary, even field of the struct=
 is
> explicitly written before it's consumed.

I'll remove this.

>
> >       kvm_for_each_vcpu(i, vcpu, kvm) {
> >               struct vcpu_svm *svm =3D to_svm(vcpu);
> >
> > +             ret =3D mutex_lock_killable(&vcpu->mutex);
> > +             if (ret)
> > +                     goto out_unlock;
>
> Rather than multiple unlock labels, move the guts of the loop to a wrappe=
r.
> As discussed off list, this really should be a vCPU-scoped ioctl, but tha=
t ship
> has sadly sailed :-(  We can at least imitate that by making the VM-scope=
d ioctl
> nothing but a wrapper.
>
> > +
> > +             /* Skip to the next vCPU if this one has already be updat=
ed. */
>
> s/be/been
>
> Uber nit, there may not be a next vCPU.  It'd be more slightly more accur=
ate to
> say something like "Do nothing if this vCPU has already been updated".
>
> > +             ret =3D sev_es_sync_vmsa(svm);
> > +             if (svm->vcpu.arch.guest_state_protected)
> > +                     goto unlock;
>
> This belongs in a separate patch, too.  It also introduces a bug (arguabl=
y two)
> in that it adds a duplicate call to sev_es_sync_vmsa().  The second bug i=
s that
> if sev_es_sync_vmsa() fails _and_ the vCPU is already protected, this wil=
l cause
> that failure to be squashed.

I'll move skipping logic to a seperate patch

>
> In the end, I think the least gross implementation will look something li=
ke this,
> implemented over two patches (one for the lock, one for the protected che=
ck).
>
> static int __sev_launch_update_vmsa(struct kvm *kvm, struct kvm_vcpu *vcp=
u,
>                                     int *error)
> {
>         struct sev_data_launch_update_vmsa vmsa;
>         struct vcpu_svm *svm =3D to_svm(vcpu);
>         int ret;
>
>         /*
>          * Do nothing if this vCPU has already been updated.  This is all=
owed
>          * to let userspace retry LAUNCH_UPDATE_VMSA if the command fails=
 on a
>          * later vCPU.
>          */
>         if (svm->vcpu.arch.guest_state_protected)
>                 return 0;
>
>         /* Perform some pre-encryption checks against the VMSA */
>         ret =3D sev_es_sync_vmsa(svm);
>         if (ret)
>                 return ret;
>
>         /*
>          * The LAUNCH_UPDATE_VMSA command will perform in-place
>          * encryption of the VMSA memory content (i.e it will write
>          * the same memory region with the guest's key), so invalidate
>          * it first.
>          */
>         clflush_cache_range(svm->vmsa, PAGE_SIZE);
>
>         vmsa.reserved =3D 0;
>         vmsa.handle =3D to_kvm_svm(kvm)->sev_info.handle;
>         vmsa.address =3D __sme_pa(svm->vmsa);
>         vmsa.len =3D PAGE_SIZE;
>         return sev_issue_cmd(kvm, SEV_CMD_LAUNCH_UPDATE_VMSA, &vmsa, erro=
r);
> }
>
> static int sev_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *ar=
gp)
> {
>         struct kvm_vcpu *vcpu;
>         int i, ret;
>
>         if (!sev_es_guest(kvm))
>                 return -ENOTTY;
>
>         kvm_for_each_vcpu(i, vcpu, kvm) {
>                 ret =3D mutex_lock_killable(&vcpu->mutex);
>                 if (ret)
>                         return ret;
>
>                 ret =3D __sev_launch_update_vmsa(kvm, vcpu, &argp->error)=
;
>
>                 mutex_unlock(&vcpu->mutex);
">                 if (ret)
>                         return ret;
>         }
>         return 0;
> }

That looks reasonable to me. I didn't know if changes headed for LTS
should be smaller so I avoided doing this refactor. From:
https://www.kernel.org/doc/html/v4.11/process/stable-kernel-rules.html#stab=
le-kernel-rules
seems to say less than 100 lines is ideal. I guess this could also be
a "theoretical race condition=E2=80=9D anyways so maybe not for LTS anyways=
.
Thoughts?