Received-SPF: pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <0716365f-3572-638b-e841-fcce7d30571a@amd.com>
Date:   Mon, 19 Sep 2022 16:38:02 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: [PATCH Part2 v6 37/49] KVM: SVM: Add support to handle MSR based
 Page State Change VMGEXIT
Content-Language: en-US
To:     Alper Gun <alpergun@google.com>, Peter Gonda <pgonda@google.com>
Cc:     Ashish Kalra <Ashish.Kalra@amd.com>,
        the arch/x86 maintainers <x86@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        kvm list <kvm@vger.kernel.org>, linux-coco@lists.linux.dev,
        Linux Memory Management List <linux-mm@kvack.org>,
        Linux Crypto Mailing List <linux-crypto@vger.kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Joerg Roedel <jroedel@suse.de>,
        "H. Peter Anvin" <hpa@zytor.com>, Ard Biesheuvel <ardb@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <seanjc@google.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Jim Mattson <jmattson@google.com>,
        Andy Lutomirski <luto@kernel.org>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Sergio Lopez <slp@redhat.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>,
        David Rientjes <rientjes@google.com>,
        Dov Murik <dovmurik@linux.ibm.com>,
        Tobin Feldman-Fitzthum <tobin@ibm.com>,
        Borislav Petkov <bp@alien8.de>,
        Michael Roth <michael.roth@amd.com>,
        Vlastimil Babka <vbabka@suse.cz>,
        "Kirill A . Shutemov" <kirill@shutemov.name>,
        Andi Kleen <ak@linux.intel.com>,
        Tony Luck <tony.luck@intel.com>, Marc Orr <marcorr@google.com>,
        Sathyanarayanan Kuppuswamy 
        <sathyanarayanan.kuppuswamy@linux.intel.com>,
        "Dr. David Alan Gilbert" <dgilbert@redhat.com>, jarkko@kernel.org
References: <cover.1655761627.git.ashish.kalra@amd.com>
 <78e30b5a25c926fcfdcaafea3d484f1bb25f20b9.1655761627.git.ashish.kalra@amd.com>
 <CAMkAt6rrGJ5DYTAJKFUTagN9i_opS8u5HPw5c_8NoyEjK7rYzA@mail.gmail.com>
 <CABpDEum157s5+yQvikjwQRaOcxau27NkMzX9eCs9=HFOW5FYnA@mail.gmail.com>
From:   Tom Lendacky <thomas.lendacky@amd.com>
In-Reply-To: <CABpDEum157s5+yQvikjwQRaOcxau27NkMzX9eCs9=HFOW5FYnA@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?d1MvTlhMVEtCWG1yMmRPc3VVQ3lTR3pjZlpmdDB0WXRhUjN4MmgxZDIyTXBU?=
 =?utf-8?B?QWxJVDhOUk9YMWdVcW8zbWthTWZ4UnBFY2YvNXlWZDRSMGRKa1RkRXlxSFow?=
 =?utf-8?B?czllZHN4VVJnb0dvby94VEQvbjVYUkM3NWRhUW9DZTNUNm5GTWpCN1FIL21P?=
 =?utf-8?B?aWJKYlRmSWgrc0RjaGsreXpmRG1VM2IvQkErMDVFYUFJam5BTEhjVGQ4UHly?=
 =?utf-8?B?Qkt5ZkNVS0dPVXZ0ZlJqN0dHdnliTXgrNldnN2RoNW5UOUhFZUJhNUFMTXlj?=
 =?utf-8?B?OUYvanN4SjVKOFIybmF1VWRZMTVwczdHbGhYWjZKRTJ5bWhaYmZURVdzdS9K?=
 =?utf-8?B?cFlSRmhid1JVa3RGTGhRSDA2UEQzZ2RNemorcHdFQWwzaUVPNERhOWsydEJ3?=
 =?utf-8?B?UFNTdGNOeXV4bXIwUnlqZEJySGlCbERSL3k1eDR3ZWJhUVp5T1p5MWhud3Fy?=
 =?utf-8?B?bVFRMVlCcXFrQmpBUHEvVm5XTURaWkNQdmswTjhRWDZ5YUJiek1VQjZ4OHdi?=
 =?utf-8?B?L3Z6eUt2cWVQOUdPWURxYXlEZlh6VFB4NjhZSVlmTTNYYzRjN0FaN2dabEt5?=
 =?utf-8?B?czFBOEM1N2ZOeUtsbFgvc3MrcHhVRk8yd3hVT2FaSFRlYzVHL3k0NS9WK2V4?=
 =?utf-8?B?NGQ0OHdnd3U3cVFTdzRQanNuM0dBcmRZNmVNUklreGFGL1dIeG5QQlMxU2c3?=
 =?utf-8?B?bC9RWXNjd1hKbTE1TDk2MXQ5MkdWM1Jyd1ljRnB6VzRqTFl3ZG9XMEJOY2Zl?=
 =?utf-8?B?ZmtUb28rb2FKdkM5bzZJdDltbmhJazYyTm9HUmFWdXVmeHp3VWY5T0VTQ0RK?=
 =?utf-8?B?ZzU3Y3c5QXg0M0N5S0FVTS9kbm9QdFE0WXFOa0E0WnhIWHBmNWQ3ZEZJMUxB?=
 =?utf-8?B?bHhVSGJTSDRPaDdlL3UyQy8rQnBYbWZvcHZFNVF3cEE2ZEg0aG12bmtRZXJL?=
 =?utf-8?B?cWpOUXBLa2dab3I2Ky8reXFxNi9jajc1WnYvb2R3Rm9pSzhvcERlZE84ZUhm?=
 =?utf-8?B?NE9GNC9aY3M5cUh3UmorV3ZTQ2xZZlBDYnlvcVFWU3VLK3JkSmt1emE3Tm11?=
 =?utf-8?B?MkY5L1N4RVZmM3dZRyt4YytINXM4QzBlZTk5T0ZQMzJsNjJVUDVlL2RQalpQ?=
 =?utf-8?B?RXlTdDgrd1hJcHk3WUN0M0dWVmtWZzVMMDhLZE53VlZZdER0bHpzTHlpazBx?=
 =?utf-8?B?eVlRNmhUUk1yeHJyVHdvd2MrSUNHclVGTVpOWEgrQnhNR29YNGY4WmRGMUJ0?=
 =?utf-8?B?Uk0vZ3JCejE5aDNXeW1KbUcyUTBLeXBwTzJDYUtDbFRCVEVCTEU2NlBBeFlv?=
 =?utf-8?B?YmZKZXJXNmtiVnJKU3Z3RW4xS3lrYXVjTVBSbmdOMXlDa1ZxdURQZ1BmMUVH?=
 =?utf-8?B?dUFxNm1yVWFIN2hKd1AzK0ZnY3IvSXB6R2dnRUlHUkVFaEJKK2JSL051Vngv?=
 =?utf-8?B?cjhJQVFuR212RDJ1OXYyMjM3cTU2cXhVbFh1K1pBMHpCb3ZmaU5sMGdzT1N0?=
 =?utf-8?B?ZDRVRVRqMGExZDYyS1cyZ1pOTFplUzNQeU90ak9BbG9oRjIxT1NuL2xrRFVO?=
 =?utf-8?B?QVJDY0IyYXc5alM5VjNmOUtXRmVzeXM2cFNPWHdUampROStYMTR4L3QycDU2?=
 =?utf-8?B?RGtPSW9jRkJiWjlQQUhIT3k0d09SMHlwUlowNHExU3ZWQVFZTlJCclNDWTho?=
 =?utf-8?B?NDNjVmZEZmF2dHJBSEx1enZFc0xsbGlieVZreUlMa1U4MXN6dzZpSTNYRnE5?=
 =?utf-8?B?clZzR3RkZXh1clEvcC9PRXJlQ0s1bGN3WU5tVkxJMGhHMjZZdWtzWXNoZFFv?=
 =?utf-8?B?cnVIaFFKaDc0azhZeWUraDkrd0QvOHpyTWhuUUkrekxmMDNiQVppTTJ3eHMr?=
 =?utf-8?B?NWZOeWhnbmdvSUVqTVlOVXlXUHpqOE94RHAxRnlrNnY4U1N4UzQ4b0ZDb1A3?=
 =?utf-8?B?VEYxc29SYmpwVU5JZHBsV2ZGUkkya09MNVlyRjVzc0xyam5QKyt5SHFFL0VM?=
 =?utf-8?B?TTVuQ3ZLayt6N21rYVpHMEs3d01Nb3pXc1RJOEFZNVN2V2hleUNQb2lSQ2lw?=
 =?utf-8?B?Zi94bWxJTEswUUZwNnVsZWVXZ0dGSDdXa0crL21ST3ZlaE9zZnhEUFpsdDZt?=
 =?utf-8?Q?5k7RN5+5TKZoe9iF8o+wqYolY?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ddb351c6-0578-4cd8-17ab-08da9a873d73
X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Sep 2022 21:38:07.0917
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: fdSuZ8IP8gURyVrrMYXYFblRNp8yI3n/Z5vg+lq3Lj+i5sCnuatDJkvj5//Lvfhsr+75qArXsCfJ6BRWa6aRUA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6288
X-Spam-Status: No, score=-3.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_FILL_THIS_FORM_SHORT
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-crypto.vger.kernel.org>
X-Mailing-List: linux-crypto@vger.kernel.org

On 9/19/22 12:53, Alper Gun wrote:
> On Fri, Aug 19, 2022 at 9:54 AM Peter Gonda <pgonda@google.com> wrote:
>>
>>> +
>>> +static int __snp_handle_page_state_change(struct kvm_vcpu *vcpu, enum psc_op op, gpa_t gpa,
>>> +                                         int level)
>>> +{
>>> +       struct kvm_sev_info *sev = &to_kvm_svm(vcpu->kvm)->sev_info;
>>> +       struct kvm *kvm = vcpu->kvm;
>>> +       int rc, npt_level;
>>> +       kvm_pfn_t pfn;
>>> +       gpa_t gpa_end;
>>> +
>>> +       gpa_end = gpa + page_level_size(level);
>>> +
>>> +       while (gpa < gpa_end) {
>>> +               /*
>>> +                * If the gpa is not present in the NPT then build the NPT.
>>> +                */
>>> +               rc = snp_check_and_build_npt(vcpu, gpa, level);
>>> +               if (rc)
>>> +                       return -EINVAL;
>>> +
>>> +               if (op == SNP_PAGE_STATE_PRIVATE) {
>>> +                       hva_t hva;
>>> +
>>> +                       if (snp_gpa_to_hva(kvm, gpa, &hva))
>>> +                               return -EINVAL;
>>> +
>>> +                       /*
>>> +                        * Verify that the hva range is registered. This enforcement is
>>> +                        * required to avoid the cases where a page is marked private
>>> +                        * in the RMP table but never gets cleanup during the VM
>>> +                        * termination path.
>>> +                        */
>>> +                       mutex_lock(&kvm->lock);
>>> +                       rc = is_hva_registered(kvm, hva, page_level_size(level));
>>> +                       mutex_unlock(&kvm->lock);
>>> +                       if (!rc)
>>> +                               return -EINVAL;
>>> +
>>> +                       /*
>>> +                        * Mark the userspace range unmerable before adding the pages
>>> +                        * in the RMP table.
>>> +                        */
>>> +                       mmap_write_lock(kvm->mm);
>>> +                       rc = snp_mark_unmergable(kvm, hva, page_level_size(level));
>>> +                       mmap_write_unlock(kvm->mm);
>>> +                       if (rc)
>>> +                               return -EINVAL;
>>> +               }
>>> +
>>> +               write_lock(&kvm->mmu_lock);
>>> +
>>> +               rc = kvm_mmu_get_tdp_walk(vcpu, gpa, &pfn, &npt_level);
>>> +               if (!rc) {
>>> +                       /*
>>> +                        * This may happen if another vCPU unmapped the page
>>> +                        * before we acquire the lock. Retry the PSC.
>>> +                        */
>>> +                       write_unlock(&kvm->mmu_lock);
>>> +                       return 0;
>>> +               }
>>
>> I think we want to return -EAGAIN or similar if we want the caller to
>> retry, right? I think returning 0 here hides the error.
>>
> 
> The problem here is that the caller(linux guest kernel) doesn't retry
> if PSC fails. The current implementation in the guest kernel is that
> if a page state change request fails, it terminates the VM with
> GHCB_TERM_PSC reason.
> Returning 0 here is not a good option because it will fail the PSC
> silently and will probably cause a nested RMP fault later. Returning

Returning 0 here is ok because the PSC current index into the PSC 
structure will not be updated and the guest will then retry (see the loop 
in vmgexit_psc() in arch/x86/kernel/sev.c).

Thanks,
Tom

> an error also terminates the guest immediately with current guest
> implementation. I think the best approach here is adding a retry logic
> to this function. Retrying without returning an error should help it
> work because snp_check_and_build_npt will be called again and in the
> second attempt this should work.
> 
>>> +
>>> +               /*
>>> +                * Adjust the level so that we don't go higher than the backing
>>> +                * page level.
>>> +                */
>>> +               level = min_t(size_t, level, npt_level);
>>> +
>>> +               trace_kvm_snp_psc(vcpu->vcpu_id, pfn, gpa, op, level);
>>> +
>>> +               switch (op) {
>>> +               case SNP_PAGE_STATE_SHARED:
>>> +                       rc = snp_make_page_shared(kvm, gpa, pfn, level);
>>> +                       break;
>>> +               case SNP_PAGE_STATE_PRIVATE:
>>> +                       rc = rmp_make_private(pfn, gpa, level, sev->asid, false);
>>> +                       break;
>>> +               default:
>>> +                       rc = -EINVAL;
>>> +                       break;
>>> +               }
>>> +
>>> +               write_unlock(&kvm->mmu_lock);
>>> +
>>> +               if (rc) {
>>> +                       pr_err_ratelimited("Error op %d gpa %llx pfn %llx level %d rc %d\n",
>>> +                                          op, gpa, pfn, level, rc);
>>> +                       return rc;
>>> +               }
>>> +
>>> +               gpa = gpa + page_level_size(level);
>>> +       }
>>> +
>>> +       return 0;
>>> +}
>>> +