Message-ID: <40bf3fed-460e-b578-6795-08e564f412db@amd.com>
Date:   Tue, 21 Feb 2023 16:06:06 -0600
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.2.1
Subject: Re: [PATCH RFC v8 24/56] crypto: ccp: Handle the legacy TMR
 allocation when SNP is enabled
Content-Language: en-US
To:     Zhi Wang <zhi.wang.linux@gmail.com>
Cc:     Michael Roth <michael.roth@amd.com>, kvm@vger.kernel.org,
        linux-coco@lists.linux.dev, linux-mm@kvack.org,
        linux-crypto@vger.kernel.org, x86@kernel.org,
        linux-kernel@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com,
        jroedel@suse.de, thomas.lendacky@amd.com, hpa@zytor.com,
        ardb@kernel.org, pbonzini@redhat.com, seanjc@google.com,
        vkuznets@redhat.com, jmattson@google.com, luto@kernel.org,
        dave.hansen@linux.intel.com, slp@redhat.com, pgonda@google.com,
        peterz@infradead.org, srinivas.pandruvada@linux.intel.com,
        rientjes@google.com, dovmurik@linux.ibm.com, tobin@ibm.com,
        bp@alien8.de, vbabka@suse.cz, kirill@shutemov.name,
        ak@linux.intel.com, tony.luck@intel.com, marcorr@google.com,
        sathyanarayanan.kuppuswamy@linux.intel.com, alpergun@google.com,
        dgilbert@redhat.com, jarkko@kernel.org, nikunj.dadhania@amd.com,
        Brijesh Singh <brijesh.singh@amd.com>
References: <20230220183847.59159-1-michael.roth@amd.com>
 <20230220183847.59159-25-michael.roth@amd.com>
 <20230221112823.000063e4@gmail.com>
 <f70a2398-bd78-24aa-b0ae-9171465d50ff@amd.com>
 <20230221231514.00004b27@gmail.com>
From:   "Kalra, Ashish" <ashish.kalra@amd.com>
In-Reply-To: <20230221231514.00004b27@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RjNMZXk3QnR2Unp2VHR4TjR5eWkzUG1kaXBqYnFGaG8yNHRMZW9EbkEzSFdz?=
 =?utf-8?B?ME02UjdHYzVDbHJNN2JUSUxsZytaZDFXalpJcWVsT3VSS3JtbjU2c1BSVTBi?=
 =?utf-8?B?czZpODA1bTVuQWhINXhueWNsaWYzOHdCbW1SZlcrY0oxYnZPRkJWdkgxaVlZ?=
 =?utf-8?B?RnZBVlFlU2dCYUgxbzRnaXVkL09VUG9GWHcyZGNYbVlibnFvbUJWZjRxYVU3?=
 =?utf-8?B?Mm9jQ1d1cTRKZ3dIL1paQXA0cmdpV254N0NLRjZtQ2lVQmNrOTlFMUdXaFhn?=
 =?utf-8?B?UVBqMmVPU3BIT1lReXV0ZnJGWm41dndOdUhaemFsVzg5Wnc4NUdsRnkwZmNy?=
 =?utf-8?B?R3IxcnZFYjNHbzRWRG5VVjE2Q0dKbjdzY3cwT2RYVnNOOUxhWDJyUnplQXV5?=
 =?utf-8?B?aEswKzhPNjlzSnoxODN1b1I2MnFncHpmczhHTUFsZ2kyMG1jMXBJM21xdDhu?=
 =?utf-8?B?K3ZiU0NxWjdOa0xtL1Z1Uk5aT3ZjSDhYREtFa2Ywa1VHMk1aTFlmNTkvYXlD?=
 =?utf-8?B?eFdFdCtGYjRvUE11QXpmdE82U0VOS0FOa29FZVJZYTJFZXRuZjlZKy9tckpX?=
 =?utf-8?B?b2FOYWtGTkg1alVvRVNva3FFOUcyTTJobUZYRWpRNUZNaDdmd1YxVTVGeWMy?=
 =?utf-8?B?a1ZjNS9jdDc1TVNQaERvem4vdUZRYnQwZ0xiV0xhMkpyY2pjQW9xREFOU3py?=
 =?utf-8?B?RkdJWnV1QkdqVVVHNlk5YWN3ejNyZWxYR3pBWXloSDAyVXdBUWxmQWx5K2tn?=
 =?utf-8?B?R2xiaGZYY2Nod0luZUhtbWRpWW1iSTFNbWZrM2NpZDFwRXAycDNtdmFuREI1?=
 =?utf-8?B?cGd1U1h5dHQ5eDIwbHloVjF6NmNiNEJlcjFkSFJld3R1QW42K2ZxMHNaOC96?=
 =?utf-8?B?bTNyOWp5eWNEa09zbU9QNnN3aW80SUY2ZHQ0YThkS0lHMTllTSswZGJKR0hm?=
 =?utf-8?B?OXYwYmlFZ0FOMko2K2RCT3lNUkRDZnN1VkNSRWNCYXlibDJwTGliOTFpL0xi?=
 =?utf-8?B?SFI3MjhYc3V3Z1ZvN3dWcDdYRVMzVmxqYXBvZU9aQTJGT3ZJZXQ2cGRCZ0lP?=
 =?utf-8?B?b3BzdzJSNytMbVNnMDdkc3R1NjAzRXpJNUZkNDE0eGYvNGh6cnZnV0pSakFa?=
 =?utf-8?B?dEF5RThpVzdmU2FGREVsOU1jWVNBSVNIVUphaG1DMk1QVVZIbzk4WURMZ2Fq?=
 =?utf-8?B?OWoyZTJQVXVWL0ZNbDJ1Q3JzYUZUaHpXUGRJd21PUEI0YkI5MVRHN0xMWHQ1?=
 =?utf-8?B?L09jeW01MThQMDg4OWtrRm1WN1JidjJNcFh4WXBoVEtRVW1VMGYzWVhUMEYx?=
 =?utf-8?B?LzVFN0c3UlNoeUtvUkVXbmMxMmpWdkI5RXpXa0d4RExlSnVFT1ZrZTFxeHFn?=
 =?utf-8?B?U3RkbVB2c01OYXM3ZlgwYW5mM1pVNGdwM0FieERuZWlSeXRjTldyTUp5b3Fh?=
 =?utf-8?B?ZUw2WTBpSW51Ynp4U2JWdVBSYjVBY1Flb2k5WEYrYW1kSVNHUUpYY1dLRFZG?=
 =?utf-8?B?cm4wOVQ2bmJNTkNVbzBFR2VuK2EvbVBFK01la2FjeHBTVU42eFMzdXFjTUVm?=
 =?utf-8?B?bDhDVVlvRW1ZeW1ycVhEdmYyK01Gd2JTbEZTWEY5Z25ZaU43cEtPaE1mZ3Jz?=
 =?utf-8?B?YWIxcSsyVVhLSzdNOWVpMWRVVko2TEJPbjJPNEMyb3dPeUNiMlVUbERMUC9K?=
 =?utf-8?B?cFVyVktLSm5DSGpkWFVYZ0VnajlYMzVTZENTKzcrNGlYOHVreUxsNjRyanRq?=
 =?utf-8?B?SWRUcEJlQW5ROW9ZWWNXcFVFTmtnNXF4QnBTaVRNY3BORGJ6eG9ZdmQ3VTZj?=
 =?utf-8?B?aDhIcERNRU1BeEYwdkxIbSsyUEpXVmtmZmhaRElqM1IvZnFpc1p3V081YUJT?=
 =?utf-8?B?OU9YZG9qRVl4VmdWQjJTQTBpaUo4eE1OTzJOQm1uWGlSMlltL3BHYk53ZmRF?=
 =?utf-8?B?SnBqUit4U2I5d2wzdVgrNENZa2FVVTY5UE5aMkx1VzIyUk9YLzdxYklZKzVE?=
 =?utf-8?B?eHdGcExwYU5NTHhHd3BBU00waTFLYWVqV2Nlci9QZDVuSEliWVEwTG9Nc1hi?=
 =?utf-8?B?aHJpNUJTTXRXeHIzbjZLU29NaUROV2IwUk9XR1JXZDFjY2xPVWY5MzNyZ2dK?=
 =?utf-8?Q?Quete3FZ2SzNrBPim16DuRJcx?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 30c7769e-8caa-446d-69e8-08db1457d8db
X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2767.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2023 22:06:13.8212
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: svIQz+e22JmVKzEp6aVwbQklLvq5Q1883OOz2OjwC4fQRfqz5l07O6xVoj4ynVIL2qf5yFluLVAfe7QdcGvJcg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB7082
Precedence: bulk
List-ID: <linux-crypto.vger.kernel.org>
X-Mailing-List: linux-crypto@vger.kernel.org

On 2/21/2023 3:15 PM, Zhi Wang wrote:
> On Tue, 21 Feb 2023 09:31:01 -0600
> "Kalra, Ashish" <ashish.kalra@amd.com> wrote:
> 
>>>> +static int snp_reclaim_pages(unsigned long paddr, unsigned int npages, bool locked)
>>>> +{
>>>> +	/* Cbit maybe set in the paddr */
>>>
>>> This is confusing.
>>>
>>> I suppose C-bit is treated as a attribute of PTE in the kernel not part of the
>>> PA. It means only a PTE might carry a C-bit.
>>>
>>
>> snp_reclaim_pages() is also called for reclaiming guest memory, in which
>> case the (guest) paddr will have the C-bit set. Hence this C-bit
>> handling is done within snp_reclaim_pages() so that the callers don't
>> need to handle it explicitly.
> 
> Thanks for the explanation.
> 
> Do you mean it will be used like that in the later patch? Sorry if it is in the
> later patch as I was making progress slowly. It is quite a big patch set.
>

Yes, these are callers in later patches, like the following code path in 
patch 25:

static int unmap_firmware_writeable(u64 *paddr, u32 len, bool guest, 
struct snp_host_map *map)
{
         unsigned int npages = PAGE_ALIGN(len) >> PAGE_SHIFT;
	...
         /* If paddr points to a guest memory then restore the page 
state to hypervisor. */
         if (guest) {
                 if (snp_reclaim_pages(*paddr, npages, true))
                         return -EFAULT;

                 goto done;
         }

       	...
	...

Or, the following as part of patch 52:

int snp_guest_dbg_decrypt_page(u64 gctx_pfn, u64 src_pfn, u64 dst_pfn, 
int *error)
{
	...
         data.gctx_paddr = sme_me_mask | (gctx_pfn << PAGE_SHIFT);
         data.src_addr = sme_me_mask | (src_pfn << PAGE_SHIFT);
         data.dst_addr = sme_me_mask | (dst_pfn << PAGE_SHIFT);

         /* The destination page must be in the firmware state. */
         if (rmp_mark_pages_firmware(data.dst_addr, 1, false))
                 return -EIO;

         ret = sev_do_cmd(SEV_CMD_SNP_DBG_DECRYPT, &data, error);

         /* Restore the page state */
         if (snp_reclaim_pages(data.dst_addr, 1, false))
	...
	...

Thanks,
Ashish

> At least, I don't see that kind of usage in the current patch. Feel free to
> correct me if I am wrong.
> 
> The call chains:
> 
> __snp_free_firmware_page()
>      snp_reclaim_pages();
> 
> As __snp_free_firmware_page() takes struct page*, all the follwing coversion
> from it would not carry C-bit.
> 
> __snp_alloc_firmware_pages()
>    rmp_mark_pages_firmware()
>      snp_reclaim_pages()
> 
> As __snp_alloc_firmware_page() allocates page with struct page*, the same
> conclusion as above.
> 
>>
>>
>>> The paddr is from __pa(page_address()). It is not extracted from a PTE. Thus, the
>>> return from them should never have a C-bit.
>>>
>>> BTW: Wouldn't it be better to have pfn as input param instead of paddr?
>>>
>>> The caller has struct page, calling snp_reclaim_pages(page_to_pfn(page), xxxxx)
>>> would be much clearer than the current conversion:
>>> page_address() (struct page is converted to VA), __pa() (VA is converted to PA)
>>> in the caller and then PA is converted to pfn here.
>>>
>>>> +	unsigned long pfn = __sme_clr(paddr) >> PAGE_SHIFT;
>>>> +	int ret, err, i, n = 0;
>>>> +
>>>
>>> should be unsigned int i, n; as the input param npage is unsigned int.
>>>
>>>> +	if (!pfn_valid(pfn)) {
>>>> +		pr_err("%s: Invalid PFN %lx\n", __func__, pfn);
>>>> +		return 0;
>>>> +	}
>>>> +
>>>> +	for (i = 0; i < npages; i++, pfn++, n++) {
>>>> +		paddr = pfn << PAGE_SHIFT;
>>>> +
>>>> +		if (locked)
>>>> +			ret = __sev_do_cmd_locked(SEV_CMD_SNP_PAGE_RECLAIM, &paddr, &err);
>>>> +		else
>>>> +			ret = sev_do_cmd(SEV_CMD_SNP_PAGE_RECLAIM, &paddr, &err);
>>>> +
>>>> +		if (ret)
>>>> +			goto cleanup;
>>>> +
>>>> +		ret = rmp_make_shared(pfn, PG_LEVEL_4K);
>>>> +		if (ret)
>>>> +			goto cleanup;
>>>> +	}
>>>> +
>>>> +	return 0;
>>>> +
>>>> +cleanup:
>>>> +	/*
>>>> +	 * If failed to reclaim the page then page is no longer safe to
>>>> +	 * be release back to the system, leak it.
>>>> +	 */
>>>> +	snp_mark_pages_offline(pfn, npages - n);
>>>> +	return ret;
>>>> +}
>>>> +
>>>> +static int rmp_mark_pages_firmware(unsigned long paddr, unsigned int npages, bool locked)
>>>
>>> The same comment as above. Better take pfn or page instead of paddr with
>>> redundant conversions.
>>>
>>
>> Again, the paddr can point to guest memory so it can have C-bit set.
>>
>> Thanks,
>> Ashish
>>
>>>> +{
>>>> +	/* Cbit maybe set in the paddr */
>>>> +	unsigned long pfn = __sme_clr(paddr) >> PAGE_SHIFT;
>>>> +	int rc, n = 0, i;
>>>> +
>>>> +	for (i = 0; i < npages; i++, n++, pfn++) {
>>>> +		rc = rmp_make_private(pfn, 0, PG_LEVEL_4K, 0, true);
>>>> +		if (rc)
>>>> +			goto cleanup;
>>>> +	}
>>>> +
>>>> +	return 0;
>>>> +
>>>> +cleanup:
>>>> +	/*
>>>> +	 * Try unrolling the firmware state changes by
>>>> +	 * reclaiming the pages which were already changed to the
>>>> +	 * firmware state.
>>>> +	 */
>>>> +	snp_reclaim_pages(paddr, n, locked);
>>>> +
>>>> +	return rc;
>>>> +}
>>>> +
>