Message-ID: <ec8f2ca8-08ca-7bc2-bb82-23482e036dc9@amd.com>
Date:   Thu, 2 Feb 2023 13:04:28 -0600
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.2.1
Subject: Re: [PATCH RFC v7 16/64] x86/sev: Add helper functions for RMPUPDATE
 and PSMASH instruction
Content-Language: en-US
To:     Alexander Graf <graf@amazon.com>,
        Michael Roth <michael.roth@amd.com>, kvm@vger.kernel.org
Cc:     linux-coco@lists.linux.dev, linux-mm@kvack.org,
        linux-crypto@vger.kernel.org, x86@kernel.org,
        linux-kernel@vger.kernel.org, tglx@linutronix.de, mingo@redhat.com,
        jroedel@suse.de, thomas.lendacky@amd.com, hpa@zytor.com,
        ardb@kernel.org, pbonzini@redhat.com, seanjc@google.com,
        vkuznets@redhat.com, wanpengli@tencent.com, jmattson@google.com,
        luto@kernel.org, dave.hansen@linux.intel.com, slp@redhat.com,
        pgonda@google.com, peterz@infradead.org,
        srinivas.pandruvada@linux.intel.com, rientjes@google.com,
        dovmurik@linux.ibm.com, tobin@ibm.com, bp@alien8.de,
        vbabka@suse.cz, kirill@shutemov.name, ak@linux.intel.com,
        tony.luck@intel.com, marcorr@google.com,
        sathyanarayanan.kuppuswamy@linux.intel.com, alpergun@google.com,
        dgilbert@redhat.com, jarkko@kernel.org, harald@profian.com,
        Brijesh Singh <brijesh.singh@amd.com>,
        "Rapan, Sabin" <sabrapan@amazon.com>
References: <20221214194056.161492-1-michael.roth@amd.com>
 <20221214194056.161492-17-michael.roth@amd.com>
 <854a0caf-6940-8381-1e20-0ddb5ed94858@amazon.com>
 <59dd6291-6a99-ab7a-865f-b333a5f57bb9@amd.com>
 <7e73cb2d-5046-66cd-5a9b-8e2faa8458dc@amazon.com>
From:   "Kalra, Ashish" <ashish.kalra@amd.com>
In-Reply-To: <7e73cb2d-5046-66cd-5a9b-8e2faa8458dc@amazon.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NExCb2QzZnA1MmxWbzQwMHZuZkNJakNRNjZyRTlobnc0LzZ4N2xxS0I2a0JJ?=
 =?utf-8?B?N1VDOHI2Z2pGTGdJZnpTZDlGdGNCejkwOGRpVXVsVUc4K1ZuYTBVWVVHZ25B?=
 =?utf-8?B?U3N4eVp3eXJEbWZyNzB5blVsM2wxeUFJZFZOUXlBWTVudnc4VkFyME43Mkpx?=
 =?utf-8?B?ZUgzS2RVbXdrUW41eGptYzFsMmIwbVNNWFBvSFlpbkN3ZXB4Y1Y1cTFTeHRi?=
 =?utf-8?B?Y2E1eExGUDgxV0NrS1RzcDZLY1hGSjd6ZEF2UmlzUmZqaEJJVEhVaFMvQ0Ra?=
 =?utf-8?B?WmV2WjhSQjNoM05saVhpUkxHTElKTDBEd0ZibEtxeEhYMUpTYlFFdEQwZUNi?=
 =?utf-8?B?TDlKSjRCc3AyZDNzMEJ4a0g3ei9aL3hQYTlMWU5aRUU5RkpTQ1JJTnhXRlQr?=
 =?utf-8?B?SnVFQXgxZ3Bucmdvc1h4OCtFWTBLVkhCVXZVR2dMVzIvdHVoc2NwamlLVlJh?=
 =?utf-8?B?bDBNSVZmbE84UjgralQrWmRzNzdlRldLcXBocGRPY0x3TnIyS0YxbkRvVVo5?=
 =?utf-8?B?OWVXOXorUzM3YjBxcHl5YmtrU3FEQUkyeUJDSkNmVlo0bm5KbjBrWWFScVRp?=
 =?utf-8?B?cmRZMVcrRG9YOHZ2NHlOS2NJWjdUZE9NdEpWRk5xRDduSTZiTk0xbHU4VWl3?=
 =?utf-8?B?Ym84ZEI0Nk9Vd1dvOWRaKzF6a2xRTmUvM1loTlduaisvV0JMWTFaQUF6MGZ6?=
 =?utf-8?B?MXJCNkRvc3JrUDJZMFFYZWFDNWhKbzNMUW8xOVJoemNuN2ZQbnB5cXNiVEoy?=
 =?utf-8?B?MkFnQ0V5MFVDc0QzdE1OdEY5ZHArRFI1TnE4SGxXZE1JeStWWHhxZmdmcE4w?=
 =?utf-8?B?emJ0UVp0MUFBenp1SmFMZzFZZ3Y3QmxRZXIvaDVmeUY5SmVHcHZMKzJGaDZr?=
 =?utf-8?B?RjVOYjNNVmxxKzY4T3hSSHcwLzdZMkRQdjJEem56dHY0QUc4TDF3WklZNlF2?=
 =?utf-8?B?M3hpTU5aR1pWRjlHMFppZ3NFSUhOcWwvOWNhUllNeCt2RGdMTXZJbExTMUdT?=
 =?utf-8?B?V0Rwb1JpS3krQ09CMXFJZHR0U1d1c3Znc0haQ2tKQWZoNzBrdFRFMnozKzIv?=
 =?utf-8?B?TzlMVFNwVktnOXVWVmppbWFhOE9VUzk0ZVcrWTFDZTQ3dnpBVVlTT2FOcmYv?=
 =?utf-8?B?bTR0Ymt0bUxKcmo5NlhjUDVXcWVLUUc0OFRJemJnTGFWMStpNEVTSWptK2tB?=
 =?utf-8?B?VktsZVNwNXBOVjJuMm9yWDVhZEtPUnVNV2tFbmdnVFBVZUFReGI5YnVNemN1?=
 =?utf-8?B?MDVreWxIK2k2RHpObGMrT0ZpS3dGbW8xdGtVOWpJeTdEaHIweGEwVWkzMlhH?=
 =?utf-8?B?cWZHTnBreXdxSm40OUZMYnpIQmZRUGNSa0ZoQ3lURTJlZGZWelFmaE9PRDB0?=
 =?utf-8?B?cWVsL3lUR1UrZHJkMEZNRkIvclN1c0IrekJ1ZGpDNU9ZdXo2bnVQNjQ0WEtm?=
 =?utf-8?B?Z09QT0l4V3pLSm1jN3FCVnVpY1pnUHFmRHVtTjc0Zi8zT20yZ2ZFcHZPaldl?=
 =?utf-8?B?NWpuRWtmZ2xlQld2NVVEMzFPWmtJbloreC94L2ZDeTIyalpEM1pDNkZPUVBy?=
 =?utf-8?B?cnJPaXI0QTV6Yi9ITHRqekNEYXUwNnVyMGl2czhPVS9BQ3I2T0c2NjR6TVJl?=
 =?utf-8?B?d0pVSldGaFdKUGJpQy9pNi9aQnlZdnFtSVl5dXpnVVY4dlNqLzdSNmpmNkoy?=
 =?utf-8?B?b0I5WHB4WlRlVmRiSUZRUHQzWFBoZXBqUzR0OHM1Rzdha2VQZGppVU5CbW80?=
 =?utf-8?B?b3lpc1AxSEwvQnE5WEYra0hDSTIvRnB6Z2tsVnBrQUhXeGJHQ1ZONGVPdGhm?=
 =?utf-8?B?NlhjaDlFYk5SSG9sVlFFSG1JZjY3MWpVYUVJcVU1enUvYmhBdXlML0tEYy9p?=
 =?utf-8?B?SjViNlExSlVPWFk3dG1EazhPd1VBd2twTkJ4d0k5RXNvN3BZdVZ4WmxqMlJQ?=
 =?utf-8?B?UGlBc3JVSEk2Wi8wc1BBcytaQUFYTXltTE1pWUJRWFJTSi9xRVljeExTMEo4?=
 =?utf-8?B?eVZWdHU3dEU2SVA2Uk8xQXNuUFdzdEQ1bzNoZ3JtRy9HZ0xrUlRTNldjMUVo?=
 =?utf-8?B?dS9HRXVNdVVFVDhmVlNHWDIyRFN3aGtEWmZkY0x5c3o5NTAxbHlOQTh1TnBI?=
 =?utf-8?Q?/EX4fajR1FQ4SApo4fa5i4BWW?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1eed7ba5-7b51-4477-f404-08db055051f8
X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2767.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Feb 2023 19:04:33.6349
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /MYeyuyn+L+r/G0XshkmwcwBZYDVBJxMvandRV9xbcUXxBGgN1MgiWZWId3Rh/AbBOo/RDUftt9tGRec20ILfQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB8576
Precedence: bulk
List-ID: <linux-crypto.vger.kernel.org>
X-Mailing-List: linux-crypto@vger.kernel.org

On 2/1/2023 11:20 AM, Alexander Graf wrote:
> 
> On 01.02.23 18:14, Kalra, Ashish wrote:
>>
>> On 1/31/2023 3:26 PM, Alexander Graf wrote:
>>>
>>> On 14.12.22 20:40, Michael Roth wrote:
>>>> From: Brijesh Singh <brijesh.singh@amd.com>
>>>>
>>>> The RMPUPDATE instruction writes a new RMP entry in the RMP Table. The
>>>> hypervisor will use the instruction to add pages to the RMP table. See
>>>> APM3 for details on the instruction operations.
>>>>
>>>> The PSMASH instruction expands a 2MB RMP entry into a corresponding set
>>>> of contiguous 4KB-Page RMP entries. The hypervisor will use this
>>>> instruction to adjust the RMP entry without invalidating the previous
>>>> RMP entry.
>>>>
>>>> Add the following external interface API functions:
>>>>
>>>> int psmash(u64 pfn);
>>>> psmash is used to smash a 2MB aligned page into 4K
>>>> pages while preserving the Validated bit in the RMP.
>>>>
>>>> int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid,
>>>> bool immutable);
>>>> Used to assign a page to guest using the RMPUPDATE instruction.
>>>>
>>>> int rmp_make_shared(u64 pfn, enum pg_level level);
>>>> Used to transition a page to hypervisor/shared state using the
>>>> RMPUPDATE instruction.
>>>>
>>>> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
>>>> Signed-off-by: Brijesh Singh <brijesh.singh@amd.com>
>>>> [mdr: add RMPUPDATE retry logic for transient FAIL_OVERLAP errors]
>>>> Signed-off-by: Michael Roth <michael.roth@amd.com>
>>>> ---
>>>>   arch/x86/include/asm/sev.h | 24 ++++++++++
>>>>   arch/x86/kernel/sev.c      | 95 
>>>> ++++++++++++++++++++++++++++++++++++++
>>>>   2 files changed, 119 insertions(+)
>>>>
>>>> diff --git a/arch/x86/include/asm/sev.h b/arch/x86/include/asm/sev.h
>>>> index 8d3ce2ad27da..4eeedcaca593 100644
>>>> --- a/arch/x86/include/asm/sev.h
>>>> +++ b/arch/x86/include/asm/sev.h
>>>> @@ -80,10 +80,15 @@ extern bool handle_vc_boot_ghcb(struct pt_regs
>>>> *regs);
>>>>
>>>>   /* Software defined (when rFlags.CF = 1) */
>>>>   #define PVALIDATE_FAIL_NOUPDATE                255
>>>> +/* RMUPDATE detected 4K page and 2MB page overlap. */
>>>> +#define RMPUPDATE_FAIL_OVERLAP         7
>>>>
>>>>   /* RMP page size */
>>>>   #define RMP_PG_SIZE_4K                 0
>>>> +#define RMP_PG_SIZE_2M                 1
>>>>   #define RMP_TO_X86_PG_LEVEL(level)     (((level) == RMP_PG_SIZE_4K)
>>>> ? PG_LEVEL_4K : PG_LEVEL_2M)
>>>> +#define X86_TO_RMP_PG_LEVEL(level)     (((level) == PG_LEVEL_4K) ?
>>>> RMP_PG_SIZE_4K : RMP_PG_SIZE_2M)
>>>> +
>>>>   #define RMPADJUST_VMSA_PAGE_BIT                BIT(16)
>>>>
>>>>   /* SNP Guest message request */
>>>> @@ -133,6 +138,15 @@ struct snp_secrets_page_layout {
>>>>          u8 rsvd3[3840];
>>>>   } __packed;
>>>>
>>>> +struct rmp_state {
>>>> +       u64 gpa;
>>>> +       u8 assigned;
>>>> +       u8 pagesize;
>>>> +       u8 immutable;
>>>> +       u8 rsvd;
>>>> +       u32 asid;
>>>> +} __packed;
>>>> +
>>>>   #ifdef CONFIG_AMD_MEM_ENCRYPT
>>>>   extern struct static_key_false sev_es_enable_key;
>>>>   extern void __sev_es_ist_enter(struct pt_regs *regs);
>>>> @@ -198,6 +212,9 @@ bool snp_init(struct boot_params *bp);
>>>>   void __init __noreturn snp_abort(void);
>>>>   int snp_issue_guest_request(u64 exit_code, struct snp_req_data
>>>> *input, unsigned long *fw_err);
>>>>   int snp_lookup_rmpentry(u64 pfn, int *level);
>>>> +int psmash(u64 pfn);
>>>> +int rmp_make_private(u64 pfn, u64 gpa, enum pg_level level, int asid,
>>>> bool immutable);
>>>> +int rmp_make_shared(u64 pfn, enum pg_level level);
>>>>   #else
>>>>   static inline void sev_es_ist_enter(struct pt_regs *regs) { }
>>>>   static inline void sev_es_ist_exit(void) { }
>>>> @@ -223,6 +240,13 @@ static inline int snp_issue_guest_request(u64
>>>> exit_code, struct snp_req_data *in
>>>>          return -ENOTTY;
>>>>   }
>>>>   static inline int snp_lookup_rmpentry(u64 pfn, int *level) { return
>>>> 0; }
>>>> +static inline int psmash(u64 pfn) { return -ENXIO; }
>>>> +static inline int rmp_make_private(u64 pfn, u64 gpa, enum pg_level
>>>> level, int asid,
>>>> +                                  bool immutable)
>>>> +{
>>>> +       return -ENODEV;
>>>> +}
>>>> +static inline int rmp_make_shared(u64 pfn, enum pg_level level) {
>>>> return -ENODEV; }
>>>>   #endif
>>>>
>>>>   #endif
>>>> diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c
>>>> index 706675561f49..67035d34adad 100644
>>>> --- a/arch/x86/kernel/sev.c
>>>> +++ b/arch/x86/kernel/sev.c
>>>> @@ -2523,3 +2523,98 @@ int snp_lookup_rmpentry(u64 pfn, int *level)
>>>>          return !!rmpentry_assigned(e);
>>>>   }
>>>>   EXPORT_SYMBOL_GPL(snp_lookup_rmpentry);
>>>> +
>>>> +/*
>>>> + * psmash is used to smash a 2MB aligned page into 4K
>>>> + * pages while preserving the Validated bit in the RMP.
>>>> + */
>>>> +int psmash(u64 pfn)
>>>> +{
>>>> +       unsigned long paddr = pfn << PAGE_SHIFT;
>>>> +       int ret;
>>>> +
>>>> +       if (!pfn_valid(pfn))
>>>> +               return -EINVAL;
>>>
>>>
>>> We (and many other clouds) use a neat trick to reduce the number of
>>> struct pages Linux allocates for guest memory: In its simplest form, add
>>> mem= to the kernel cmdline and mmap() /dev/mem to access the reserved
>>> memory instead.
>>>
>>> This means that the system covers more RAM than Linux contains, which
>>> means pfn_valid() is no longer a good indication whether a page is
>>> indeed valid. KVM handles this case fine, but this code does not.
>>
>> Hmm...but then is also using max_pfn reliable ?
> 
> 
> I would expect it to not be reliable as it only looks at E820_TYPE_RAM, 
> yes. Do you rely on max_pfn anywhere?
>  

We use it to check if the RMP table is covering the whole system RAM, 
to get the max. addressable PFN, which should be fine.

Thanks,
Ashish