Received-SPF: pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
Date:   Mon, 19 Dec 2022 09:00:26 -0600
From:   Michael Roth <michael.roth@amd.com>
To:     Borislav Petkov <bp@alien8.de>
CC:     Ashish Kalra <Ashish.Kalra@amd.com>, <x86@kernel.org>,
        <linux-kernel@vger.kernel.org>, <kvm@vger.kernel.org>,
        <linux-coco@lists.linux.dev>, <linux-mm@kvack.org>,
        <linux-crypto@vger.kernel.org>, <tglx@linutronix.de>,
        <mingo@redhat.com>, <jroedel@suse.de>, <thomas.lendacky@amd.com>,
        <hpa@zytor.com>, <ardb@kernel.org>, <pbonzini@redhat.com>,
        <seanjc@google.com>, <vkuznets@redhat.com>, <jmattson@google.com>,
        <luto@kernel.org>, <dave.hansen@linux.intel.com>, <slp@redhat.com>,
        <pgonda@google.com>, <peterz@infradead.org>,
        <srinivas.pandruvada@linux.intel.com>, <rientjes@google.com>,
        <dovmurik@linux.ibm.com>, <tobin@ibm.com>, <vbabka@suse.cz>,
        <kirill@shutemov.name>, <ak@linux.intel.com>,
        <tony.luck@intel.com>, <marcorr@google.com>,
        <sathyanarayanan.kuppuswamy@linux.intel.com>,
        <alpergun@google.com>, <dgilbert@redhat.com>, <jarkko@kernel.org>
Subject: Re: [PATCH Part2 v6 07/49] x86/sev: Invalid pages from direct map
 when adding it to RMP table
Message-ID: <20221219150026.bltiyk72pmdc2ic3@amd.com>
References: <cover.1655761627.git.ashish.kalra@amd.com>
 <243778c282cd55a554af9c11d2ecd3ff9ea6820f.1655761627.git.ashish.kalra@amd.com>
 <YuFvbm/Zck9Tr5pq@zn.tnic>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <YuFvbm/Zck9Tr5pq@zn.tnic>
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Dec 2022 15:00:44.8038
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1a3d93a4-984a-447d-486b-08dae1d1ebad
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource: BL02EPF00010209.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6344
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-crypto.vger.kernel.org>
X-Mailing-List: linux-crypto@vger.kernel.org

On Wed, Jul 27, 2022 at 07:01:34PM +0200, Borislav Petkov wrote:
> On Mon, Jun 20, 2022 at 11:03:07PM +0000, Ashish Kalra wrote:
> 
> > Subject: x86/sev: Invalid pages from direct map when adding it to RMP table
> 
> "...: Invalidate pages from the direct map when adding them to the RMP table"
> 
> > +static int restore_direct_map(u64 pfn, int npages)
> > +{
> > +	int i, ret = 0;
> > +
> > +	for (i = 0; i < npages; i++) {
> > +		ret = set_direct_map_default_noflush(pfn_to_page(pfn + i));
> 
> set_memory_p() ?

We implemented this approach for v7, but it causes a fairly significant
performance regression, particularly for the case for npages > 1 which
this change was meant to optimize.

I still need to dig in a big but I'm guessing it's related to flushing
behavior.

It would however be nice to have a set_direct_map_default_noflush()
variant that accepted a 'npages' argument, since it would be more
performant here and also would potentially allow for restoring the 2M
direct mapping in some cases. Will look into this more for v8.

-Mike

> 
> > +		if (ret)
> > +			goto cleanup;
> > +	}
> > +
> > +cleanup:
> > +	WARN(ret > 0, "Failed to restore direct map for pfn 0x%llx\n", pfn + i);
> 
> Warn for each pfn?!
> 
> That'll flood dmesg mightily.
> 
> > +	return ret;
> > +}
> > +
> > +static int invalid_direct_map(unsigned long pfn, int npages)
> > +{
> > +	int i, ret = 0;
> > +
> > +	for (i = 0; i < npages; i++) {
> > +		ret = set_direct_map_invalid_noflush(pfn_to_page(pfn + i));
> 
> As above, set_memory_np() doesn't work here instead of looping over each
> page?
> 
> > @@ -2462,11 +2494,38 @@ static int rmpupdate(u64 pfn, struct rmpupdate *val)
> >  	if (!cpu_feature_enabled(X86_FEATURE_SEV_SNP))
> >  		return -ENXIO;
> >  
> > +	level = RMP_TO_X86_PG_LEVEL(val->pagesize);
> > +	npages = page_level_size(level) / PAGE_SIZE;
> > +
> > +	/*
> > +	 * If page is getting assigned in the RMP table then unmap it from the
> > +	 * direct map.
> > +	 */
> > +	if (val->assigned) {
> > +		if (invalid_direct_map(pfn, npages)) {
> > +			pr_err("Failed to unmap pfn 0x%llx pages %d from direct_map\n",
> 
> "Failed to unmap %d pages at pfn 0x... from the direct map\n"
> 
> > +			       pfn, npages);
> > +			return -EFAULT;
> > +		}
> > +	}
> > +
> >  	/* Binutils version 2.36 supports the RMPUPDATE mnemonic. */
> >  	asm volatile(".byte 0xF2, 0x0F, 0x01, 0xFE"
> >  		     : "=a"(ret)
> >  		     : "a"(paddr), "c"((unsigned long)val)
> >  		     : "memory", "cc");
> > +
> > +	/*
> > +	 * Restore the direct map after the page is removed from the RMP table.
> > +	 */
> > +	if (!ret && !val->assigned) {
> > +		if (restore_direct_map(pfn, npages)) {
> > +			pr_err("Failed to map pfn 0x%llx pages %d in direct_map\n",
> 
> "Failed to map %d pages at pfn 0x... into the direct map\n"
> 
> Thx.
> 
> -- 
> Regards/Gruss,
>     Boris.
> 
> https://people.kernel.org/tglx/notes-about-netiquette