Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5;
From:   Rick Edgecombe <rick.p.edgecombe@intel.com>
To:     x86@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
        dave.hansen@linux.intel.com, hpa@zytor.com, luto@kernel.org,
        peterz@infradead.org, kirill.shutemov@linux.intel.com,
        elena.reshetova@intel.com, isaku.yamahata@intel.com,
        seanjc@google.com, Michael Kelley <mikelley@microsoft.com>,
        thomas.lendacky@amd.com, decui@microsoft.com,
        sathyanarayanan.kuppuswamy@linux.intel.com, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org
Cc:     rick.p.edgecombe@intel.com, Heiko Carstens <hca@linux.ibm.com>,
        Vasily Gorbik <gor@linux.ibm.com>,
        Alexander Gordeev <agordeev@linux.ibm.com>,
        Christian Borntraeger <borntraeger@linux.ibm.com>,
        Sven Schnelle <svens@linux.ibm.com>,
        Dave Hansen <dave.hansen@intel.com>
Subject: [PATCH 01/10] mm: Add helper for freeing decrypted memory
Date:   Tue, 17 Oct 2023 13:24:56 -0700
Message-Id: <20231017202505.340906-2-rick.p.edgecombe@intel.com>
In-Reply-To: <20231017202505.340906-1-rick.p.edgecombe@intel.com>
References: <20231017202505.340906-1-rick.p.edgecombe@intel.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

When freeing decrypted memory to the page allocator the memory needs to be
manually re-encrypted beforehand. If this step is skipped, then the next
user of those pages will have the contents inadvertently exposed to
the guest, or cause the guest to crash if the page is used in way
disallowed by HW (i.e. for executable code or as a page table).

Unfortunately, there are many instance of patterns like:
set_memory_encrypted(pages);
free_pages(pages);

...or...

if (set_memory_decrypted(addr, 1))
	free_pages(pages);

This is a problem because set_memory_encrypted() and
set_memory_decrypted() can be failed by the untrusted host in such a way
that an error is returned and the resulting memory is shared.

To aid in a tree-wide cleanup of these callers, add a
free_decrypted_pages() function that will first try to encrypt the pages
before returning them. If it is not successful, have it leak the pages and
warn about this. This is preferable to returning shared pages to allocator
or panicking.

In some cases the code path's for freeing decrypted memory handle both
encrypted and decrypted pages. In this case, rely on set_memory() to
handle being asked to convert memory to the state it is already in.

Going forward, rely on cross-arch callers to find and use
free_decrypted_pages() instead of resorting to more heavy handed solutions
like terminating the guest when nasty VMM behavior is observed.

To make s390's arch set_memory_XXcrypted() definitions available in
linux/set_memory.h, add include for s390's asm version of set_memory.h.

Cc: Heiko Carstens <hca@linux.ibm.com>
Cc: Vasily Gorbik <gor@linux.ibm.com>
Cc: Alexander Gordeev <agordeev@linux.ibm.com>
Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Sven Schnelle <svens@linux.ibm.com>
Cc: linux-s390@vger.kernel.org
Suggested-by: Dave Hansen <dave.hansen@intel.com>
Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
 arch/s390/include/asm/set_memory.h |  1 +
 include/linux/set_memory.h         | 13 +++++++++++++
 2 files changed, 14 insertions(+)

diff --git a/arch/s390/include/asm/set_memory.h b/arch/s390/include/asm/set_memory.h
index 06fbabe2f66c..09d36ebd64b5 100644
--- a/arch/s390/include/asm/set_memory.h
+++ b/arch/s390/include/asm/set_memory.h
@@ -3,6 +3,7 @@
 #define _ASMS390_SET_MEMORY_H
 
 #include <linux/mutex.h>
+#include <linux/mem_encrypt.h>
 
 extern struct mutex cpa_mutex;
 
diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h
index 95ac8398ee72..a898b14b6b1f 100644
--- a/include/linux/set_memory.h
+++ b/include/linux/set_memory.h
@@ -5,6 +5,8 @@
 #ifndef _LINUX_SET_MEMORY_H_
 #define _LINUX_SET_MEMORY_H_
 
+#include <linux/gfp.h>
+
 #ifdef CONFIG_ARCH_HAS_SET_MEMORY
 #include <asm/set_memory.h>
 #else
@@ -78,4 +80,15 @@ static inline int set_memory_decrypted(unsigned long addr, int numpages)
 }
 #endif /* CONFIG_ARCH_HAS_MEM_ENCRYPT */
 
+static inline void free_decrypted_pages(unsigned long addr, int order)
+{
+	int ret = set_memory_encrypted(addr, 1 << order);
+
+	if (ret) {
+		WARN_ONCE(1, "Failed to re-encrypt memory before freeing, leaking pages!\n");
+		return;
+	}
+	free_pages(addr, order);
+}
+
 #endif /* _LINUX_SET_MEMORY_H_ */
-- 
2.34.1