LinuxLists.cc - [PATCH v2 00/18] Add Cgroup support for SGX EPC memory

[permalink] [raw]

Subject: [PATCH v2 18/18] Docs/x86/sgx: Add description for cgroup support

Add initial documentation of how to regulate the distribution of
SGX Enclave Page Cache (EPC) memory via the Miscellaneous cgroup
controller.

Signed-off-by: Sean Christopherson <[email protected]>
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Cc: Sean Christopherson <[email protected]>
Reviewed-by: Bagas Sanjaya <[email protected]>
---
Documentation/x86/sgx.rst | 77 +++++++++++++++++++++++++++++++++++++++
1 file changed, 77 insertions(+)

diff --git a/Documentation/x86/sgx.rst b/Documentation/x86/sgx.rst
index 2bcbffacbed5..f6ca5594dcf2 100644
--- a/Documentation/x86/sgx.rst
+++ b/Documentation/x86/sgx.rst
@@ -300,3 +300,80 @@ to expected failures and handle them as follows:
first call. It indicates a bug in the kernel or the userspace client
if any of the second round of ``SGX_IOC_VEPC_REMOVE_ALL`` calls has
a return code other than 0.
+
+
+Cgroup Support
+==============
+
+The "sgx_epc" resource within the Miscellaneous cgroup controller regulates
+distribution of SGX EPC memory, which is a subset of system RAM that
+is used to provide SGX-enabled applications with protected memory,
+and is otherwise inaccessible, i.e. shows up as reserved in
+/proc/iomem and cannot be read/written outside of an SGX enclave.
+
+Although current systems implement EPC by stealing memory from RAM,
+for all intents and purposes the EPC is independent from normal system
+memory, e.g. must be reserved at boot from RAM and cannot be converted
+between EPC and normal memory while the system is running. The EPC is
+managed by the SGX subsystem and is not accounted by the memory
+controller. Note that this is true only for EPC memory itself, i.e.
+normal memory allocations related to SGX and EPC memory, e.g. the
+backing memory for evicted EPC pages, are accounted, limited and
+protected by the memory controller.
+
+Much like normal system memory, EPC memory can be overcommitted via
+virtual memory techniques and pages can be swapped out of the EPC
+to their backing store (normal system memory allocated via shmem).
+The SGX EPC subsystem is analogous to the memory subsytem, and
+it implements limit and protection models for EPC memory.
+
+SGX EPC Interface Files
+-----------------------
+
+For a generic description of the Miscellaneous controller interface
+files, please see Documentation/admin-guide/cgroup-v2.rst
+
+All SGX EPC memory amounts are in bytes unless explicitly stated
+otherwise. If a value which is not PAGE_SIZE aligned is written,
+the actual value used by the controller will be rounded down to
+the closest PAGE_SIZE multiple.
+
+ misc.capacity
+ A read-only flat-keyed file shown only in the root cgroup.
+ The sgx_epc resource will show the total amount of EPC
+ memory available on the platform.
+
+ misc.current
+ A read-only flat-keyed file shown in the non-root cgroups.
+ The sgx_epc resource will show the current active EPC memory
+ usage of the cgroup and its descendants. EPC pages that are
+ swapped out to backing RAM are not included in the current count.
+
+ misc.max
+ A read-write single value file which exists on non-root
+ cgroups. The sgx_epc resource will show the EPC usage
+ hard limit. The default is "max".
+
+ If a cgroup's EPC usage reaches this limit, EPC allocations,
+ e.g. for page fault handling, will be blocked until EPC can
+ be reclaimed from the cgroup. If EPC cannot be reclaimed in
+ a timely manner, reclaim will be forced, e.g. by ignoring LRU.
+
+ misc.events
+ A read-write flat-keyed file which exists on non-root cgroups.
+ Writes to the file reset the event counters to zero. A value
+ change in this file generates a file modified event.
+
+ max
+ The number of times the cgroup has triggered a reclaim
+ due to its EPC usage approaching (or exceeding) its max
+ EPC boundary.
+
+Migration
+---------
+
+Once an EPC page is charged to a cgroup (during allocation), it
+remains charged to the original cgroup until the page is released
+or reclaimed. Migrating a process to a different cgroup doesn't
+move the EPC charges that it incurred while in the previous cgroup
+to its new cgroup.
--
2.38.1

2022-12-02 19:00:08

[permalink] [raw]

Subject: [PATCH v2 11/18] x86/sgx: Prepare for multiple LRUs

From: Sean Christopherson <[email protected]>

Add sgx_can_reclaim() wrapper so that in a subsequent patch, multiple LRUs
can be used cleanly.

Signed-off-by: Sean Christopherson <[email protected]>
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Cc: Sean Christopherson <[email protected]>
---
arch/x86/kernel/cpu/sgx/main.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index db96483e2e74..96399e2016a8 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -394,10 +394,15 @@ static int sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
return ret;
}

+static bool sgx_can_reclaim(void)
+{
+ return !list_empty(&sgx_global_lru.reclaimable);
+}
+
static bool sgx_should_reclaim(unsigned long watermark)
{
return atomic_long_read(&sgx_nr_free_pages) < watermark &&
- !list_empty(&sgx_global_lru.reclaimable);
+ sgx_can_reclaim();
}

/*
@@ -606,7 +611,7 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
break;
}

- if (list_empty(&sgx_global_lru.reclaimable))
+ if (!sgx_can_reclaim())
return ERR_PTR(-ENOMEM);

if (!reclaim) {
--
2.38.1

2022-12-02 19:00:23

[permalink] [raw]

Subject: [PATCH v2 10/18] x86/sgx: Add option to ignore age of page during EPC reclaim

From: Sean Christopherson <[email protected]>

Add a flag to sgx_reclaim_pages() to instruct it to ignore the age of
page, i.e. reclaim the page even if it's young. The EPC cgroup will use
the flag to enforce its limits by draining the reclaimable lists before
resorting to other measures, e.g. forcefully reclaimable "unreclaimable"
pages by killing enclaves.

Signed-off-by: Sean Christopherson <[email protected]>
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Cc: Sean Christopherson <[email protected]>
---
arch/x86/kernel/cpu/sgx/main.c | 46 +++++++++++++++++++++-------------
1 file changed, 29 insertions(+), 17 deletions(-)

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index a4a65eadfb79..db96483e2e74 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -31,6 +31,10 @@ static DEFINE_XARRAY(sgx_epc_address_space);
* with sgx_global_lru.lock acquired.
*/
static struct sgx_epc_lru_lists sgx_global_lru;
+static inline struct sgx_epc_lru_lists *sgx_lru_lists(struct sgx_epc_page *epc_page)
+{
+ return &sgx_global_lru;
+}

static atomic_long_t sgx_nr_free_pages = ATOMIC_LONG_INIT(0);

@@ -278,6 +282,7 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
/**
* sgx_reclaim_pages() - Reclaim EPC pages from the consumers
* @nr_to_scan: Number of EPC pages to scan for reclaim
+ * @ignore_age: Reclaim a page even if it is young
*
* Take a fixed number of pages from the head of the active page pool and
* reclaim them to the enclave's private shmem files. Skip the pages, which have
@@ -291,11 +296,12 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
* problematic as it would increase the lock contention too much, which would
* halt forward progress.
*/
-static int __sgx_reclaim_pages(int nr_to_scan)
+static int __sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
{
struct sgx_backing backing[SGX_MAX_NR_TO_RECLAIM];
struct sgx_epc_page *epc_page, *tmp;
struct sgx_encl_page *encl_page;
+ struct sgx_epc_lru_lists *lru;
pgoff_t page_index;
LIST_HEAD(iso);
int ret;
@@ -332,7 +338,8 @@ static int __sgx_reclaim_pages(int nr_to_scan)
list_for_each_entry_safe(epc_page, tmp, &iso, list) {
encl_page = epc_page->encl_owner;

- if (i == SGX_MAX_NR_TO_RECLAIM || !sgx_reclaimer_age(epc_page))
+ if (i == SGX_MAX_NR_TO_RECLAIM ||
+ (!ignore_age && !sgx_reclaimer_age(epc_page)))
goto skip;

page_index = PFN_DOWN(encl_page->desc - encl_page->encl->base);
@@ -350,10 +357,11 @@ static int __sgx_reclaim_pages(int nr_to_scan)
continue;

skip:
- spin_lock(&sgx_global_lru.lock);
+ lru = sgx_lru_lists(epc_page);
+ spin_lock(&lru->lock);
epc_page->flags &= ~SGX_EPC_PAGE_RECLAIM_IN_PROGRESS;
- sgx_epc_move_reclaimable(&sgx_global_lru, epc_page);
- spin_unlock(&sgx_global_lru.lock);
+ sgx_epc_move_reclaimable(lru, epc_page);
+ spin_unlock(&lru->lock);

kref_put(&encl_page->encl->refcount, sgx_encl_release);
}
@@ -377,11 +385,11 @@ static int __sgx_reclaim_pages(int nr_to_scan)
return i;
}

-static int sgx_reclaim_pages(int nr_to_scan)
+static int sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
{
int ret;

- ret = __sgx_reclaim_pages(nr_to_scan);
+ ret = __sgx_reclaim_pages(nr_to_scan, ignore_age);
cond_resched();
return ret;
}
@@ -402,7 +410,7 @@ static bool sgx_should_reclaim(unsigned long watermark)
void sgx_reclaim_direct(void)
{
if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
- __sgx_reclaim_pages(SGX_NR_TO_SCAN);
+ __sgx_reclaim_pages(SGX_NR_TO_SCAN, false);
}

static int ksgxd(void *p)
@@ -428,7 +436,7 @@ static int ksgxd(void *p)
sgx_should_reclaim(SGX_NR_HIGH_PAGES));

if (sgx_should_reclaim(SGX_NR_HIGH_PAGES))
- sgx_reclaim_pages(SGX_NR_TO_SCAN);
+ sgx_reclaim_pages(SGX_NR_TO_SCAN, false);
}

return 0;
@@ -522,15 +530,17 @@ struct sgx_epc_page *__sgx_alloc_epc_page(void)
*/
void sgx_record_epc_page(struct sgx_epc_page *page, unsigned long flags)
{
- spin_lock(&sgx_global_lru.lock);
+ struct sgx_epc_lru_lists *lru = sgx_lru_lists(page);
+
+ spin_lock(&lru->lock);
WARN_ON(page->flags & (SGX_EPC_PAGE_RECLAIMER_TRACKED |
SGX_EPC_PAGE_RECLAIM_IN_PROGRESS));
page->flags |= flags;
if (flags & SGX_EPC_PAGE_RECLAIMER_TRACKED)
- sgx_epc_push_reclaimable(&sgx_global_lru, page);
+ sgx_epc_push_reclaimable(lru, page);
else
- sgx_epc_push_unreclaimable(&sgx_global_lru, page);
- spin_unlock(&sgx_global_lru.lock);
+ sgx_epc_push_unreclaimable(lru, page);
+ spin_unlock(&lru->lock);
}

/**
@@ -545,14 +555,16 @@ void sgx_record_epc_page(struct sgx_epc_page *page, unsigned long flags)
*/
int sgx_drop_epc_page(struct sgx_epc_page *page)
{
- spin_lock(&sgx_global_lru.lock);
+ struct sgx_epc_lru_lists *lru = sgx_lru_lists(page);
+
+ spin_lock(&lru->lock);
if ((page->flags & SGX_EPC_PAGE_RECLAIMER_TRACKED) &&
(page->flags & SGX_EPC_PAGE_RECLAIM_IN_PROGRESS)) {
- spin_unlock(&sgx_global_lru.lock);
+ spin_unlock(&lru->lock);
return -EBUSY;
}
list_del(&page->list);
- spin_unlock(&sgx_global_lru.lock);
+ spin_unlock(&lru->lock);

page->flags &= ~(SGX_EPC_PAGE_RECLAIMER_TRACKED |
SGX_EPC_PAGE_RECLAIM_IN_PROGRESS |
@@ -607,7 +619,7 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
break;
}

- sgx_reclaim_pages(SGX_NR_TO_SCAN);
+ sgx_reclaim_pages(SGX_NR_TO_SCAN, false);
}

if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
--
2.38.1

2022-12-02 19:00:35

[permalink] [raw]

Subject: [PATCH v2 15/18] cgroup/misc: Add per resource callbacks for css events

Consumers of the misc cgroup controller might need to perform separate actions
in the event of a cgroup alloc, free or release call. In addition,
writes to the max value may also need separate action. Add the ability
to allow downstream users to setup callbacks for these operations, and
call the per resource type callback when appropriate.

This code will be utilized by the SGX driver in a future patch.

Signed-off-by: Kristen Carlson Accardi <[email protected]>
---
include/linux/misc_cgroup.h | 6 +++++
kernel/cgroup/misc.c | 51 ++++++++++++++++++++++++++++++++++---
2 files changed, 54 insertions(+), 3 deletions(-)

diff --git a/include/linux/misc_cgroup.h b/include/linux/misc_cgroup.h
index c238207d1615..83620e7c4bb1 100644
--- a/include/linux/misc_cgroup.h
+++ b/include/linux/misc_cgroup.h
@@ -37,6 +37,12 @@ struct misc_res {
unsigned long max;
atomic_long_t usage;
atomic_long_t events;
+
+ /* per resource callback ops */
+ int (*misc_cg_alloc)(struct misc_cg *cg);
+ void (*misc_cg_free)(struct misc_cg *cg);
+ void (*misc_cg_released)(struct misc_cg *cg);
+ void (*misc_cg_max_write)(struct misc_cg *cg);
};

/**
diff --git a/kernel/cgroup/misc.c b/kernel/cgroup/misc.c
index fe3e8a0eb7ed..3d17afd5b7a8 100644
--- a/kernel/cgroup/misc.c
+++ b/kernel/cgroup/misc.c
@@ -278,10 +278,13 @@ static ssize_t misc_cg_max_write(struct kernfs_open_file *of, char *buf,

cg = css_misc(of_css(of));

- if (READ_ONCE(misc_res_capacity[type]))
+ if (READ_ONCE(misc_res_capacity[type])) {
WRITE_ONCE(cg->res[type].max, max);
- else
+ if (cg->res[type].misc_cg_max_write)
+ cg->res[type].misc_cg_max_write(cg);
+ } else {
ret = -EINVAL;
+ }

return ret ? ret : nbytes;
}
@@ -385,23 +388,39 @@ static struct cftype misc_cg_files[] = {
static struct cgroup_subsys_state *
misc_cg_alloc(struct cgroup_subsys_state *parent_css)
{
+ struct misc_cg *parent_cg;
enum misc_res_type i;
struct misc_cg *cg;
+ int ret;

if (!parent_css) {
cg = &root_cg;
+ parent_cg = &root_cg;
} else {
cg = kzalloc(sizeof(*cg), GFP_KERNEL);
if (!cg)
return ERR_PTR(-ENOMEM);
+ parent_cg = css_misc(parent_css);
}

for (i = 0; i < MISC_CG_RES_TYPES; i++) {
WRITE_ONCE(cg->res[i].max, MAX_NUM);
atomic_long_set(&cg->res[i].usage, 0);
+ if (parent_cg->res[i].misc_cg_alloc) {
+ ret = parent_cg->res[i].misc_cg_alloc(cg);
+ if (ret)
+ goto alloc_err;
+ }
}

return &cg->css;
+
+alloc_err:
+ for (i = 0; i < MISC_CG_RES_TYPES; i++)
+ if (parent_cg->res[i].misc_cg_free)
+ cg->res[i].misc_cg_free(cg);
+ kfree(cg);
+ return ERR_PTR(ret);
}

/**
@@ -412,13 +431,39 @@ misc_cg_alloc(struct cgroup_subsys_state *parent_css)
*/
static void misc_cg_free(struct cgroup_subsys_state *css)
{
- kfree(css_misc(css));
+ struct misc_cg *cg = css_misc(css);
+ enum misc_res_type i;
+
+ for (i = 0; i < MISC_CG_RES_TYPES; i++)
+ if (cg->res[i].misc_cg_free)
+ cg->res[i].misc_cg_free(cg);
+
+ kfree(cg);
+}
+
+/**
+ * misc_cg_released() - Release the misc cgroup
+ * @css: cgroup subsys object.
+ *
+ * Call the misc_cg resource type released callbacks.
+ *
+ * Context: Any context.
+ */
+static void misc_cg_released(struct cgroup_subsys_state *css)
+{
+ struct misc_cg *cg = css_misc(css);
+ enum misc_res_type i;
+
+ for (i = 0; i < MISC_CG_RES_TYPES; i++)
+ if (cg->res[i].misc_cg_released)
+ cg->res[i].misc_cg_released(cg);
}

/* Cgroup controller callbacks */
struct cgroup_subsys misc_cgrp_subsys = {
.css_alloc = misc_cg_alloc,
.css_free = misc_cg_free,
+ .css_released = misc_cg_released,
.legacy_cftypes = misc_cg_files,
.dfl_cftypes = misc_cg_files,
};
--
2.38.1

2022-12-02 19:01:56

[permalink] [raw]

Subject: [PATCH v2 09/18] x86/sgx: Return the number of EPC pages that were successfully reclaimed

From: Sean Christopherson <[email protected]>

Return the number of reclaimed pages from sgx_reclaim_pages(), the EPC
cgroup will use the result to track the success rate of its reclaim
calls, e.g. to escalate to a more forceful reclaiming mode if necessary.

Signed-off-by: Sean Christopherson <[email protected]>
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Cc: Sean Christopherson <[email protected]>
---
arch/x86/kernel/cpu/sgx/main.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index f201ca85212f..a4a65eadfb79 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -291,7 +291,7 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
* problematic as it would increase the lock contention too much, which would
* halt forward progress.
*/
-static void __sgx_reclaim_pages(int nr_to_scan)
+static int __sgx_reclaim_pages(int nr_to_scan)
{
struct sgx_backing backing[SGX_MAX_NR_TO_RECLAIM];
struct sgx_epc_page *epc_page, *tmp;
@@ -326,7 +326,7 @@ static void __sgx_reclaim_pages(int nr_to_scan)
spin_unlock(&sgx_global_lru.lock);

if (list_empty(&iso))
- return;
+ return 0;

i = 0;
list_for_each_entry_safe(epc_page, tmp, &iso, list) {
@@ -374,12 +374,16 @@ static void __sgx_reclaim_pages(int nr_to_scan)

sgx_free_epc_page(epc_page);
}
+ return i;
}

-static void sgx_reclaim_pages(int nr_to_scan)
+static int sgx_reclaim_pages(int nr_to_scan)
{
- __sgx_reclaim_pages(nr_to_scan);
+ int ret;
+
+ ret = __sgx_reclaim_pages(nr_to_scan);
cond_resched();
+ return ret;
}

static bool sgx_should_reclaim(unsigned long watermark)
--
2.38.1

2022-12-02 19:38:06

[permalink] [raw]

Subject: [PATCH v2 02/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

From: Sean Christopherson <[email protected]>

When allocating new Version Array (VA) pages, pass the struct sgx_encl
of the enclave that is allocating the page. sgx_alloc_epc_page() will
store this value in the encl_owner field of the struct sgx_epc_page. In
a later patch, VA pages will be placed in an unreclaimable queue,
and then when the cgroup max limit is reached and there are no more
reclaimable pages and the enclave must be oom killed, all the
VA pages associated with that enclave can be uncharged and freed.

Signed-off-by: Sean Christopherson <[email protected]>
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Cc: Sean Christopherson <[email protected]>
---
arch/x86/kernel/cpu/sgx/encl.c | 5 +++--
arch/x86/kernel/cpu/sgx/encl.h | 2 +-
arch/x86/kernel/cpu/sgx/ioctl.c | 2 +-
arch/x86/kernel/cpu/sgx/sgx.h | 1 +
4 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c
index f40d64206ded..4eaf9d21e71b 100644
--- a/arch/x86/kernel/cpu/sgx/encl.c
+++ b/arch/x86/kernel/cpu/sgx/encl.c
@@ -1193,6 +1193,7 @@ void sgx_zap_enclave_ptes(struct sgx_encl *encl, unsigned long addr)

/**
* sgx_alloc_va_page() - Allocate a Version Array (VA) page
+ * @encl: The enclave that this page is allocated to.
* @reclaim: Reclaim EPC pages directly if none available. Enclave
* mutex should not be held if this is set.
*
@@ -1202,12 +1203,12 @@ void sgx_zap_enclave_ptes(struct sgx_encl *encl, unsigned long addr)
* a VA page,
* -errno otherwise
*/
-struct sgx_epc_page *sgx_alloc_va_page(bool reclaim)
+struct sgx_epc_page *sgx_alloc_va_page(struct sgx_encl *encl, bool reclaim)
{
struct sgx_epc_page *epc_page;
int ret;

- epc_page = sgx_alloc_epc_page(NULL, reclaim);
+ epc_page = sgx_alloc_epc_page(encl, reclaim);
if (IS_ERR(epc_page))
return ERR_CAST(epc_page);

diff --git a/arch/x86/kernel/cpu/sgx/encl.h b/arch/x86/kernel/cpu/sgx/encl.h
index f94ff14c9486..831d63f80f5a 100644
--- a/arch/x86/kernel/cpu/sgx/encl.h
+++ b/arch/x86/kernel/cpu/sgx/encl.h
@@ -116,7 +116,7 @@ struct sgx_encl_page *sgx_encl_page_alloc(struct sgx_encl *encl,
unsigned long offset,
u64 secinfo_flags);
void sgx_zap_enclave_ptes(struct sgx_encl *encl, unsigned long addr);
-struct sgx_epc_page *sgx_alloc_va_page(bool reclaim);
+struct sgx_epc_page *sgx_alloc_va_page(struct sgx_encl *encl, bool reclaim);
unsigned int sgx_alloc_va_slot(struct sgx_va_page *va_page);
void sgx_free_va_slot(struct sgx_va_page *va_page, unsigned int offset);
bool sgx_va_page_full(struct sgx_va_page *va_page);
diff --git a/arch/x86/kernel/cpu/sgx/ioctl.c b/arch/x86/kernel/cpu/sgx/ioctl.c
index ebe79d60619f..9a1bb3c3211a 100644
--- a/arch/x86/kernel/cpu/sgx/ioctl.c
+++ b/arch/x86/kernel/cpu/sgx/ioctl.c
@@ -30,7 +30,7 @@ struct sgx_va_page *sgx_encl_grow(struct sgx_encl *encl, bool reclaim)
if (!va_page)
return ERR_PTR(-ENOMEM);

- va_page->epc_page = sgx_alloc_va_page(reclaim);
+ va_page->epc_page = sgx_alloc_va_page(encl, reclaim);
if (IS_ERR(va_page->epc_page)) {
err = ERR_CAST(va_page->epc_page);
kfree(va_page);
diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
index d16a8baa28d4..39cb15a8abcb 100644
--- a/arch/x86/kernel/cpu/sgx/sgx.h
+++ b/arch/x86/kernel/cpu/sgx/sgx.h
@@ -39,6 +39,7 @@ struct sgx_epc_page {
struct sgx_encl_page *encl_owner;
/* Use when SGX_EPC_PAGE_KVM_GUEST set in ->flags: */
void __user *vepc_vaddr;
+ struct sgx_encl *encl;
};
struct list_head list;
};
--
2.38.1

2022-12-02 19:44:42

[permalink] [raw]

Subject: [PATCH v2 13/18] x86/sgx: Add helper to grab pages from an arbitrary EPC LRU

From: Sean Christopherson <[email protected]>

Move the isolation loop into a standalone helper, sgx_isolate_pages(),
in preparation for existence of multiple LRUs. Expose the helper to
other SGX code so that it can be called from the EPC cgroup code, e.g.
to isolate pages from a single cgroup LRU. Exposing the isolation loop
allows the cgroup iteration logic to be wholly encapsulated within the
cgroup code.

Signed-off-by: Sean Christopherson <[email protected]>
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Cc: Sean Christopherson <[email protected]>
---
arch/x86/kernel/cpu/sgx/main.c | 68 +++++++++++++++++++++-------------
arch/x86/kernel/cpu/sgx/sgx.h | 2 +
2 files changed, 44 insertions(+), 26 deletions(-)

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index c947b4ae06f3..a59550fa150b 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -280,7 +280,46 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
}

/**
- * sgx_reclaim_pages() - Reclaim EPC pages from the consumers
+ * sgx_isolate_epc_pages() - Isolate pages from an LRU for reclaim
+ * @lru: LRU from which to reclaim
+ * @nr_to_scan: Number of pages to scan for reclaim
+ * @dst: Destination list to hold the isolated pages
+ */
+void sgx_isolate_epc_pages(struct sgx_epc_lru_lists *lru, int *nr_to_scan,
+ struct list_head *dst)
+{
+ struct sgx_encl_page *encl_page;
+ struct sgx_epc_page *epc_page;
+
+ spin_lock(&lru->lock);
+ for (; *nr_to_scan > 0; --(*nr_to_scan)) {
+ if (list_empty(&lru->reclaimable))
+ break;
+
+ epc_page = sgx_epc_peek_reclaimable(lru);
+ if (!epc_page)
+ break;
+
+ encl_page = epc_page->encl_owner;
+
+ if (WARN_ON_ONCE(!(epc_page->flags & SGX_EPC_PAGE_ENCLAVE)))
+ continue;
+
+ if (kref_get_unless_zero(&encl_page->encl->refcount)) {
+ epc_page->flags |= SGX_EPC_PAGE_RECLAIM_IN_PROGRESS;
+ list_move_tail(&epc_page->list, dst);
+ } else {
+ /* The owner is freeing the page, remove it from the
+ * LRU list
+ */
+ epc_page->flags &= ~SGX_EPC_PAGE_RECLAIMER_TRACKED;
+ list_del_init(&epc_page->list);
+ }
+ }
+ spin_unlock(&lru->lock);
+}
+
+/**
* sgx_reclaim_epc_pages() - Reclaim EPC pages from the consumers
* @nr_to_scan: Number of EPC pages to scan for reclaim
* @ignore_age: Reclaim a page even if it is young
@@ -305,37 +344,14 @@ static int __sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
struct sgx_epc_lru_lists *lru;
pgoff_t page_index;
LIST_HEAD(iso);
+ int i = 0;
int ret;
- int i;
-
- spin_lock(&sgx_global_lru.lock);
- for (i = 0; i < nr_to_scan; i++) {
- epc_page = sgx_epc_peek_reclaimable(&sgx_global_lru);
- if (!epc_page)
- break;
-
- encl_page = epc_page->encl_owner;

- if (WARN_ON_ONCE(!(epc_page->flags & SGX_EPC_PAGE_ENCLAVE)))
- continue;
-
- if (kref_get_unless_zero(&encl_page->encl->refcount) != 0) {
- epc_page->flags |= SGX_EPC_PAGE_RECLAIM_IN_PROGRESS;
- list_move_tail(&epc_page->list, &iso);
- } else {
- /* The owner is freeing the page, remove it from the
- * LRU list
- */
- epc_page->flags &= ~SGX_EPC_PAGE_RECLAIMER_TRACKED;
- list_del_init(&epc_page->list);
- }
- }
- spin_unlock(&sgx_global_lru.lock);
+ sgx_isolate_epc_pages(&sgx_global_lru, &nr_to_scan, &iso);

if (list_empty(&iso))
return 0;

- i = 0;
list_for_each_entry_safe(epc_page, tmp, &iso, list) {
encl_page = epc_page->encl_owner;

diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
index ce859331ddf5..4499a5d5547d 100644
--- a/arch/x86/kernel/cpu/sgx/sgx.h
+++ b/arch/x86/kernel/cpu/sgx/sgx.h
@@ -207,6 +207,8 @@ void sgx_record_epc_page(struct sgx_epc_page *page, unsigned long flags);
int sgx_drop_epc_page(struct sgx_epc_page *page);
struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim);
int sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age);
+void sgx_isolate_epc_pages(struct sgx_epc_lru_lists *lrus, int *nr_to_scan,
+ struct list_head *dst);

void sgx_ipi_cb(void *info);

--
2.38.1

2022-12-02 19:46:37

[permalink] [raw]

Subject: [PATCH v2 17/18] x86/sgx: Add support for misc cgroup controller

Implement support for cgroup control of SGX Enclave Page Cache (EPC)
memory using the misc cgroup controller. EPC memory is independent
from normal system memory, e.g. must be reserved at boot from RAM and
cannot be converted between EPC and normal memory while the system is
running. EPC is managed by the SGX subsystem and is not accounted by
the memory controller.

Much like normal system memory, EPC memory can be overcommitted via
virtual memory techniques and pages can be swapped out of the EPC to
their backing store (normal system memory, e.g. shmem). The SGX EPC
subsystem is analogous to the memory subsytem and the SGX EPC controller
is in turn analogous to the memory controller; it implements limit and
protection models for EPC memory.

The misc controller provides a mechanism to set a hard limit of EPC
usage via the "sgx_epc" resource in "misc.max". The total EPC memory
available on the system is reported via the "sgx_epc" resource in
"misc.capacity".

This patch was modified from its original version to use the misc cgroup
controller instead of a custom controller.

Signed-off-by: Sean Christopherson <[email protected]>
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Cc: Sean Christopherson <[email protected]>
---
arch/x86/Kconfig | 13 +
arch/x86/kernel/cpu/sgx/Makefile | 1 +
arch/x86/kernel/cpu/sgx/epc_cgroup.c | 539 +++++++++++++++++++++++++++
arch/x86/kernel/cpu/sgx/epc_cgroup.h | 59 +++
arch/x86/kernel/cpu/sgx/main.c | 86 ++++-
arch/x86/kernel/cpu/sgx/sgx.h | 6 +-
6 files changed, 688 insertions(+), 16 deletions(-)
create mode 100644 arch/x86/kernel/cpu/sgx/epc_cgroup.c
create mode 100644 arch/x86/kernel/cpu/sgx/epc_cgroup.h

diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index f9920f1341c8..0eeae4ebe1c3 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -1936,6 +1936,19 @@ config X86_SGX

If unsure, say N.

+config CGROUP_SGX_EPC
+ bool "Miscellaneous Cgroup Controller for Enclave Page Cache (EPC) for Intel SGX"
+ depends on X86_SGX && CGROUP_MISC
+ help
+ Provides control over the EPC footprint of tasks in a cgroup via
+ the Miscellaneous cgroup controller.
+
+ EPC is a subset of regular memory that is usable only by SGX
+ enclaves and is very limited in quantity, e.g. less than 1%
+ of total DRAM.
+
+ Say N if unsure.
+
config EFI
bool "EFI runtime service support"
depends on ACPI
diff --git a/arch/x86/kernel/cpu/sgx/Makefile b/arch/x86/kernel/cpu/sgx/Makefile
index 9c1656779b2a..12901a488da7 100644
--- a/arch/x86/kernel/cpu/sgx/Makefile
+++ b/arch/x86/kernel/cpu/sgx/Makefile
@@ -4,3 +4,4 @@ obj-y += \
ioctl.o \
main.o
obj-$(CONFIG_X86_SGX_KVM) += virt.o
+obj-$(CONFIG_CGROUP_SGX_EPC) += epc_cgroup.o
diff --git a/arch/x86/kernel/cpu/sgx/epc_cgroup.c b/arch/x86/kernel/cpu/sgx/epc_cgroup.c
new file mode 100644
index 000000000000..d668a67fde84
--- /dev/null
+++ b/arch/x86/kernel/cpu/sgx/epc_cgroup.c
@@ -0,0 +1,539 @@
+// SPDX-License-Identifier: GPL-2.0
+// Copyright(c) 2022 Intel Corporation.
+
+#include <linux/atomic.h>
+#include <linux/kernel.h>
+#include <linux/ratelimit.h>
+#include <linux/sched/signal.h>
+#include <linux/slab.h>
+#include <linux/threads.h>
+
+#include "epc_cgroup.h"
+
+#define SGX_EPC_RECLAIM_MIN_PAGES 16UL
+#define SGX_EPC_RECLAIM_MAX_PAGES 64UL
+#define SGX_EPC_RECLAIM_IGNORE_AGE_THRESHOLD 5
+#define SGX_EPC_RECLAIM_OOM_THRESHOLD 5
+
+static struct workqueue_struct *sgx_epc_cg_wq;
+
+struct sgx_epc_reclaim_control {
+ struct sgx_epc_cgroup *epc_cg;
+ int nr_fails;
+ bool ignore_age;
+};
+
+static inline unsigned long sgx_epc_cgroup_page_counter_read(struct sgx_epc_cgroup *epc_cg)
+{
+ return atomic_long_read(&epc_cg->cg->res[MISC_CG_RES_SGX_EPC].usage) / PAGE_SIZE;
+}
+
+static inline unsigned long sgx_epc_cgroup_max_pages(struct sgx_epc_cgroup *epc_cg)
+{
+ return READ_ONCE(epc_cg->cg->res[MISC_CG_RES_SGX_EPC].max) / PAGE_SIZE;
+}
+
+static inline struct sgx_epc_cgroup *sgx_epc_cgroup_from_misc_cg(struct misc_cg *cg)
+{
+ if (cg)
+ return (struct sgx_epc_cgroup *)(cg->res[MISC_CG_RES_SGX_EPC].priv);
+
+ return NULL;
+}
+
+static inline struct sgx_epc_cgroup *parent_epc_cgroup(struct sgx_epc_cgroup *epc_cg)
+{
+ return sgx_epc_cgroup_from_misc_cg(misc_cg_parent(epc_cg->cg));
+}
+
+static inline bool sgx_epc_cgroup_disabled(void)
+{
+ return !cgroup_subsys_enabled(misc_cgrp_subsys);
+}
+
+/**
+ * sgx_epc_cgroup_iter - iterate over the EPC cgroup hierarchy
+ * @root: hierarchy root
+ * @prev: previously returned epc_cg, NULL on first invocation
+ * @reclaim_epoch: epoch for shared reclaim walks, NULL for full walks
+ *
+ * Return: references to children of the hierarchy below @root, or
+ * @root itself, or %NULL after a full round-trip.
+ *
+ * Caller must pass the return value in @prev on subsequent invocations
+ * for reference counting, or use sgx_epc_cgroup_iter_break() to cancel
+ * a hierarchy walk before the round-trip is complete.
+ */
+static struct sgx_epc_cgroup *sgx_epc_cgroup_iter(struct sgx_epc_cgroup *prev,
+ struct sgx_epc_cgroup *root,
+ unsigned long *reclaim_epoch)
+{
+ struct cgroup_subsys_state *css = NULL;
+ struct sgx_epc_cgroup *epc_cg = NULL;
+ struct sgx_epc_cgroup *pos = NULL;
+ bool inc_epoch = false;
+
+ if (sgx_epc_cgroup_disabled())
+ return NULL;
+
+ if (!root)
+ root = sgx_epc_cgroup_from_misc_cg(misc_cg_root());
+
+ if (prev && !reclaim_epoch)
+ pos = prev;
+
+ rcu_read_lock();
+
+start:
+ if (reclaim_epoch) {
+ /*
+ * Abort the walk if a reclaimer working from the same root has
+ * started a new walk after this reclaimer has already scanned
+ * at least one cgroup.
+ */
+ if (prev && *reclaim_epoch != root->epoch)
+ goto out;
+
+ while (1) {
+ pos = READ_ONCE(root->reclaim_iter);
+ if (!pos || css_tryget(&pos->cg->css))
+ break;
+
+ /*
+ * The css is dying, clear the reclaim_iter immediately
+ * instead of waiting for ->css_released to be called.
+ * Busy waiting serves no purpose and attempting to wait
+ * for ->css_released may actually block it from being
+ * called.
+ */
+ (void)cmpxchg(&root->reclaim_iter, pos, NULL);
+ }
+ }
+
+ if (pos)
+ css = &pos->cg->css;
+
+ while (!epc_cg) {
+ struct misc_cg *cg;
+
+ css = css_next_descendant_pre(css, &root->cg->css);
+ if (!css) {
+ /*
+ * Increment the epoch as we've reached the end of the
+ * tree and the next call to css_next_descendant_pre
+ * will restart at root. Do not update root->epoch
+ * directly as we should only do so if we update the
+ * reclaim_iter, i.e. a different thread may win the
+ * race and update the epoch for us.
+ */
+ inc_epoch = true;
+
+ /*
+ * Reclaimers share the hierarchy walk, and a new one
+ * might jump in at the end of the hierarchy. Restart
+ * at root so that we don't return NULL on a thread's
+ * initial call.
+ */
+ if (!prev)
+ continue;
+ break;
+ }
+
+ cg = css_misc(css);
+ /*
+ * Verify the css and acquire a reference. Don't take an
+ * extra reference to root as it's either the global root
+ * or is provided by the caller and so is guaranteed to be
+ * alive. Keep walking if this css is dying.
+ */
+ if (cg != root->cg && !css_tryget(&cg->css))
+ continue;
+
+ epc_cg = sgx_epc_cgroup_from_misc_cg(cg);
+ }
+
+ if (reclaim_epoch) {
+ /*
+ * reclaim_iter could have already been updated by a competing
+ * thread; check that the value hasn't changed since we read
+ * it to avoid reclaiming from the same cgroup twice. If the
+ * value did change, put all of our references and restart the
+ * entire process, for all intents and purposes we're making a
+ * new call.
+ */
+ if (cmpxchg(&root->reclaim_iter, pos, epc_cg) != pos) {
+ if (epc_cg && epc_cg != root)
+ put_misc_cg(epc_cg->cg);
+ if (pos)
+ put_misc_cg(pos->cg);
+ css = NULL;
+ epc_cg = NULL;
+ inc_epoch = false;
+ goto start;
+ }
+
+ if (inc_epoch)
+ root->epoch++;
+ if (!prev)
+ *reclaim_epoch = root->epoch;
+
+ if (pos)
+ put_misc_cg(pos->cg);
+ }
+
+out:
+ rcu_read_unlock();
+ if (prev && prev != root)
+ put_misc_cg(prev->cg);
+
+ return epc_cg;
+}
+
+/**
+ * sgx_epc_cgroup_iter_break - abort a hierarchy walk prematurely
+ * @prev: last visited cgroup as returned by sgx_epc_cgroup_iter()
+ * @root: hierarchy root
+ */
+static void sgx_epc_cgroup_iter_break(struct sgx_epc_cgroup *prev,
+ struct sgx_epc_cgroup *root)
+{
+ if (!root)
+ root = sgx_epc_cgroup_from_misc_cg(misc_cg_root());
+ if (prev && prev != root)
+ put_misc_cg(prev->cg);
+}
+
+/**
+ * sgx_epc_cgroup_lru_empty - check if a cgroup tree has no pages on its lrus
+ * @root: root of the tree to check
+ *
+ * Return: %true if all cgroups under the specified root have empty LRU lists.
+ * Used to avoid livelocks due to a cgroup having a non-zero charge count but
+ * no pages on its LRUs, e.g. due to a dead enclave waiting to be released or
+ * because all pages in the cgroup are unreclaimable.
+ */
+bool sgx_epc_cgroup_lru_empty(struct sgx_epc_cgroup *root)
+{
+ struct sgx_epc_cgroup *epc_cg;
+
+ for (epc_cg = sgx_epc_cgroup_iter(NULL, root, NULL);
+ epc_cg;
+ epc_cg = sgx_epc_cgroup_iter(epc_cg, root, NULL)) {
+ if (!list_empty(&epc_cg->lru.reclaimable)) {
+ sgx_epc_cgroup_iter_break(epc_cg, root);
+ return false;
+ }
+ }
+ return true;
+}
+
+/**
+ * sgx_epc_cgroup_isolate_pages - walk a cgroup tree and separate pages
+ * @root: root of the tree to start walking
+ * @nr_to_scan: The number of pages that need to be isolated
+ * @dst: Destination list to hold the isolated pages
+ *
+ * Walk the cgroup tree and isolate the pages in the hierarchy
+ * for reclaiming.
+ */
+void sgx_epc_cgroup_isolate_pages(struct sgx_epc_cgroup *root,
+ int *nr_to_scan, struct list_head *dst)
+{
+ struct sgx_epc_cgroup *epc_cg;
+ unsigned long epoch;
+
+ if (!*nr_to_scan)
+ return;
+
+ for (epc_cg = sgx_epc_cgroup_iter(NULL, root, &epoch);
+ epc_cg;
+ epc_cg = sgx_epc_cgroup_iter(epc_cg, root, &epoch)) {
+ sgx_isolate_epc_pages(&epc_cg->lru, nr_to_scan, dst);
+ if (!*nr_to_scan) {
+ sgx_epc_cgroup_iter_break(epc_cg, root);
+ break;
+ }
+ }
+}
+
+static int sgx_epc_cgroup_reclaim_pages(unsigned long nr_pages,
+ struct sgx_epc_reclaim_control *rc)
+{
+ /*
+ * Ensure sgx_reclaim_pages is called with a minimum and maximum
+ * number of pages. Attempting to reclaim only a few pages will
+ * often fail and is inefficient, while reclaiming a huge number
+ * of pages can result in soft lockups due to holding various
+ * locks for an extended duration. This also bounds nr_pages so
+ */
+ nr_pages = max(nr_pages, SGX_EPC_RECLAIM_MIN_PAGES);
+ nr_pages = min(nr_pages, SGX_EPC_RECLAIM_MAX_PAGES);
+
+ return sgx_reclaim_epc_pages(nr_pages, rc->ignore_age, rc->epc_cg);
+}
+
+static int sgx_epc_cgroup_reclaim_failed(struct sgx_epc_reclaim_control *rc)
+{
+ if (sgx_epc_cgroup_lru_empty(rc->epc_cg))
+ return -ENOMEM;
+
+ ++rc->nr_fails;
+ if (rc->nr_fails > SGX_EPC_RECLAIM_IGNORE_AGE_THRESHOLD)
+ rc->ignore_age = true;
+
+ return 0;
+}
+
+static inline
+void sgx_epc_reclaim_control_init(struct sgx_epc_reclaim_control *rc,
+ struct sgx_epc_cgroup *epc_cg)
+{
+ rc->epc_cg = epc_cg;
+ rc->nr_fails = 0;
+ rc->ignore_age = false;
+}
+
+/*
+ * Scheduled by sgx_epc_cgroup_try_charge() to reclaim pages from the
+ * cgroup when the cgroup is at/near its maximum capacity
+ */
+static void sgx_epc_cgroup_reclaim_work_func(struct work_struct *work)
+{
+ struct sgx_epc_reclaim_control rc;
+ struct sgx_epc_cgroup *epc_cg;
+ unsigned long cur, max;
+
+ epc_cg = container_of(work, struct sgx_epc_cgroup, reclaim_work);
+
+ sgx_epc_reclaim_control_init(&rc, epc_cg);
+
+ for (;;) {
+ max = sgx_epc_cgroup_max_pages(epc_cg);
+
+ /*
+ * Adjust the limit down by one page, the goal is to free up
+ * pages for fault allocations, not to simply obey the limit.
+ * Conditionally decrementing max also means the cur vs. max
+ * check will correctly handle the case where both are zero.
+ */
+ if (max)
+ max--;
+
+ /*
+ * Unless the limit is extremely low, in which case forcing
+ * reclaim will likely cause thrashing, force the cgroup to
+ * reclaim at least once if it's operating *near* its maximum
+ * limit by adjusting @max down by half the min reclaim size.
+ * This work func is scheduled by sgx_epc_cgroup_try_charge
+ * when it cannot directly reclaim due to being in an atomic
+ * context, e.g. EPC allocation in a fault handler. Waiting
+ * to reclaim until the cgroup is actually at its limit is less
+ * performant as it means the faulting task is effectively
+ * blocked until a worker makes its way through the global work
+ * queue.
+ */
+ if (max > SGX_EPC_RECLAIM_MAX_PAGES)
+ max -= (SGX_EPC_RECLAIM_MIN_PAGES/2);
+
+ cur = sgx_epc_cgroup_page_counter_read(epc_cg);
+ if (cur <= max)
+ break;
+
+ if (!sgx_epc_cgroup_reclaim_pages(cur - max, &rc)) {
+ if (sgx_epc_cgroup_reclaim_failed(&rc))
+ break;
+ }
+ }
+}
+
+static int __sgx_epc_cgroup_try_charge(struct sgx_epc_cgroup *epc_cg,
+ unsigned long nr_pages, bool reclaim)
+{
+ struct sgx_epc_reclaim_control rc;
+ unsigned long cur, max, over;
+ unsigned int nr_empty = 0;
+
+ if (epc_cg == sgx_epc_cgroup_from_misc_cg(misc_cg_root())) {
+ misc_cg_try_charge(MISC_CG_RES_SGX_EPC, epc_cg->cg,
+ nr_pages * PAGE_SIZE);
+ return 0;
+ }
+
+ sgx_epc_reclaim_control_init(&rc, NULL);
+
+ for (;;) {
+ if (!misc_cg_try_charge(MISC_CG_RES_SGX_EPC, epc_cg->cg,
+ nr_pages * PAGE_SIZE))
+ break;
+
+ rc.epc_cg = epc_cg;
+ max = sgx_epc_cgroup_max_pages(rc.epc_cg);
+ if (nr_pages > max)
+ return -ENOMEM;
+
+ if (signal_pending(current))
+ return -ERESTARTSYS;
+
+ if (!reclaim) {
+ queue_work(sgx_epc_cg_wq, &rc.epc_cg->reclaim_work);
+ return -EBUSY;
+ }
+
+ cur = sgx_epc_cgroup_page_counter_read(rc.epc_cg);
+ over = ((cur + nr_pages) > max) ?
+ (cur + nr_pages) - max : SGX_EPC_RECLAIM_MIN_PAGES;
+
+ if (!sgx_epc_cgroup_reclaim_pages(over, &rc)) {
+ if (sgx_epc_cgroup_reclaim_failed(&rc)) {
+ if (++nr_empty > SGX_EPC_RECLAIM_OOM_THRESHOLD)
+ return -ENOMEM;
+ schedule();
+ }
+ }
+ }
+
+ css_get_many(&epc_cg->cg->css, nr_pages);
+
+ return 0;
+}
+
+
+/**
+ * sgx_epc_cgroup_try_charge - hierarchically try to charge a single EPC page
+ * @mm: the mm_struct of the process to charge
+ * @reclaim: whether or not synchronous reclaim is allowed
+ *
+ * Returns EPC cgroup or NULL on success, -errno on failure.
+ */
+struct sgx_epc_cgroup *sgx_epc_cgroup_try_charge(struct mm_struct *mm,
+ bool reclaim)
+{
+ struct sgx_epc_cgroup *epc_cg;
+ int ret;
+
+ if (sgx_epc_cgroup_disabled())
+ return NULL;
+
+ epc_cg = sgx_epc_cgroup_from_misc_cg(get_current_misc_cg());
+ ret = __sgx_epc_cgroup_try_charge(epc_cg, 1, reclaim);
+ put_misc_cg(epc_cg->cg);
+
+ if (ret)
+ return ERR_PTR(ret);
+
+ return epc_cg;
+}
+
+/**
+ * sgx_epc_cgroup_uncharge - hierarchically uncharge EPC pages
+ * @epc_cg: the charged epc cgroup
+ */
+void sgx_epc_cgroup_uncharge(struct sgx_epc_cgroup *epc_cg)
+{
+ if (sgx_epc_cgroup_disabled())
+ return;
+
+ misc_cg_uncharge(MISC_CG_RES_SGX_EPC, epc_cg->cg, PAGE_SIZE);
+
+ if (epc_cg->cg != misc_cg_root())
+ put_misc_cg(epc_cg->cg);
+}
+
+static void sgx_epc_cgroup_oom(struct sgx_epc_cgroup *root)
+{
+ struct sgx_epc_cgroup *epc_cg;
+
+ for (epc_cg = sgx_epc_cgroup_iter(NULL, root, NULL);
+ epc_cg;
+ epc_cg = sgx_epc_cgroup_iter(epc_cg, root, NULL)) {
+ if (sgx_epc_oom(&epc_cg->lru)) {
+ sgx_epc_cgroup_iter_break(epc_cg, root);
+ return;
+ }
+ }
+}
+
+static void sgx_epc_cgroup_released(struct misc_cg *cg)
+{
+ struct sgx_epc_cgroup *dead_cg;
+ struct sgx_epc_cgroup *epc_cg;
+
+ epc_cg = sgx_epc_cgroup_from_misc_cg(cg);
+ dead_cg = epc_cg;
+
+ while ((epc_cg = parent_epc_cgroup(epc_cg)))
+ cmpxchg(&epc_cg->reclaim_iter, dead_cg, NULL);
+}
+
+static void sgx_epc_cgroup_free(struct misc_cg *cg)
+{
+ struct sgx_epc_cgroup *epc_cg;
+
+ epc_cg = sgx_epc_cgroup_from_misc_cg(cg);
+ cancel_work_sync(&epc_cg->reclaim_work);
+ kfree(epc_cg);
+}
+
+static void sgx_epc_cgroup_max_write(struct misc_cg *cg)
+{
+ struct sgx_epc_reclaim_control rc;
+ struct sgx_epc_cgroup *epc_cg;
+ unsigned int nr_empty = 0;
+ unsigned long cur, max;
+
+ epc_cg = sgx_epc_cgroup_from_misc_cg(cg);
+
+ sgx_epc_reclaim_control_init(&rc, epc_cg);
+
+ max = sgx_epc_cgroup_max_pages(epc_cg);
+
+ for (;;) {
+ cur = sgx_epc_cgroup_page_counter_read(epc_cg);
+ if (cur <= max)
+ break;
+
+ if (signal_pending(current))
+ break;
+
+ if (!sgx_epc_cgroup_reclaim_pages(cur - max, &rc)) {
+ if (sgx_epc_cgroup_reclaim_failed(&rc)) {
+ if (++nr_empty > SGX_EPC_RECLAIM_OOM_THRESHOLD)
+ sgx_epc_cgroup_oom(epc_cg);
+ schedule();
+ }
+ }
+ }
+}
+
+static int sgx_epc_cgroup_alloc(struct misc_cg *cg)
+{
+ struct sgx_epc_cgroup *epc_cg;
+
+ epc_cg = kzalloc(sizeof(struct sgx_epc_cgroup), GFP_KERNEL);
+ if (!epc_cg)
+ return -ENOMEM;
+
+ sgx_lru_init(&epc_cg->lru);
+ INIT_WORK(&epc_cg->reclaim_work, sgx_epc_cgroup_reclaim_work_func);
+ cg->res[MISC_CG_RES_SGX_EPC].misc_cg_alloc = sgx_epc_cgroup_alloc;
+ cg->res[MISC_CG_RES_SGX_EPC].misc_cg_free = sgx_epc_cgroup_free;
+ cg->res[MISC_CG_RES_SGX_EPC].misc_cg_released = sgx_epc_cgroup_released;
+ cg->res[MISC_CG_RES_SGX_EPC].misc_cg_max_write = sgx_epc_cgroup_max_write;
+ cg->res[MISC_CG_RES_SGX_EPC].priv = epc_cg;
+ epc_cg->cg = cg;
+ return 0;
+}
+
+static int __init sgx_epc_cgroup_init(void)
+{
+ if (!boot_cpu_has(X86_FEATURE_SGX))
+ return 0;
+
+ sgx_epc_cg_wq = alloc_workqueue("sgx_epc_cg_wq",
+ WQ_UNBOUND | WQ_FREEZABLE,
+ WQ_UNBOUND_MAX_ACTIVE);
+ BUG_ON(!sgx_epc_cg_wq);
+
+ return sgx_epc_cgroup_alloc(misc_cg_root());
+}
+subsys_initcall(sgx_epc_cgroup_init);
diff --git a/arch/x86/kernel/cpu/sgx/epc_cgroup.h b/arch/x86/kernel/cpu/sgx/epc_cgroup.h
new file mode 100644
index 000000000000..bc358934dbe2
--- /dev/null
+++ b/arch/x86/kernel/cpu/sgx/epc_cgroup.h
@@ -0,0 +1,59 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2022 Intel Corporation. */
+#ifndef _INTEL_SGX_EPC_CGROUP_H_
+#define _INTEL_SGX_EPC_CGROUP_H_
+
+#include <asm/sgx.h>
+#include <linux/cgroup.h>
+#include <linux/list.h>
+#include <linux/misc_cgroup.h>
+#include <linux/page_counter.h>
+#include <linux/workqueue.h>
+
+#include "sgx.h"
+
+#ifndef CONFIG_CGROUP_SGX_EPC
+#define MISC_CG_RES_SGX_EPC MISC_CG_RES_TYPES
+struct sgx_epc_cgroup;
+
+static inline struct sgx_epc_cgroup *sgx_epc_cgroup_try_charge(struct mm_struct *mm,
+ bool reclaim)
+{
+ return NULL;
+}
+static inline void sgx_epc_cgroup_uncharge(struct sgx_epc_cgroup *epc_cg) { }
+static inline void sgx_epc_cgroup_isolate_pages(struct sgx_epc_cgroup *root,
+ int *nr_to_scan,
+ struct list_head *dst) { }
+static inline struct sgx_epc_lru_lists *epc_cg_lru(struct sgx_epc_cgroup *epc_cg)
+{
+ return NULL;
+}
+static bool sgx_epc_cgroup_lru_empty(struct sgx_epc_cgroup *root)
+{
+ return true;
+}
+#else
+struct sgx_epc_cgroup {
+ struct misc_cg *cg;
+ struct sgx_epc_lru_lists lru;
+ struct sgx_epc_cgroup *reclaim_iter;
+ struct work_struct reclaim_work;
+ unsigned int epoch;
+};
+
+struct sgx_epc_cgroup *sgx_epc_cgroup_try_charge(struct mm_struct *mm,
+ bool reclaim);
+void sgx_epc_cgroup_uncharge(struct sgx_epc_cgroup *epc_cg);
+bool sgx_epc_cgroup_lru_empty(struct sgx_epc_cgroup *root);
+void sgx_epc_cgroup_isolate_pages(struct sgx_epc_cgroup *root,
+ int *nr_to_scan, struct list_head *dst);
+static inline struct sgx_epc_lru_lists *epc_cg_lru(struct sgx_epc_cgroup *epc_cg)
+{
+ if (epc_cg)
+ return &epc_cg->lru;
+ return NULL;
+}
+#endif
+
+#endif /* _INTEL_SGX_EPC_CGROUP_H_ */
diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 70046c4e332a..a9d5cfd4e024 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -6,6 +6,7 @@
#include <linux/highmem.h>
#include <linux/kthread.h>
#include <linux/miscdevice.h>
+#include <linux/misc_cgroup.h>
#include <linux/node.h>
#include <linux/pagemap.h>
#include <linux/ratelimit.h>
@@ -17,6 +18,7 @@
#include "driver.h"
#include "encl.h"
#include "encls.h"
+#include "epc_cgroup.h"

#define SGX_MAX_NR_TO_RECLAIM 32

@@ -33,9 +35,20 @@ static DEFINE_XARRAY(sgx_epc_address_space);
static struct sgx_epc_lru_lists sgx_global_lru;
static inline struct sgx_epc_lru_lists *sgx_lru_lists(struct sgx_epc_page *epc_page)
{
+ if (IS_ENABLED(CONFIG_CGROUP_SGX_EPC))
+ return epc_cg_lru(epc_page->epc_cg);
+
return &sgx_global_lru;
}

+static inline bool sgx_can_reclaim(void)
+{
+ if (!IS_ENABLED(CONFIG_CGROUP_SGX_EPC))
+ return !list_empty(&sgx_global_lru.reclaimable);
+
+ return !sgx_epc_cgroup_lru_empty(NULL);
+}
+
static atomic_long_t sgx_nr_free_pages = ATOMIC_LONG_INIT(0);

/* Nodes with one or more EPC sections. */
@@ -320,9 +333,10 @@ void sgx_isolate_epc_pages(struct sgx_epc_lru_lists *lru, int *nr_to_scan,
}

/**
- * sgx_reclaim_epc_pages() - Reclaim EPC pages from the consumers
+ * __sgx_reclaim_epc_pages() - Reclaim EPC pages from the consumers
* @nr_to_scan: Number of EPC pages to scan for reclaim
* @ignore_age: Reclaim a page even if it is young
+ * @epc_cg: EPC cgroup from which to reclaim
*
* Take a fixed number of pages from the head of the active page pool and
* reclaim them to the enclave's private shmem files. Skip the pages, which have
@@ -336,7 +350,8 @@ void sgx_isolate_epc_pages(struct sgx_epc_lru_lists *lru, int *nr_to_scan,
* problematic as it would increase the lock contention too much, which would
* halt forward progress.
*/
-static int __sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
+static int __sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age,
+ struct sgx_epc_cgroup *epc_cg)
{
struct sgx_backing backing[SGX_MAX_NR_TO_RECLAIM];
struct sgx_epc_page *epc_page, *tmp;
@@ -347,7 +362,15 @@ static int __sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
int i = 0;
int ret;

- sgx_isolate_epc_pages(&sgx_global_lru, &nr_to_scan, &iso);
+ /*
+ * If a specific cgroup is not being targetted, take from the global
+ * list first, even when cgroups are enabled. If there are
+ * pages on the global LRU then they should get reclaimed asap.
+ */
+ if (!IS_ENABLED(CONFIG_CGROUP_SGX_EPC) || !epc_cg)
+ sgx_isolate_epc_pages(&sgx_global_lru, &nr_to_scan, &iso);
+
+ sgx_epc_cgroup_isolate_pages(epc_cg, &nr_to_scan, &iso);

if (list_empty(&iso))
return 0;
@@ -397,25 +420,33 @@ static int __sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
SGX_EPC_PAGE_ENCLAVE |
SGX_EPC_PAGE_VERSION_ARRAY);

+ if (epc_page->epc_cg) {
+ sgx_epc_cgroup_uncharge(epc_page->epc_cg);
+ epc_page->epc_cg = NULL;
+ }
+
sgx_free_epc_page(epc_page);
}
return i;
}

-int sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age)
+/**
+ * sgx_reclaim_epc_pages() - wrapper for __sgx_reclaim_epc_pages which
+ * calls cond_resched() upon completion.
+ * @nr_to_scan: Number of EPC pages to scan for reclaim
+ * @ignore_age: Reclaim a page even if it is young
+ * @epc_cg: EPC cgroup from which to reclaim
+ */
+int sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age,
+ struct sgx_epc_cgroup *epc_cg)
{
int ret;

- ret = __sgx_reclaim_pages(nr_to_scan, ignore_age);
+ ret = __sgx_reclaim_epc_pages(nr_to_scan, ignore_age, epc_cg);
cond_resched();
return ret;
}

-static bool sgx_can_reclaim(void)
-{
- return !list_empty(&sgx_global_lru.reclaimable);
-}
-
static bool sgx_should_reclaim(unsigned long watermark)
{
return atomic_long_read(&sgx_nr_free_pages) < watermark &&
@@ -432,7 +463,7 @@ static bool sgx_should_reclaim(unsigned long watermark)
void sgx_reclaim_direct(void)
{
if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
- __sgx_reclaim_pages(SGX_NR_TO_SCAN, false);
+ __sgx_reclaim_epc_pages(SGX_NR_TO_SCAN, false, NULL);
}

static int ksgxd(void *p)
@@ -458,7 +489,7 @@ static int ksgxd(void *p)
sgx_should_reclaim(SGX_NR_HIGH_PAGES));

if (sgx_should_reclaim(SGX_NR_HIGH_PAGES))
- sgx_reclaim_epc_pages(SGX_NR_TO_SCAN, false);
+ sgx_reclaim_epc_pages(SGX_NR_TO_SCAN, false, NULL);
}

return 0;
@@ -620,6 +651,11 @@ int sgx_drop_epc_page(struct sgx_epc_page *page)
struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
{
struct sgx_epc_page *page;
+ struct sgx_epc_cgroup *epc_cg;
+
+ epc_cg = sgx_epc_cgroup_try_charge(current->mm, reclaim);
+ if (IS_ERR(epc_cg))
+ return ERR_CAST(epc_cg);

for ( ; ; ) {
page = __sgx_alloc_epc_page();
@@ -628,8 +664,10 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
break;
}

- if (!sgx_can_reclaim())
- return ERR_PTR(-ENOMEM);
+ if (!sgx_can_reclaim()) {
+ page = ERR_PTR(-ENOMEM);
+ break;
+ }

if (!reclaim) {
page = ERR_PTR(-EBUSY);
@@ -641,7 +679,14 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
break;
}

- sgx_reclaim_epc_pages(SGX_NR_TO_SCAN, false);
+ sgx_reclaim_epc_pages(SGX_NR_TO_SCAN, false, NULL);
+ }
+
+ if (!IS_ERR(page)) {
+ WARN_ON(page->epc_cg);
+ page->epc_cg = epc_cg;
+ } else {
+ sgx_epc_cgroup_uncharge(epc_cg);
}

if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
@@ -674,6 +719,12 @@ void sgx_free_epc_page(struct sgx_epc_page *page)
page->flags = SGX_EPC_PAGE_IS_FREE;

spin_unlock(&node->lock);
+
+ if (page->epc_cg) {
+ sgx_epc_cgroup_uncharge(page->epc_cg);
+ page->epc_cg = NULL;
+ }
+
atomic_long_inc(&sgx_nr_free_pages);
}

@@ -838,6 +889,7 @@ static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size,
section->pages[i].flags = 0;
section->pages[i].encl_owner = NULL;
section->pages[i].poison = 0;
+ section->pages[i].epc_cg = NULL;
list_add_tail(&section->pages[i].list, &sgx_dirty_page_list);
}

@@ -1002,6 +1054,7 @@ static void __init arch_update_sysfs_visibility(int nid) {}
static bool __init sgx_page_cache_init(void)
{
u32 eax, ebx, ecx, edx, type;
+ u64 capacity = 0;
u64 pa, size;
int nid;
int i;
@@ -1052,6 +1105,7 @@ static bool __init sgx_page_cache_init(void)

sgx_epc_sections[i].node = &sgx_numa_nodes[nid];
sgx_numa_nodes[nid].size += size;
+ capacity += size;

sgx_nr_epc_sections++;
}
@@ -1061,6 +1115,8 @@ static bool __init sgx_page_cache_init(void)
return false;
}

+ misc_cg_set_capacity(MISC_CG_RES_SGX_EPC, capacity);
+
return true;
}

diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
index 1c666b25294b..defb48f51145 100644
--- a/arch/x86/kernel/cpu/sgx/sgx.h
+++ b/arch/x86/kernel/cpu/sgx/sgx.h
@@ -35,6 +35,8 @@
#define SGX_EPC_PAGE_ENCLAVE BIT(4)
#define SGX_EPC_PAGE_VERSION_ARRAY BIT(5)

+struct sgx_epc_cgroup;
+
struct sgx_epc_page {
unsigned int section;
u16 flags;
@@ -46,6 +48,7 @@ struct sgx_epc_page {
struct sgx_encl *encl;
};
struct list_head list;
+ struct sgx_epc_cgroup *epc_cg;
};

/*
@@ -206,7 +209,8 @@ void sgx_reclaim_direct(void);
void sgx_record_epc_page(struct sgx_epc_page *page, unsigned long flags);
int sgx_drop_epc_page(struct sgx_epc_page *page);
struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim);
-int sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age);
+int sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age,
+ struct sgx_epc_cgroup *epc_cg);
void sgx_isolate_epc_pages(struct sgx_epc_lru_lists *lrus, int *nr_to_scan,
struct list_head *dst);
bool sgx_epc_oom(struct sgx_epc_lru_lists *lrus);
--
2.38.1

2022-12-02 19:52:00

[permalink] [raw]

Subject: [PATCH v2 12/18] x86/sgx: Expose sgx_reclaim_pages() for use by EPC cgroup

From: Sean Christopherson <[email protected]>

Expose the top-level reclaim function as sgx_reclaim_epc_pages() for use
by the upcoming EPC cgroup, which will initiate reclaim to enforce
changes to high/max limits.

Signed-off-by: Sean Christopherson <[email protected]>
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Cc: Sean Christopherson <[email protected]>
---
arch/x86/kernel/cpu/sgx/main.c | 7 ++++---
arch/x86/kernel/cpu/sgx/sgx.h | 1 +
2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 96399e2016a8..c947b4ae06f3 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -281,6 +281,7 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,

/**
* sgx_reclaim_pages() - Reclaim EPC pages from the consumers
+ * sgx_reclaim_epc_pages() - Reclaim EPC pages from the consumers
* @nr_to_scan: Number of EPC pages to scan for reclaim
* @ignore_age: Reclaim a page even if it is young
*
@@ -385,7 +386,7 @@ static int __sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
return i;
}

-static int sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
+int sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age)
{
int ret;

@@ -441,7 +442,7 @@ static int ksgxd(void *p)
sgx_should_reclaim(SGX_NR_HIGH_PAGES));

if (sgx_should_reclaim(SGX_NR_HIGH_PAGES))
- sgx_reclaim_pages(SGX_NR_TO_SCAN, false);
+ sgx_reclaim_epc_pages(SGX_NR_TO_SCAN, false);
}

return 0;
@@ -624,7 +625,7 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
break;
}

- sgx_reclaim_pages(SGX_NR_TO_SCAN, false);
+ sgx_reclaim_epc_pages(SGX_NR_TO_SCAN, false);
}

if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
index ec8d567cd975..ce859331ddf5 100644
--- a/arch/x86/kernel/cpu/sgx/sgx.h
+++ b/arch/x86/kernel/cpu/sgx/sgx.h
@@ -206,6 +206,7 @@ void sgx_reclaim_direct(void);
void sgx_record_epc_page(struct sgx_epc_page *page, unsigned long flags);
int sgx_drop_epc_page(struct sgx_epc_page *page);
struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim);
+int sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age);

void sgx_ipi_cb(void *info);

--
2.38.1

2022-12-02 19:52:53

[permalink] [raw]

Subject: [PATCH v2 03/18] x86/sgx: Add 'struct sgx_epc_lru_lists' to encapsulate lru list(s)

Introduce a data structure to wrap the existing reclaimable list
and its spinlock in a struct to minimize the code changes needed
to handle multiple LRUs as well as reclaimable and non-reclaimable
lists, both of which will be introduced and used by SGX EPC cgroups.

Signed-off-by: Sean Christopherson <[email protected]>
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Cc: Sean Christopherson <[email protected]>
---
arch/x86/kernel/cpu/sgx/sgx.h | 65 +++++++++++++++++++++++++++++++++++
1 file changed, 65 insertions(+)

diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
index 39cb15a8abcb..5e6d88438fae 100644
--- a/arch/x86/kernel/cpu/sgx/sgx.h
+++ b/arch/x86/kernel/cpu/sgx/sgx.h
@@ -90,6 +90,71 @@ static inline void *sgx_get_epc_virt_addr(struct sgx_epc_page *page)
return section->virt_addr + index * PAGE_SIZE;
}

+/*
+ * This data structure wraps a list of reclaimable EPC pages, and a list of
+ * non-reclaimable EPC pages and is used to implement a LRU policy during
+ * reclamation.
+ */
+struct sgx_epc_lru_lists {
+ spinlock_t lock;
+ struct list_head reclaimable;
+ struct list_head unreclaimable;
+};
+
+static inline void sgx_lru_init(struct sgx_epc_lru_lists *lrus)
+{
+ spin_lock_init(&lrus->lock);
+ INIT_LIST_HEAD(&lrus->reclaimable);
+ INIT_LIST_HEAD(&lrus->unreclaimable);
+}
+
+/*
+ * Must be called with queue lock acquired
+ */
+static inline void __sgx_epc_page_list_push(struct list_head *list, struct sgx_epc_page *page)
+{
+ list_add_tail(&page->list, list);
+}
+
+/*
+ * Must be called with queue lock acquired
+ */
+static inline struct sgx_epc_page * __sgx_epc_page_list_pop(struct list_head *list)
+{
+ struct sgx_epc_page *epc_page;
+
+ if (list_empty(list))
+ return NULL;
+
+ epc_page = list_first_entry(list, struct sgx_epc_page, list);
+ list_del_init(&epc_page->list);
+ return epc_page;
+}
+
+static inline struct sgx_epc_page *
+sgx_epc_pop_reclaimable(struct sgx_epc_lru_lists *lrus)
+{
+ return __sgx_epc_page_list_pop(&(lrus)->reclaimable);
+}
+
+static inline void sgx_epc_push_reclaimable(struct sgx_epc_lru_lists *lrus,
+ struct sgx_epc_page *page)
+{
+ __sgx_epc_page_list_push(&(lrus)->reclaimable, page);
+}
+
+static inline struct sgx_epc_page *
+sgx_epc_pop_unreclaimable(struct sgx_epc_lru_lists *lrus)
+{
+ return __sgx_epc_page_list_pop(&(lrus)->unreclaimable);
+}
+
+static inline void sgx_epc_push_unreclaimable(struct sgx_epc_lru_lists *lrus,
+ struct sgx_epc_page *page)
+{
+ __sgx_epc_page_list_push(&(lrus)->unreclaimable, page);
+}
+
struct sgx_epc_page *__sgx_alloc_epc_page(void);
void sgx_free_epc_page(struct sgx_epc_page *page);

--
2.38.1

2022-12-02 20:06:57

[permalink] [raw]

Subject: [PATCH v2 01/18] x86/sgx: Call cond_resched() at the end of sgx_reclaim_pages()

From: Sean Christopherson <[email protected]>

In order to avoid repetition of cond_resched() in ksgxd() and
sgx_alloc_epc_page(), move the invocation of post-reclaim cond_resched()
inside sgx_reclaim_pages(). Except in the case of sgx_reclaim_direct(),
sgx_reclaim_pages() is always called in a loop and is always followed
by a call to cond_resched(). This will hold true for the EPC cgroup
as well, which adds even more calls to sgx_reclaim_pages() and thus
cond_resched(). Calls to sgx_reclaim_direct() may be performance
sensitive. Allow sgx_reclaim_direct() to avoid the cond_resched()
call by moving the original sgx_reclaim_pages() call to
__sgx_reclaim_pages() and then have sgx_reclaim_pages() become a
wrapper around that call with a cond_resched().

Signed-off-by: Sean Christopherson <[email protected]>
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Cc: Sean Christopherson <[email protected]>
---
arch/x86/kernel/cpu/sgx/main.c | 17 +++++++++++------
1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index 160c8dbee0ab..ffce6fc70a1f 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -287,7 +287,7 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
* problematic as it would increase the lock contention too much, which would
* halt forward progress.
*/
-static void sgx_reclaim_pages(void)
+static void __sgx_reclaim_pages(void)
{
struct sgx_epc_page *chunk[SGX_NR_TO_SCAN];
struct sgx_backing backing[SGX_NR_TO_SCAN];
@@ -369,6 +369,12 @@ static void sgx_reclaim_pages(void)
}
}

+static void sgx_reclaim_pages(void)
+{
+ __sgx_reclaim_pages();
+ cond_resched();
+}
+
static bool sgx_should_reclaim(unsigned long watermark)
{
return atomic_long_read(&sgx_nr_free_pages) < watermark &&
@@ -378,12 +384,14 @@ static bool sgx_should_reclaim(unsigned long watermark)
/*
* sgx_reclaim_direct() should be called (without enclave's mutex held)
* in locations where SGX memory resources might be low and might be
- * needed in order to make forward progress.
+ * needed in order to make forward progress. This call to
+ * __sgx_reclaim_pages() avoids the cond_resched() in sgx_reclaim_pages()
+ * to improve performance.
*/
void sgx_reclaim_direct(void)
{
if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
- sgx_reclaim_pages();
+ __sgx_reclaim_pages();
}

static int ksgxd(void *p)
@@ -410,8 +418,6 @@ static int ksgxd(void *p)

if (sgx_should_reclaim(SGX_NR_HIGH_PAGES))
sgx_reclaim_pages();
-
- cond_resched();
}

return 0;
@@ -582,7 +588,6 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
}

sgx_reclaim_pages();
- cond_resched();
}

if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
--
2.38.1

2022-12-02 20:47:43

[permalink] [raw]

Subject: [PATCH v2 14/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

From: Sean Christopherson <[email protected]>

Introduce the OOM path for killing an enclave with the reclaimer
is no longer able to reclaim enough EPC pages. Find a victim enclave,
which will be an enclave with EPC pages remaining that are not
accessible to the reclaimer ("unreclaimable"). Once a victim is
identified, mark the enclave as OOM and zap the enclaves entire
page range. Release all the enclaves resources except for the
struct sgx_encl memory itself.

Signed-off-by: Sean Christopherson <[email protected]>
Signed-off-by: Kristen Carlson Accardi <[email protected]>
Cc: Sean Christopherson <[email protected]>
---
arch/x86/kernel/cpu/sgx/encl.c | 74 +++++++++++++++---
arch/x86/kernel/cpu/sgx/encl.h | 2 +
arch/x86/kernel/cpu/sgx/main.c | 135 +++++++++++++++++++++++++++++++++
arch/x86/kernel/cpu/sgx/sgx.h | 1 +
4 files changed, 201 insertions(+), 11 deletions(-)

diff --git a/arch/x86/kernel/cpu/sgx/encl.c b/arch/x86/kernel/cpu/sgx/encl.c
index 9ee306ac2a8e..ba350b2961d1 100644
--- a/arch/x86/kernel/cpu/sgx/encl.c
+++ b/arch/x86/kernel/cpu/sgx/encl.c
@@ -623,7 +623,8 @@ static int sgx_vma_access(struct vm_area_struct *vma, unsigned long addr,
if (!encl)
return -EFAULT;

- if (!test_bit(SGX_ENCL_DEBUG, &encl->flags))
+ if (!test_bit(SGX_ENCL_DEBUG, &encl->flags) ||
+ test_bit(SGX_ENCL_OOM, &encl->flags))
return -EFAULT;

for (i = 0; i < len; i += cnt) {
@@ -669,16 +670,8 @@ const struct vm_operations_struct sgx_vm_ops = {
.access = sgx_vma_access,
};

-/**
- * sgx_encl_release - Destroy an enclave instance
- * @ref: address of a kref inside &sgx_encl
- *
- * Used together with kref_put(). Frees all the resources associated with the
- * enclave and the instance itself.
- */
-void sgx_encl_release(struct kref *ref)
+static void __sgx_encl_release(struct sgx_encl *encl)
{
- struct sgx_encl *encl = container_of(ref, struct sgx_encl, refcount);
struct sgx_va_page *va_page;
struct sgx_encl_page *entry;
unsigned long index;
@@ -713,7 +706,7 @@ void sgx_encl_release(struct kref *ref)
while (!list_empty(&encl->va_pages)) {
va_page = list_first_entry(&encl->va_pages, struct sgx_va_page,
list);
- list_del(&va_page->list);
+ list_del_init(&va_page->list);
sgx_drop_epc_page(va_page->epc_page);
sgx_encl_free_epc_page(va_page->epc_page);
kfree(va_page);
@@ -729,10 +722,66 @@ void sgx_encl_release(struct kref *ref)
/* Detect EPC page leak's. */
WARN_ON_ONCE(encl->secs_child_cnt);
WARN_ON_ONCE(encl->secs.epc_page);
+}
+
+/**
+ * sgx_encl_release - Destroy an enclave instance
+ * @ref: address of a kref inside &sgx_encl
+ *
+ * Used together with kref_put(). Frees all the resources associated with the
+ * enclave and the instance itself.
+ */
+void sgx_encl_release(struct kref *ref)
+{
+ struct sgx_encl *encl = container_of(ref, struct sgx_encl, refcount);
+
+ /* if the enclave was OOM killed previously, it just needs to be freed */
+ if (!test_bit(SGX_ENCL_OOM, &encl->flags))
+ __sgx_encl_release(encl);

kfree(encl);
}

+/**
+ * sgx_encl_destroy - prepare the enclave for release
+ * @encl: address of the sgx_encl to drain
+ *
+ * Used during oom kill to empty the mm_list entries after they have
+ * been zapped. Release the remaining enclave resources without freeing
+ * struct sgx_encl.
+ */
+void sgx_encl_destroy(struct sgx_encl *encl)
+{
+ struct sgx_encl_mm *encl_mm;
+
+ for ( ; ; ) {
+ spin_lock(&encl->mm_lock);
+
+ if (list_empty(&encl->mm_list)) {
+ encl_mm = NULL;
+ } else {
+ encl_mm = list_first_entry(&encl->mm_list,
+ struct sgx_encl_mm, list);
+ list_del_rcu(&encl_mm->list);
+ }
+
+ spin_unlock(&encl->mm_lock);
+
+ /* The enclave is no longer mapped by any mm. */
+ if (!encl_mm)
+ break;
+
+ synchronize_srcu(&encl->srcu);
+ mmu_notifier_unregister(&encl_mm->mmu_notifier, encl_mm->mm);
+ kfree(encl_mm);
+
+ /* 'encl_mm' is gone, put encl_mm->encl reference: */
+ kref_put(&encl->refcount, sgx_encl_release);
+ }
+
+ __sgx_encl_release(encl);
+}
+
/*
* 'mm' is exiting and no longer needs mmu notifications.
*/
@@ -802,6 +851,9 @@ int sgx_encl_mm_add(struct sgx_encl *encl, struct mm_struct *mm)
struct sgx_encl_mm *encl_mm;
int ret;

+ if (test_bit(SGX_ENCL_OOM, &encl->flags))
+ return -ENOMEM;
+
/*
* Even though a single enclave may be mapped into an mm more than once,
* each 'mm' only appears once on encl->mm_list. This is guaranteed by
diff --git a/arch/x86/kernel/cpu/sgx/encl.h b/arch/x86/kernel/cpu/sgx/encl.h
index 831d63f80f5a..f4935632e53a 100644
--- a/arch/x86/kernel/cpu/sgx/encl.h
+++ b/arch/x86/kernel/cpu/sgx/encl.h
@@ -39,6 +39,7 @@ enum sgx_encl_flags {
SGX_ENCL_DEBUG = BIT(1),
SGX_ENCL_CREATED = BIT(2),
SGX_ENCL_INITIALIZED = BIT(3),
+ SGX_ENCL_OOM = BIT(4),
};

struct sgx_encl_mm {
@@ -125,5 +126,6 @@ struct sgx_encl_page *sgx_encl_load_page(struct sgx_encl *encl,
unsigned long addr);
struct sgx_va_page *sgx_encl_grow(struct sgx_encl *encl, bool reclaim);
void sgx_encl_shrink(struct sgx_encl *encl, struct sgx_va_page *va_page);
+void sgx_encl_destroy(struct sgx_encl *encl);

#endif /* _X86_ENCL_H */
diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
index a59550fa150b..70046c4e332a 100644
--- a/arch/x86/kernel/cpu/sgx/main.c
+++ b/arch/x86/kernel/cpu/sgx/main.c
@@ -677,6 +677,141 @@ void sgx_free_epc_page(struct sgx_epc_page *page)
atomic_long_inc(&sgx_nr_free_pages);
}

+static bool sgx_oom_get_ref(struct sgx_epc_page *epc_page)
+{
+ struct sgx_encl *encl;
+
+ if (epc_page->flags & SGX_EPC_PAGE_ENCLAVE)
+ encl = ((struct sgx_encl_page *)epc_page->encl_owner)->encl;
+ else if (epc_page->flags & SGX_EPC_PAGE_VERSION_ARRAY)
+ encl = epc_page->encl;
+ else
+ return false;
+
+ return kref_get_unless_zero(&encl->refcount);
+}
+
+static struct sgx_epc_page *sgx_oom_get_victim(struct sgx_epc_lru_lists *lru)
+{
+ struct sgx_epc_page *epc_page, *tmp;
+
+ if (list_empty(&lru->unreclaimable))
+ return NULL;
+
+ list_for_each_entry_safe(epc_page, tmp, &lru->unreclaimable, list) {
+ list_del_init(&epc_page->list);
+
+ if (sgx_oom_get_ref(epc_page))
+ return epc_page;
+ }
+ return NULL;
+}
+
+static void sgx_epc_oom_zap(void *owner, struct mm_struct *mm, unsigned long start,
+ unsigned long end, const struct vm_operations_struct *ops)
+{
+ struct vm_area_struct *vma, *tmp;
+ unsigned long vm_end;
+
+ vma = find_vma(mm, start);
+ if (!vma || vma->vm_ops != ops || vma->vm_private_data != owner ||
+ vma->vm_start >= end)
+ return;
+
+ for (tmp = vma; tmp->vm_start < end; tmp = tmp->vm_next) {
+ do {
+ vm_end = tmp->vm_end;
+ tmp = tmp->vm_next;
+ } while (tmp && tmp->vm_ops == ops &&
+ vma->vm_private_data == owner && tmp->vm_start < end);
+
+ zap_page_range(vma, vma->vm_start, vm_end - vma->vm_start);
+
+ if (!tmp)
+ break;
+ }
+}
+
+static void sgx_oom_encl(struct sgx_encl *encl)
+{
+ unsigned long mm_list_version;
+ struct sgx_encl_mm *encl_mm;
+ int idx;
+
+ set_bit(SGX_ENCL_OOM, &encl->flags);
+
+ if (!test_bit(SGX_ENCL_CREATED, &encl->flags))
+ goto out;
+
+ do {
+ mm_list_version = encl->mm_list_version;
+
+ /* Pairs with smp_rmb() in sgx_encl_mm_add(). */
+ smp_rmb();
+
+ idx = srcu_read_lock(&encl->srcu);
+
+ list_for_each_entry_rcu(encl_mm, &encl->mm_list, list) {
+ if (!mmget_not_zero(encl_mm->mm))
+ continue;
+
+ mmap_read_lock(encl_mm->mm);
+
+ sgx_epc_oom_zap(encl, encl_mm->mm, encl->base,
+ encl->base + encl->size, &sgx_vm_ops);
+
+ mmap_read_unlock(encl_mm->mm);
+
+ mmput_async(encl_mm->mm);
+ }
+
+ srcu_read_unlock(&encl->srcu, idx);
+ } while (WARN_ON_ONCE(encl->mm_list_version != mm_list_version));
+
+ mutex_lock(&encl->lock);
+ sgx_encl_destroy(encl);
+ mutex_unlock(&encl->lock);
+
+out:
+ /*
+ * This puts the refcount we took when we identified this enclave as
+ * an OOM victim.
+ */
+ kref_put(&encl->refcount, sgx_encl_release);
+}
+
+static inline void sgx_oom_encl_page(struct sgx_encl_page *encl_page)
+{
+ return sgx_oom_encl(encl_page->encl);
+}
+
+/**
+ * sgx_epc_oom() - invoke EPC out-of-memory handling on target LRU
+ * @lru: LRU that is low
+ *
+ * Return: %true if a victim was found and kicked.
+ */
+bool sgx_epc_oom(struct sgx_epc_lru_lists *lru)
+{
+ struct sgx_epc_page *victim;
+
+ spin_lock(&lru->lock);
+ victim = sgx_oom_get_victim(lru);
+ spin_unlock(&lru->lock);
+
+ if (!victim)
+ return false;
+
+ if (victim->flags & SGX_EPC_PAGE_ENCLAVE)
+ sgx_oom_encl_page(victim->encl_owner);
+ else if (victim->flags & SGX_EPC_PAGE_VERSION_ARRAY)
+ sgx_oom_encl(victim->encl);
+ else
+ WARN_ON_ONCE(1);
+
+ return true;
+}
+
static bool __init sgx_setup_epc_section(u64 phys_addr, u64 size,
unsigned long index,
struct sgx_epc_section *section)
diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
index 4499a5d5547d..1c666b25294b 100644
--- a/arch/x86/kernel/cpu/sgx/sgx.h
+++ b/arch/x86/kernel/cpu/sgx/sgx.h
@@ -209,6 +209,7 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim);
int sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age);
void sgx_isolate_epc_pages(struct sgx_epc_lru_lists *lrus, int *nr_to_scan,
struct list_head *dst);
+bool sgx_epc_oom(struct sgx_epc_lru_lists *lrus);

void sgx_ipi_cb(void *info);

--
2.38.1

2022-12-02 21:48:59

[permalink] [raw]

Subject: Re: [PATCH v2 02/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On 12/2/22 10:36, Kristen Carlson Accardi wrote:
> When allocating new Version Array (VA) pages, pass the struct sgx_encl
> of the enclave that is allocating the page. sgx_alloc_epc_page() will
> store this value in the encl_owner field of the struct sgx_epc_page. In
> a later patch, VA pages will be placed in an unreclaimable queue,
> and then when the cgroup max limit is reached and there are no more
> reclaimable pages and the enclave must be oom killed, all the
> VA pages associated with that enclave can be uncharged and freed.

What does this have to do with the 'encl' that is being passed, though?

In other words, why is this new sgx_epc_page-to-encl mapping needed for
VA pages now, but it wasn't before?

2022-12-02 21:49:21

[permalink] [raw]

Subject: Re: [PATCH v2 01/18] x86/sgx: Call cond_resched() at the end of sgx_reclaim_pages()

On 12/2/22 10:36, Kristen Carlson Accardi wrote:
> From: Sean Christopherson <[email protected]>
>
> In order to avoid repetition of cond_resched() in ksgxd() and
> sgx_alloc_epc_page(), move the invocation of post-reclaim cond_resched()
> inside sgx_reclaim_pages(). Except in the case of sgx_reclaim_direct(),
> sgx_reclaim_pages() is always called in a loop and is always followed
> by a call to cond_resched(). This will hold true for the EPC cgroup
> as well, which adds even more calls to sgx_reclaim_pages() and thus
> cond_resched(). Calls to sgx_reclaim_direct() may be performance
> sensitive. Allow sgx_reclaim_direct() to avoid the cond_resched()
> call by moving the original sgx_reclaim_pages() call to
> __sgx_reclaim_pages() and then have sgx_reclaim_pages() become a
> wrapper around that call with a cond_resched().
>
> Signed-off-by: Sean Christopherson <[email protected]>
> Signed-off-by: Kristen Carlson Accardi <[email protected]>
> Cc: Sean Christopherson <[email protected]>
> ---
> arch/x86/kernel/cpu/sgx/main.c | 17 +++++++++++------
> 1 file changed, 11 insertions(+), 6 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> index 160c8dbee0ab..ffce6fc70a1f 100644
> --- a/arch/x86/kernel/cpu/sgx/main.c
> +++ b/arch/x86/kernel/cpu/sgx/main.c
> @@ -287,7 +287,7 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
> * problematic as it would increase the lock contention too much, which would
> * halt forward progress.
> */
> -static void sgx_reclaim_pages(void)
> +static void __sgx_reclaim_pages(void)
> {
> struct sgx_epc_page *chunk[SGX_NR_TO_SCAN];
> struct sgx_backing backing[SGX_NR_TO_SCAN];
> @@ -369,6 +369,12 @@ static void sgx_reclaim_pages(void)
> }
> }
>
> +static void sgx_reclaim_pages(void)
> +{
> + __sgx_reclaim_pages();
> + cond_resched();
> +}

Why bother with the wrapper? Can't we just put cond_resched() in the
existing sgx_reclaim_pages()?

2022-12-02 21:50:34

[permalink] [raw]

Subject: Re: [PATCH v2 02/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On Fri, 2022-12-02 at 13:35 -0800, Dave Hansen wrote:
> On 12/2/22 10:36, Kristen Carlson Accardi wrote:
> > When allocating new Version Array (VA) pages, pass the struct
> > sgx_encl
> > of the enclave that is allocating the page. sgx_alloc_epc_page()
> > will
> > store this value in the encl_owner field of the struct
> > sgx_epc_page. In
> > a later patch, VA pages will be placed in an unreclaimable queue,
> > and then when the cgroup max limit is reached and there are no more
> > reclaimable pages and the enclave must be oom killed, all the
> > VA pages associated with that enclave can be uncharged and freed.
>
> What does this have to do with the 'encl' that is being passed,
> though?
>
> In other words, why is this new sgx_epc_page-to-encl mapping needed
> for
> VA pages now, but it wasn't before?

When we OOM kill an enclave, we want to get rid of all the associated
VA pages too. Prior to this patch, there wasn't a way to easily get the
VA pages associated with an enclave.

2022-12-02 21:54:04

[permalink] [raw]

Subject: Re: [PATCH v2 01/18] x86/sgx: Call cond_resched() at the end of sgx_reclaim_pages()

On 12/2/22 13:37, Kristen Carlson Accardi wrote:
>>> +static void sgx_reclaim_pages(void)
>>> +{
>>> + __sgx_reclaim_pages();
>>> + cond_resched();
>>> +}
>> Why bother with the wrapper? Can't we just put cond_resched() in the
>> existing sgx_reclaim_pages()?
> Because sgx_reclaim_direct() needs to call sgx_reclaim_pages() but not
> do the cond_resched(). It was this or add a boolean or something to let
> caller's opt out of the resched.

Is there a reason sgx_reclaim_direct() *can't* or shouldn't call
cond_resched()?

2022-12-02 22:02:16

[permalink] [raw]

Subject: Re: [PATCH v2 01/18] x86/sgx: Call cond_resched() at the end of sgx_reclaim_pages()

On Fri, 2022-12-02 at 13:33 -0800, Dave Hansen wrote:
> On 12/2/22 10:36, Kristen Carlson Accardi wrote:
> > From: Sean Christopherson <[email protected]>
> >
> > In order to avoid repetition of cond_resched() in ksgxd() and
> > sgx_alloc_epc_page(), move the invocation of post-reclaim
> > cond_resched()
> > inside sgx_reclaim_pages(). Except in the case of
> > sgx_reclaim_direct(),
> > sgx_reclaim_pages() is always called in a loop and is always
> > followed
> > by a call to cond_resched(). This will hold true for the EPC
> > cgroup
> > as well, which adds even more calls to sgx_reclaim_pages() and thus
> > cond_resched(). Calls to sgx_reclaim_direct() may be performance
> > sensitive. Allow sgx_reclaim_direct() to avoid the cond_resched()
> > call by moving the original sgx_reclaim_pages() call to
> > __sgx_reclaim_pages() and then have sgx_reclaim_pages() become a
> > wrapper around that call with a cond_resched().
> >
> > Signed-off-by: Sean Christopherson
> > <[email protected]>
> > Signed-off-by: Kristen Carlson Accardi <[email protected]>
> > Cc: Sean Christopherson <[email protected]>
> > ---
> > arch/x86/kernel/cpu/sgx/main.c | 17 +++++++++++------
> > 1 file changed, 11 insertions(+), 6 deletions(-)
> >
> > diff --git a/arch/x86/kernel/cpu/sgx/main.c
> > b/arch/x86/kernel/cpu/sgx/main.c
> > index 160c8dbee0ab..ffce6fc70a1f 100644
> > --- a/arch/x86/kernel/cpu/sgx/main.c
> > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > @@ -287,7 +287,7 @@ static void sgx_reclaimer_write(struct
> > sgx_epc_page *epc_page,
> > * problematic as it would increase the lock contention too much,
> > which would
> > * halt forward progress.
> > */
> > -static void sgx_reclaim_pages(void)
> > +static void __sgx_reclaim_pages(void)
> > {
> >         struct sgx_epc_page *chunk[SGX_NR_TO_SCAN];
> >         struct sgx_backing backing[SGX_NR_TO_SCAN];
> > @@ -369,6 +369,12 @@ static void sgx_reclaim_pages(void)
> >         }
> > }
> >
> > +static void sgx_reclaim_pages(void)
> > +{
> > +       __sgx_reclaim_pages();
> > +       cond_resched();
> > +}
>
> Why bother with the wrapper? Can't we just put cond_resched() in the
> existing sgx_reclaim_pages()?

Because sgx_reclaim_direct() needs to call sgx_reclaim_pages() but not
do the cond_resched(). It was this or add a boolean or something to let
caller's opt out of the resched.

2022-12-02 22:25:24

[permalink] [raw]

Subject: Re: [PATCH v2 02/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On 12/2/22 13:40, Kristen Carlson Accardi wrote:
> On Fri, 2022-12-02 at 13:35 -0800, Dave Hansen wrote:
>> On 12/2/22 10:36, Kristen Carlson Accardi wrote:
>>> When allocating new Version Array (VA) pages, pass the struct
>>> sgx_encl
>>> of the enclave that is allocating the page. sgx_alloc_epc_page()
>>> will
>>> store this value in the encl_owner field of the struct
>>> sgx_epc_page. In
>>> a later patch, VA pages will be placed in an unreclaimable queue,
>>> and then when the cgroup max limit is reached and there are no more
>>> reclaimable pages and the enclave must be oom killed, all the
>>> VA pages associated with that enclave can be uncharged and freed.
>> What does this have to do with the 'encl' that is being passed,
>> though?
>>
>> In other words, why is this new sgx_epc_page-to-encl mapping needed
>> for
>> VA pages now, but it wasn't before?
> When we OOM kill an enclave, we want to get rid of all the associated
> VA pages too. Prior to this patch, there wasn't a way to easily get the
> VA pages associated with an enclave.

Given an enclave, we have encl->va_pages to look up all the VA pages.
Also, this patch's code allows you to go from a va page to an enclave.
That seems like it's going the other direction from what an OOM-kill
would need to do.

2022-12-02 22:25:41

[permalink] [raw]

Subject: Re: [PATCH v2 03/18] x86/sgx: Add 'struct sgx_epc_lru_lists' to encapsulate lru list(s)

On 12/2/22 10:36, Kristen Carlson Accardi wrote:
> Introduce a data structure to wrap the existing reclaimable list
> and its spinlock in a struct to minimize the code changes needed
> to handle multiple LRUs as well as reclaimable and non-reclaimable
> lists, both of which will be introduced and used by SGX EPC cgroups.
>
> Signed-off-by: Sean Christopherson <[email protected]>
> Signed-off-by: Kristen Carlson Accardi <[email protected]>
> Cc: Sean Christopherson <[email protected]>

Tiny nits: Let's also allude to the fact that this doesn't do anything
with the new helpers or structures for now.

I also think it's probably a sane idea to mention that the core VM also
has parallel LRU lists for cgroups.

> +static inline struct sgx_epc_page *
> +sgx_epc_pop_reclaimable(struct sgx_epc_lru_lists *lrus)
> +{
> + return __sgx_epc_page_list_pop(&(lrus)->reclaimable);
> +}

Are those '(lrus)' parenthesis doing anything useful?

2022-12-02 22:26:28

[permalink] [raw]

Subject: Re: [PATCH v2 01/18] x86/sgx: Call cond_resched() at the end of sgx_reclaim_pages()

On Fri, 2022-12-02 at 13:45 -0800, Dave Hansen wrote:
> On 12/2/22 13:37, Kristen Carlson Accardi wrote:
> > > > +static void sgx_reclaim_pages(void)
> > > > +{
> > > > + __sgx_reclaim_pages();
> > > > + cond_resched();
> > > > +}
> > > Why bother with the wrapper? Can't we just put cond_resched() in
> > > the
> > > existing sgx_reclaim_pages()?
> > Because sgx_reclaim_direct() needs to call sgx_reclaim_pages() but
> > not
> > do the cond_resched(). It was this or add a boolean or something to
> > let
> > caller's opt out of the resched.
>
> Is there a reason sgx_reclaim_direct() *can't* or shouldn't call
> cond_resched()?

Yes, it is due to performance concerns. It is explained most succinctly
by Reinette here:

https://lore.kernel.org/linux-sgx/[email protected]/

2022-12-02 22:49:38

[permalink] [raw]

Subject: Re: [PATCH v2 01/18] x86/sgx: Call cond_resched() at the end of sgx_reclaim_pages()

On 12/2/22 14:17, Kristen Carlson Accardi wrote:
> On Fri, 2022-12-02 at 13:45 -0800, Dave Hansen wrote:
>> On 12/2/22 13:37, Kristen Carlson Accardi wrote:
>>>>> +static void sgx_reclaim_pages(void)
>>>>> +{
>>>>> + __sgx_reclaim_pages();
>>>>> + cond_resched();
>>>>> +}
>>>> Why bother with the wrapper? Can't we just put cond_resched() in
>>>> the
>>>> existing sgx_reclaim_pages()?
>>> Because sgx_reclaim_direct() needs to call sgx_reclaim_pages()
>>> but not do the cond_resched(). It was this or add a boolean or
>>> something to let caller's opt out of the resched.
>>
>> Is there a reason sgx_reclaim_direct() *can't* or shouldn't call
>> cond_resched()?
>
> Yes, it is due to performance concerns. It is explained most succinctly
> by Reinette here:
>
> https://lore.kernel.org/linux-sgx/[email protected]/

I think I'd much rather have 3 cond_resched()s in the code that
effectively self-document than one __something() in there that's a bit
of a mystery.

Everyone knows what cond_resched() means.

2022-12-02 22:50:52

[permalink] [raw]

Subject: Re: [PATCH v2 02/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On 12/2/22 14:35, Sean Christopherson wrote:
>> That seems like it's going the other direction from what an OOM-kill
>> would need to do.
> Providing a backpointer from a VA page to its enclave allows OOM-killing the enclave
> if its cgroup is over the limit but there are no reclaimable pages for said cgroup
> (for SGX's definition of "reclaimable"). I.e. if all of an enclave's "regular"
> pages have been swapped out, the only thing left resident in the EPC will be the
> enclave's VA pages, which are not reclaimable in the kernel's current SGX
> implementation.

Ooooooooooooooooooooh. I'm a dummy.

So, we've got a cgroup. It's in OOM-kill mode and we're looking at the
*cgroup* LRU lists. We've done everything we can to the enclave and
swapped everything out that we can. All we're left with are these
crummy VA pages on the LRU (or equally crummy pages). We want to
reclaim them but can't swap VA pages. Our only recourse is to go to the
enclave and kill *it*.

Right now, we can easily find an enclave's VA pages and free them. We
do that all the time when freeing whole enclaves. But, what we can't
easily do is find an enclave given a VA page.

A reverse pointer from VA page back to enclave allows the VA page's
enclave to be located and efficiently killed.

Right?

Could we add that context to the changelog, please?

2022-12-02 22:58:03

by Sean Christopherson

[permalink] [raw]

Subject: Re: [PATCH v2 02/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On Fri, Dec 02, 2022, Dave Hansen wrote:
> On 12/2/22 13:40, Kristen Carlson Accardi wrote:
> > On Fri, 2022-12-02 at 13:35 -0800, Dave Hansen wrote:
> >> On 12/2/22 10:36, Kristen Carlson Accardi wrote:
> >>> When allocating new Version Array (VA) pages, pass the struct
> >>> sgx_encl
> >>> of the enclave that is allocating the page. sgx_alloc_epc_page()
> >>> will
> >>> store this value in the encl_owner field of the struct
> >>> sgx_epc_page. In
> >>> a later patch, VA pages will be placed in an unreclaimable queue,
> >>> and then when the cgroup max limit is reached and there are no more
> >>> reclaimable pages and the enclave must be oom killed, all the
> >>> VA pages associated with that enclave can be uncharged and freed.
> >> What does this have to do with the 'encl' that is being passed,
> >> though?
> >>
> >> In other words, why is this new sgx_epc_page-to-encl mapping needed
> >> for
> >> VA pages now, but it wasn't before?
> > When we OOM kill an enclave, we want to get rid of all the associated
> > VA pages too. Prior to this patch, there wasn't a way to easily get the
> > VA pages associated with an enclave.
>
> Given an enclave, we have encl->va_pages to look up all the VA pages.
> Also, this patch's code allows you to go from a va page to an enclave.

Yep.

> That seems like it's going the other direction from what an OOM-kill
> would need to do.

Providing a backpointer from a VA page to its enclave allows OOM-killing the enclave
if its cgroup is over the limit but there are no reclaimable pages for said cgroup
(for SGX's definition of "reclaimable"). I.e. if all of an enclave's "regular"
pages have been swapped out, the only thing left resident in the EPC will be the
enclave's VA pages, which are not reclaimable in the kernel's current SGX
implementation.

2022-12-02 22:58:53

by Sean Christopherson

[permalink] [raw]

Subject: Re: [PATCH v2 02/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On Fri, Dec 02, 2022, Dave Hansen wrote:
> On 12/2/22 14:35, Sean Christopherson wrote:
> >> That seems like it's going the other direction from what an OOM-kill
> >> would need to do.
> > Providing a backpointer from a VA page to its enclave allows OOM-killing the enclave
> > if its cgroup is over the limit but there are no reclaimable pages for said cgroup
> > (for SGX's definition of "reclaimable"). I.e. if all of an enclave's "regular"
> > pages have been swapped out, the only thing left resident in the EPC will be the
> > enclave's VA pages, which are not reclaimable in the kernel's current SGX
> > implementation.
>
> Ooooooooooooooooooooh. I'm a dummy.
>
>
> So, we've got a cgroup. It's in OOM-kill mode and we're looking at the
> *cgroup* LRU lists. We've done everything we can to the enclave and
> swapped everything out that we can. All we're left with are these
> crummy VA pages on the LRU (or equally crummy pages). We want to
> reclaim them but can't swap VA pages. Our only recourse is to go to the
> enclave and kill *it*.
>
> Right now, we can easily find an enclave's VA pages and free them. We
> do that all the time when freeing whole enclaves. But, what we can't
> easily do is find an enclave given a VA page.
>
> A reverse pointer from VA page back to enclave allows the VA page's
> enclave to be located and efficiently killed.
>
> Right?

Yep, exactly.

2022-12-08 09:43:34

[permalink] [raw]

Subject: Re: [PATCH v2 09/18] x86/sgx: Return the number of EPC pages that were successfully reclaimed

On Fri, Dec 02, 2022 at 10:36:45AM -0800, Kristen Carlson Accardi wrote:
> From: Sean Christopherson <[email protected]>
>
> Return the number of reclaimed pages from sgx_reclaim_pages(), the EPC
> cgroup will use the result to track the success rate of its reclaim
> calls, e.g. to escalate to a more forceful reclaiming mode if necessary.
>
> Signed-off-by: Sean Christopherson <[email protected]>
> Signed-off-by: Kristen Carlson Accardi <[email protected]>
> Cc: Sean Christopherson <[email protected]>
> ---
> arch/x86/kernel/cpu/sgx/main.c | 12 ++++++++----
> 1 file changed, 8 insertions(+), 4 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> index f201ca85212f..a4a65eadfb79 100644
> --- a/arch/x86/kernel/cpu/sgx/main.c
> +++ b/arch/x86/kernel/cpu/sgx/main.c
> @@ -291,7 +291,7 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
> * problematic as it would increase the lock contention too much, which would
> * halt forward progress.
> */
> -static void __sgx_reclaim_pages(int nr_to_scan)
> +static int __sgx_reclaim_pages(int nr_to_scan)

Nit: I wonder if we should use ssize_t here?

If nothing else, it would document better than 'int'.

BR, Jarkko

2022-12-08 10:10:21

[permalink] [raw]

Subject: Re: [PATCH v2 12/18] x86/sgx: Expose sgx_reclaim_pages() for use by EPC cgroup

On Fri, Dec 02, 2022 at 10:36:48AM -0800, Kristen Carlson Accardi wrote:
> From: Sean Christopherson <[email protected]>
>
> Expose the top-level reclaim function as sgx_reclaim_epc_pages() for use
> by the upcoming EPC cgroup, which will initiate reclaim to enforce
> changes to high/max limits.
>
> Signed-off-by: Sean Christopherson <[email protected]>
> Signed-off-by: Kristen Carlson Accardi <[email protected]>
> Cc: Sean Christopherson <[email protected]>
> ---
> arch/x86/kernel/cpu/sgx/main.c | 7 ++++---
> arch/x86/kernel/cpu/sgx/sgx.h | 1 +
> 2 files changed, 5 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> index 96399e2016a8..c947b4ae06f3 100644
> --- a/arch/x86/kernel/cpu/sgx/main.c
> +++ b/arch/x86/kernel/cpu/sgx/main.c
> @@ -281,6 +281,7 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
>
> /**
> * sgx_reclaim_pages() - Reclaim EPC pages from the consumers
> + * sgx_reclaim_epc_pages() - Reclaim EPC pages from the consumers
> * @nr_to_scan: Number of EPC pages to scan for reclaim
> * @ignore_age: Reclaim a page even if it is young
> *
> @@ -385,7 +386,7 @@ static int __sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
> return i;
> }
>
> -static int sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
> +int sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age)
> {
> int ret;
>
> @@ -441,7 +442,7 @@ static int ksgxd(void *p)
> sgx_should_reclaim(SGX_NR_HIGH_PAGES));
>
> if (sgx_should_reclaim(SGX_NR_HIGH_PAGES))
> - sgx_reclaim_pages(SGX_NR_TO_SCAN, false);
> + sgx_reclaim_epc_pages(SGX_NR_TO_SCAN, false);
> }
>
> return 0;
> @@ -624,7 +625,7 @@ struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim)
> break;
> }
>
> - sgx_reclaim_pages(SGX_NR_TO_SCAN, false);
> + sgx_reclaim_epc_pages(SGX_NR_TO_SCAN, false);
> }
>
> if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
> diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
> index ec8d567cd975..ce859331ddf5 100644
> --- a/arch/x86/kernel/cpu/sgx/sgx.h
> +++ b/arch/x86/kernel/cpu/sgx/sgx.h
> @@ -206,6 +206,7 @@ void sgx_reclaim_direct(void);
> void sgx_record_epc_page(struct sgx_epc_page *page, unsigned long flags);
> int sgx_drop_epc_page(struct sgx_epc_page *page);
> struct sgx_epc_page *sgx_alloc_epc_page(void *owner, bool reclaim);
> +int sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age);
>
> void sgx_ipi_cb(void *info);
>
> --
> 2.38.1
>

Unless, there is a risk of name collision, I think this rename is
just adding unnecessary convolution to the patch set.

I would revert the rename part, and just export.

BR, Jarkko

2022-12-08 10:12:33

[permalink] [raw]

Subject: Re: [PATCH v2 11/18] x86/sgx: Prepare for multiple LRUs

On Fri, Dec 02, 2022 at 10:36:47AM -0800, Kristen Carlson Accardi wrote:
> From: Sean Christopherson <[email protected]>
>
> Add sgx_can_reclaim() wrapper so that in a subsequent patch, multiple LRUs
> can be used cleanly.

Nit: Patch is the transient form of a change. Once a change has been
committed, it is a commit.

Further, you should instead explain why subsequent changes require
sgx_can_reclaim() wrapper, than just claim it without argument.

Alternatively, you can consider squashing this to the subsequent
patch, which makes use of the wrapper.

BR, Jarkko

2022-12-08 10:23:59

[permalink] [raw]

Subject: Re: [PATCH v2 10/18] x86/sgx: Add option to ignore age of page during EPC reclaim

On Fri, Dec 02, 2022 at 10:36:46AM -0800, Kristen Carlson Accardi wrote:
> From: Sean Christopherson <[email protected]>
>
> Add a flag to sgx_reclaim_pages() to instruct it to ignore the age of
> page, i.e. reclaim the page even if it's young. The EPC cgroup will use
> the flag to enforce its limits by draining the reclaimable lists before
> resorting to other measures, e.g. forcefully reclaimable "unreclaimable"
> pages by killing enclaves.
>
> Signed-off-by: Sean Christopherson <[email protected]>
> Signed-off-by: Kristen Carlson Accardi <[email protected]>
> Cc: Sean Christopherson <[email protected]>
> ---
> arch/x86/kernel/cpu/sgx/main.c | 46 +++++++++++++++++++++-------------
> 1 file changed, 29 insertions(+), 17 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> index a4a65eadfb79..db96483e2e74 100644
> --- a/arch/x86/kernel/cpu/sgx/main.c
> +++ b/arch/x86/kernel/cpu/sgx/main.c
> @@ -31,6 +31,10 @@ static DEFINE_XARRAY(sgx_epc_address_space);
> * with sgx_global_lru.lock acquired.
> */
> static struct sgx_epc_lru_lists sgx_global_lru;

Please, separate these by an empty line.

> +static inline struct sgx_epc_lru_lists *sgx_lru_lists(struct sgx_epc_page *epc_page)
> +{
> + return &sgx_global_lru;
> +}

Should be named by the thing it returns, not by the type.

BR, Jarkko

2022-12-08 10:44:53

[permalink] [raw]

Subject: Re: [PATCH v2 13/18] x86/sgx: Add helper to grab pages from an arbitrary EPC LRU

On Fri, Dec 02, 2022 at 10:36:49AM -0800, Kristen Carlson Accardi wrote:
> From: Sean Christopherson <[email protected]>
>
> Move the isolation loop into a standalone helper, sgx_isolate_pages(),
> in preparation for existence of multiple LRUs. Expose the helper to
> other SGX code so that it can be called from the EPC cgroup code, e.g.
> to isolate pages from a single cgroup LRU. Exposing the isolation loop
> allows the cgroup iteration logic to be wholly encapsulated within the
> cgroup code.
>
> Signed-off-by: Sean Christopherson <[email protected]>
> Signed-off-by: Kristen Carlson Accardi <[email protected]>
> Cc: Sean Christopherson <[email protected]>
> ---
> arch/x86/kernel/cpu/sgx/main.c | 68 +++++++++++++++++++++-------------
> arch/x86/kernel/cpu/sgx/sgx.h | 2 +
> 2 files changed, 44 insertions(+), 26 deletions(-)
>
> diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> index c947b4ae06f3..a59550fa150b 100644
> --- a/arch/x86/kernel/cpu/sgx/main.c
> +++ b/arch/x86/kernel/cpu/sgx/main.c
> @@ -280,7 +280,46 @@ static void sgx_reclaimer_write(struct sgx_epc_page *epc_page,
> }
>
> /**
> - * sgx_reclaim_pages() - Reclaim EPC pages from the consumers
> + * sgx_isolate_epc_pages() - Isolate pages from an LRU for reclaim
> + * @lru: LRU from which to reclaim
> + * @nr_to_scan: Number of pages to scan for reclaim
> + * @dst: Destination list to hold the isolated pages
> + */
> +void sgx_isolate_epc_pages(struct sgx_epc_lru_lists *lru, int *nr_to_scan,
> + struct list_head *dst)

Why not instead return the number of pages scanned, and pass
'int nr_to_scan'?

That would just be more idiomatic choice.

BR, Jarkko

2022-12-08 15:11:41

[permalink] [raw]

Subject: Re: [PATCH v2 15/18] cgroup/misc: Add per resource callbacks for css events

On Fri, Dec 02, 2022 at 10:36:51AM -0800, Kristen Carlson Accardi wrote:
> Consumers of the misc cgroup controller might need to perform separate actions
> in the event of a cgroup alloc, free or release call. In addition,
> writes to the max value may also need separate action. Add the ability
> to allow downstream users to setup callbacks for these operations, and
> call the per resource type callback when appropriate.
>
> This code will be utilized by the SGX driver in a future patch.
>
> Signed-off-by: Kristen Carlson Accardi <[email protected]>

I don't know what css is.

BR, Jarkko

2022-12-08 15:32:28

[permalink] [raw]

Subject: Re: [PATCH v2 15/18] cgroup/misc: Add per resource callbacks for css events

On Thu, Dec 08, 2022 at 02:53:24PM +0000, Jarkko Sakkinen wrote:
> On Fri, Dec 02, 2022 at 10:36:51AM -0800, Kristen Carlson Accardi wrote:
> > Consumers of the misc cgroup controller might need to perform separate actions
> > in the event of a cgroup alloc, free or release call. In addition,
> > writes to the max value may also need separate action. Add the ability
> > to allow downstream users to setup callbacks for these operations, and
> > call the per resource type callback when appropriate.
> >
> > This code will be utilized by the SGX driver in a future patch.
> >
> > Signed-off-by: Kristen Carlson Accardi <[email protected]>
>
> I don't know what css is.

Now I know but it should be described in the commit message, i.e.
what the css is abbrevation of, and what does it mean in practice.

BR, Jarkko

2022-12-08 15:34:17

[permalink] [raw]

Subject: Re: [PATCH v2 14/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Fri, Dec 02, 2022 at 10:36:50AM -0800, Kristen Carlson Accardi wrote:
> From: Sean Christopherson <[email protected]>
>
> Introduce the OOM path for killing an enclave with the reclaimer
> is no longer able to reclaim enough EPC pages. Find a victim enclave,
> which will be an enclave with EPC pages remaining that are not
> accessible to the reclaimer ("unreclaimable"). Once a victim is
> identified, mark the enclave as OOM and zap the enclaves entire
> page range. Release all the enclaves resources except for the
> struct sgx_encl memory itself.
>
> Signed-off-by: Sean Christopherson <[email protected]>
> Signed-off-by: Kristen Carlson Accardi <[email protected]>
> Cc: Sean Christopherson <[email protected]>

Why this patch is dependent of all 13 patches before it?

Looks like something that is orthogonal to cgroups and could be
live by its own. At least it probably does not require all of
those patches, or does it?

Even without cgroups it would make sense to killing enclaves if
reclaimer gets stuck.

BR, Jarkko

2022-12-08 15:37:48

[permalink] [raw]

Subject: Re: [PATCH v2 17/18] x86/sgx: Add support for misc cgroup controller

Perhaps "x86/sgx: Limit process EPC usage with misc cgroup controller"?

Or something more to the point than "add support".

On Fri, Dec 02, 2022 at 10:36:53AM -0800, Kristen Carlson Accardi wrote:

> /**
> - * sgx_reclaim_epc_pages() - Reclaim EPC pages from the consumers
> + * __sgx_reclaim_epc_pages() - Reclaim EPC pages from the consumers
> * @nr_to_scan: Number of EPC pages to scan for reclaim
> * @ignore_age: Reclaim a page even if it is young
> + * @epc_cg: EPC cgroup from which to reclaim
> *
> * Take a fixed number of pages from the head of the active page pool and
> * reclaim them to the enclave's private shmem files. Skip the pages, which have
> @@ -336,7 +350,8 @@ void sgx_isolate_epc_pages(struct sgx_epc_lru_lists *lru, int *nr_to_scan,
> * problematic as it would increase the lock contention too much, which would
> * halt forward progress.
> */
> -static int __sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
> +static int __sgx_reclaim_epc_pages(int nr_to_scan, bool ignore_age,
> + struct sgx_epc_cgroup *epc_cg)
> {
> struct sgx_backing backing[SGX_MAX_NR_TO_RECLAIM];
> struct sgx_epc_page *epc_page, *tmp;
> @@ -347,7 +362,15 @@ static int __sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
> int i = 0;
> int ret;
>
> - sgx_isolate_epc_pages(&sgx_global_lru, &nr_to_scan, &iso);
> + /*
> + * If a specific cgroup is not being targetted, take from the global
> + * list first, even when cgroups are enabled. If there are
> + * pages on the global LRU then they should get reclaimed asap.
> + */
> + if (!IS_ENABLED(CONFIG_CGROUP_SGX_EPC) || !epc_cg)
> + sgx_isolate_epc_pages(&sgx_global_lru, &nr_to_scan, &iso);
> +
> + sgx_epc_cgroup_isolate_pages(epc_cg, &nr_to_scan, &iso);
>
> if (list_empty(&iso))
> return 0;
> @@ -397,25 +420,33 @@ static int __sgx_reclaim_pages(int nr_to_scan, bool ignore_age)
> SGX_EPC_PAGE_ENCLAVE |
> SGX_EPC_PAGE_VERSION_ARRAY);
>
> + if (epc_page->epc_cg) {
> + sgx_epc_cgroup_uncharge(epc_page->epc_cg);
> + epc_page->epc_cg = NULL;
> + }
> +
> sgx_free_epc_page(epc_page);
> }
> return i;
> }

I would consider changes to sgx_reclaim_epc_pages() as a separate patch,
perhaps squashing with the patch that does the export. And generally
separate from this patch all internal arch/x86/kernel/cpu/sgx changes,
and leave only cgroup bindings.

BR, Jarkko

2022-12-08 16:39:01

[permalink] [raw]

Subject: Re: [PATCH v2 03/18] x86/sgx: Add 'struct sgx_epc_lru_lists' to encapsulate lru list(s)

On Fri, Dec 02, 2022 at 10:36:39AM -0800, Kristen Carlson Accardi wrote:
> Introduce a data structure to wrap the existing reclaimable list
> and its spinlock in a struct to minimize the code changes needed
> to handle multiple LRUs as well as reclaimable and non-reclaimable
> lists, both of which will be introduced and used by SGX EPC cgroups.
>
> Signed-off-by: Sean Christopherson <[email protected]>
> Signed-off-by: Kristen Carlson Accardi <[email protected]>
> Cc: Sean Christopherson <[email protected]>
> ---
> arch/x86/kernel/cpu/sgx/sgx.h | 65 +++++++++++++++++++++++++++++++++++
> 1 file changed, 65 insertions(+)
>
> diff --git a/arch/x86/kernel/cpu/sgx/sgx.h b/arch/x86/kernel/cpu/sgx/sgx.h
> index 39cb15a8abcb..5e6d88438fae 100644
> --- a/arch/x86/kernel/cpu/sgx/sgx.h
> +++ b/arch/x86/kernel/cpu/sgx/sgx.h
> @@ -90,6 +90,71 @@ static inline void *sgx_get_epc_virt_addr(struct sgx_epc_page *page)
> return section->virt_addr + index * PAGE_SIZE;
> }
>
> +/*
> + * This data structure wraps a list of reclaimable EPC pages, and a list of
> + * non-reclaimable EPC pages and is used to implement a LRU policy during
> + * reclamation.
> + */
> +struct sgx_epc_lru_lists {
> + spinlock_t lock;
> + struct list_head reclaimable;
> + struct list_head unreclaimable;
> +};

Why this is named like this, and not sgx_epc_global_rcu? Are there
any other use cases?

BR, Jarkko

2022-12-08 18:24:51

[permalink] [raw]

Subject: Re: [PATCH v2 03/18] x86/sgx: Add 'struct sgx_epc_lru_lists' to encapsulate lru list(s)

On Thu, 2022-12-08 at 15:31 +0000, Jarkko Sakkinen wrote:
> On Fri, Dec 02, 2022 at 10:36:39AM -0800, Kristen Carlson Accardi
> wrote:
> > Introduce a data structure to wrap the existing reclaimable list
> > and its spinlock in a struct to minimize the code changes needed
> > to handle multiple LRUs as well as reclaimable and non-reclaimable
> > lists, both of which will be introduced and used by SGX EPC
> > cgroups.
> >
> > Signed-off-by: Sean Christopherson
> > <[email protected]>
> > Signed-off-by: Kristen Carlson Accardi <[email protected]>
> > Cc: Sean Christopherson <[email protected]>
> > ---
> > arch/x86/kernel/cpu/sgx/sgx.h | 65
> > +++++++++++++++++++++++++++++++++++
> > 1 file changed, 65 insertions(+)
> >
> > diff --git a/arch/x86/kernel/cpu/sgx/sgx.h
> > b/arch/x86/kernel/cpu/sgx/sgx.h
> > index 39cb15a8abcb..5e6d88438fae 100644
> > --- a/arch/x86/kernel/cpu/sgx/sgx.h
> > +++ b/arch/x86/kernel/cpu/sgx/sgx.h
> > @@ -90,6 +90,71 @@ static inline void *sgx_get_epc_virt_addr(struct
> > sgx_epc_page *page)
> >         return section->virt_addr + index * PAGE_SIZE;
> > }
> >
> > +/*
> > + * This data structure wraps a list of reclaimable EPC pages, and
> > a list of
> > + * non-reclaimable EPC pages and is used to implement a LRU policy
> > during
> > + * reclamation.
> > + */
> > +struct sgx_epc_lru_lists {
> > +       spinlock_t lock;
> > +       struct list_head reclaimable;
> > +       struct list_head unreclaimable;
> > +};
>
> Why this is named like this, and not sgx_epc_global_rcu? Are there
> any other use cases?
>
> BR, Jarkko

Yes, there are other use cases that are introduced in the other
patches. This structure is used to in the cgroup struct to hold cgroup
specific LRUs.

2022-12-09 16:22:59

[permalink] [raw]

Subject: Re: [PATCH v2 14/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Thu, 2022-12-08 at 15:21 +0000, Jarkko Sakkinen wrote:
> On Fri, Dec 02, 2022 at 10:36:50AM -0800, Kristen Carlson Accardi
> wrote:
> > From: Sean Christopherson <[email protected]>
> >
> > Introduce the OOM path for killing an enclave with the reclaimer
> > is no longer able to reclaim enough EPC pages. Find a victim
> > enclave,
> > which will be an enclave with EPC pages remaining that are not
> > accessible to the reclaimer ("unreclaimable"). Once a victim is
> > identified, mark the enclave as OOM and zap the enclaves entire
> > page range. Release all the enclaves resources except for the
> > struct sgx_encl memory itself.
> >
> > Signed-off-by: Sean Christopherson
> > <[email protected]>
> > Signed-off-by: Kristen Carlson Accardi <[email protected]>
> > Cc: Sean Christopherson <[email protected]>
>
> Why this patch is dependent of all 13 patches before it?
>
> Looks like something that is orthogonal to cgroups and could be
> live by its own. At least it probably does not require all of
> those patches, or does it?
>
> Even without cgroups it would make sense to killing enclaves if
> reclaimer gets stuck.
>
> BR, Jarkko

It is dependent first of all of having the LRU struct with the
unreclaimable/reclaimable lists. Which means it requires storing the
enclave pointer in the page as well. It's dependent on knowing how many
pages are available, being able to ignore the age of a page etc. Right
now, without cgroups, sgx will be unable to allocate memory when an
enclave is created if it cannot reclaim enough memory from the existing
in use enclaves.

Aside from that though, I don't think that killing enclaves makes sense
outside the context of cgroup limits. Without cgroup limits, you have a
max number of EPC pages that you can have active at any one time. If an
enclave attempts to allocate a new page and the reclaimer can't free up
any, how would you decide whether it's ok to kill an entire enclave in
order to grant this other enclave the higher priority for getting a
page? With a cgroup limit, the system owner explicitly can decide what
the limits on usage will be, but without that, you'd have a situation
where one new enclave could kill others I would think. Better to just
have it the way it is - new page allocations fail if there are not free
pages, but you don't kill enclaves that already exist.

2022-12-09 16:42:34

[permalink] [raw]

Subject: Re: [PATCH v2 14/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On 12/9/22 08:05, Kristen Carlson Accardi wrote:
> Aside from that though, I don't think that killing enclaves makes sense
> outside the context of cgroup limits.

I think it makes a lot of sense in theory. Whatever situation we get
into with a cgroup's EPC we can also get into with the whole system's EPC.

*But*, it's orders of magnitude harder to hit on the whole system.
Basically, it has to be at a point where all of the EPC is consumed in
non-SGX-swappable page types like SECS or VEPC pages. That's
_possible_, of course, but it's really hard to create because one VEPC
page can hold the info of several (32??) swapped-out EPC pages.

So, you'd need roughly 4GB of swapped-out normal enclave memory to
exhaust a system with 128MB of total enclave memory.

OOM handling *much* necessary in practice if you have a cgroup with some
modestly sized enclaves and a very tiny EPC limit. If someone wants to
extend this OOM support to system-wide EPC later, then go ahead. But, I
don't think it makes a lot of sense to invert this series for it.

2022-12-12 18:30:03

by Sean Christopherson

[permalink] [raw]

Subject: Re: [PATCH v2 14/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Fri, Dec 09, 2022, Dave Hansen wrote:
> On 12/9/22 08:05, Kristen Carlson Accardi wrote:
> > Aside from that though, I don't think that killing enclaves makes sense
> > outside the context of cgroup limits.
>
> I think it makes a lot of sense in theory. Whatever situation we get
> into with a cgroup's EPC we can also get into with the whole system's EPC.
>
> *But*, it's orders of magnitude harder to hit on the whole system.

...

> If someone wants to extend this OOM support to system-wide EPC later, then go
> ahead. But, I don't think it makes a lot of sense to invert this series for
> it.

+1 from the peanut gallery. With VMM EPC oversubscription suport, no sane VMM
will oversubscribe VEPC pages. And for VA pages, supporting swap of VA pages is
likely a more userspace-friendly approach if system-wide EPC OOM is a concern.

2022-12-26 21:24:11