[Kernel Patchset]
===============
Git tree:
https://github.com/userzj/linux.git zhujia/shared-domain-v1
Git web:
https://github.com/userzj/linux/tree/zhujia/shared-domain-v1
[Background]
============
In ondemand read mode, we use individual volume to present an erofs
mountpoint, cookies to present bootstrap and data blobs.
In which case, since cookies can't be shared between fscache volumes,
even if the data blobs between different mountpoints are exactly same,
they can't be shared.
[Introduction]
==============
Here we introduce erofs shared domain to resolve above mentioned case.
Several erofs filesystems can belong to one domain, and data blobs can
be shared among these erofs filesystems of same domain.
[Usage]
Users could specify 'domain_id' mount option to create or join into a
domain which reuses the same cookies(blobs).
[Design]
========
1. Use pseudo mnt to manage domain's lifecycle.
2. Use a linked list to maintain & traverse domains.
3. Use pseudo sb to create anonymous inode for recording cookie's info
and manage cookies lifecycle.
[Flow Path]
===========
1. User specify a new 'domain_id' in mount option.
1.1 Traverse domain list, compare domain_id with existing domain.[Miss]
1.2 Create a new domain(volume), add it to domain list.
1.3 Traverse pseudo sb's inode list, compare cookie name with
existing cookies.[Miss]
1.4 Alloc new anonymous inodes and cookies.
2. User specify an existing 'domain_id' in mount option and the data
blob is existed in domain.
2.1 Traverse domain list, compare domain_id with existing domain.[Hit]
2.2 Reuse the domain and increase its refcnt.
2.3 Traverse pseudo sb's inode list, compare cookie name with
existing cookies.[Hit]
2.4 Reuse the cookie and increase its refcnt.
[Test]
======
Git web: (More test cases will be added.)
https://github.com/userzj/demand-read-cachefilesd/tree/shared-domain
Jia Zhu (5):
erofs: add 'domain_id' mount option for on-demand read sementics
erofs: introduce fscache-based domain
erofs: add 'domain_id' prefix when register sysfs
erofs: remove duplicated unregister_cookie
erofs: support fscache based shared domain
fs/erofs/Makefile | 2 +-
fs/erofs/domain.c | 175 ++++++++++++++++++++++++++++++++++++++++++++
fs/erofs/fscache.c | 17 ++++-
fs/erofs/internal.h | 34 ++++++++-
fs/erofs/super.c | 45 +++++++++---
fs/erofs/sysfs.c | 11 ++-
6 files changed, 270 insertions(+), 14 deletions(-)
create mode 100644 fs/erofs/domain.c
--
2.20.1
Hi Jia,
On Wed, Aug 31, 2022 at 08:31:20PM +0800, Jia Zhu wrote:
> [Kernel Patchset]
> ===============
> Git tree:
> https://github.com/userzj/linux.git zhujia/shared-domain-v1
> Git web:
> https://github.com/userzj/linux/tree/zhujia/shared-domain-v1
>
> [Background]
> ============
> In ondemand read mode, we use individual volume to present an erofs
> mountpoint, cookies to present bootstrap and data blobs.
>
> In which case, since cookies can't be shared between fscache volumes,
> even if the data blobs between different mountpoints are exactly same,
> they can't be shared.
>
> [Introduction]
> ==============
> Here we introduce erofs shared domain to resolve above mentioned case.
> Several erofs filesystems can belong to one domain, and data blobs can
> be shared among these erofs filesystems of same domain.
As we discussed in the previous community meeting, I agree that is useful
and it's the prerequisite of storage/page cache sharing between blocks
among different images (filesystems).
Thanks for your time and effort on this!
>
> [Usage]
> Users could specify 'domain_id' mount option to create or join into a
> domain which reuses the same cookies(blobs).
>
> [Design]
> ========
> 1. Use pseudo mnt to manage domain's lifecycle.
> 2. Use a linked list to maintain & traverse domains.
> 3. Use pseudo sb to create anonymous inode for recording cookie's info
> and manage cookies lifecycle.
>
> [Flow Path]
> ===========
> 1. User specify a new 'domain_id' in mount option.
> 1.1 Traverse domain list, compare domain_id with existing domain.[Miss]
> 1.2 Create a new domain(volume), add it to domain list.
> 1.3 Traverse pseudo sb's inode list, compare cookie name with
> existing cookies.[Miss]
> 1.4 Alloc new anonymous inodes and cookies.
>
> 2. User specify an existing 'domain_id' in mount option and the data
> blob is existed in domain.
> 2.1 Traverse domain list, compare domain_id with existing domain.[Hit]
> 2.2 Reuse the domain and increase its refcnt.
> 2.3 Traverse pseudo sb's inode list, compare cookie name with
> existing cookies.[Hit]
> 2.4 Reuse the cookie and increase its refcnt.
>
> [Test]
> ======
> Git web: (More test cases will be added.)
> https://github.com/userzj/demand-read-cachefilesd/tree/shared-domain
I'd suggest integrating to erofs-utils testcases directly, see
https://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git/log/?h=experimental-tests-fscache
Thanks,
Gao Xiang