Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35;
Message-ID: <4082bc40-a99a-4b54-91e5-a1b55828d202@oracle.com>
Date:   Mon, 23 Oct 2023 16:44:29 -0600
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To:     Matthew Wilcox <willy@infradead.org>,
        David Hildenbrand <david@redhat.com>,
        Mike Kravetz <mike.kravetz@oracle.com>,
        Peter Xu <peterx@redhat.com>, rongwei.wang@linux.alibaba.com,
        Mark Hemment <markhemm@googlemail.com>
Cc:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-arch@vger.kernel.org" <linux-arch@vger.kernel.org>
From:   Khalid Aziz <khalid.aziz@oracle.com>
Subject: Sharing page tables across processes (mshare)
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SzY0VmtkNFhZUVVFNnVNZExqcGxmNGZVTGhaZUkvcm5nWlZwOUVyd3A2cEJr?=
 =?utf-8?B?Vm5SYXZXaFJjSVpXYUF4Y2Y3V0NraDFjT0hQUWlqcm81dHBTd1FxUlkzcSta?=
 =?utf-8?B?VDMzWXdzd004WXJjM2IzWGhjelNDRmp0bUVKUWtPbnpKdDZBVExVMEM2ZW5N?=
 =?utf-8?B?UERXOURYUDVZWkxYR20vRXdQdTJHeXQwdGtPTk4vTUpSQVdmQlMwTXUyRkhu?=
 =?utf-8?B?WU5pMCsxSTJSS1RXQmZhQ2l6TVVzdmY4SkJKZ201aVJILzIvdFVUeFJ4M09U?=
 =?utf-8?B?a0VKUXBMZzY2eTdNaGZaaVNQMTczVjc4aTluQWJyY0hlT21KQUhjeXkyUzBi?=
 =?utf-8?B?eVk5TWdUcmJkM3k2VEg4dStBR3J5dGErUDZKcDEzTUlhUmJZNkpEVDJLa1Yv?=
 =?utf-8?B?cUJGZ0xaS2ZSalMwNzIzc0dxOFdvRGM3djIvSlVqNjNyb1d5UWhpTExYRkhq?=
 =?utf-8?B?Wm9QalJVWDhQbDZRYjNneUlaenJsNCtRN1hTd2pkOHZtL0crWHVEM0ZpcnlV?=
 =?utf-8?B?eVg5VGQxRmIwWXU4cXVleVRIUHhOOVdTY3I3RFZsbVgyV1pybGR0MGhsTTZD?=
 =?utf-8?B?UUZqRjUxQlRuaE5kbDBrcmU1aHJTWTJUU0FEMXlOcVUyRERlZ0pheTlHWTU2?=
 =?utf-8?B?K20ybVl5QnRxeUdBS3AyblZVQ2NEb05MTzd2SmgrbUJhL2xCbWVISWJRaFRL?=
 =?utf-8?B?QXdycitJb2tCczlrOEE3MkRXU3ZGUEhuQXdENjZHZEo4ZFhqUmlDMXZUWFZL?=
 =?utf-8?B?dE5sRm1ZbVlrbzY4cGl4TWhyTys2b3RYeFFBdXBOVW1LaGFjWmJZUHhEQUND?=
 =?utf-8?B?SERyWEd6SlZYRkFHUnZvQmNsUmxyUFJQNW5kUitNeHBnR2RjYWI5VEVpejFw?=
 =?utf-8?B?Kzh5TXJtSVBEbitHTTgzS05uWkNydzNZanRaajNkQVF1SHkzT0JVeXUvR213?=
 =?utf-8?B?dnVpLzk3R0dwbnFxWktLRnQ2Rmx2ZzUzZFY0VE9yZW5TOEVjU042QTk0R0ZF?=
 =?utf-8?B?aVZDdi9keUlxZnMzbS9uU216QytXWlBhakRDWFhtZEFyQVkyVGlWUkpLVmVN?=
 =?utf-8?B?WE52K21jSnhleUkyaTRZazhSQWlFM0Y3dkFqbzVSTDRmaHN3UDBzTU4rWUov?=
 =?utf-8?B?cGVKZUNZNStyRnhBYU1wbXV0NkxjMmd2RGllT1U4OExkb0owS0pYKzhSUTBL?=
 =?utf-8?B?bytoYnFZTEl3TnZPUDdQbUd5TThNNS84ejR1eU9qQmpRWUpoWG9nN0xtcEJt?=
 =?utf-8?B?eTlwVHFCd2VVQ1BxU3V1ckgzaGpqcElvcU5TWG9lVVdtSHNydnBTa1hOT0VP?=
 =?utf-8?B?a1ZqQXB0R1VCMHpjQ2hwc0JQNlIxbnNZTVVER01DMXlycW14SVo1SVI0bnlR?=
 =?utf-8?B?TTQ5MHNjb1Y2aFB3ekRoaTlpaEJja1kvbVI3eFFHQTd4NXVFZGg1MU9IdzNF?=
 =?utf-8?B?b1I2bk1hU2EwcGQ1ZElBRU1aOGdaazRCak5lNU1kS2plSDNFaFhUd09nNEpy?=
 =?utf-8?B?MTIrcXpUZzRYV3pQL2RXZWNFV21zRlQrSlNITzZnb1ljWUo4aWlEYW96eGlI?=
 =?utf-8?B?OFR2TFQ0dHEyU2JIQVRwcW5UdVRxL29jd1JnV2hXWGpCZldUT2x1R2paaCtn?=
 =?utf-8?B?UTF2MHhrUlVyQ2FKMkgrWERKT0FmU2xDajE0V2FjV3M0U1R4MW05djgxUSs0?=
 =?utf-8?B?RUNKaFgyaC8wdC9tRDJMSWd5RkVjMkFNWTZ5RitEL3Y2LzdrZU1qZ1hYWGlw?=
 =?utf-8?B?YS83Q1JxU2tMbHZ3Uy9WNEI1WTR5djlVdHFXZFl1ZGRpNk41cFR4bWhEaElR?=
 =?utf-8?B?STlGbnJaU3JaWjJJZCtTT0tMVEVZU0djejJPTWd0UERLTmxxT2pMSzNWc01k?=
 =?utf-8?B?eVFSS0tpT1NxNWpFUkdsYWdVZzBBRStuS3B2c2NTYnV2eCtNelVsV1YvRk5W?=
 =?utf-8?B?a0hLRVpIRWJTUEtoZjZuRDgwSFh4eEEwSytYWGJvN0Z4WVZaTDdnZjdndVEr?=
 =?utf-8?B?cjdGb2tDVTNPcUxxdFBlMzlLVTUwdStWcVpaTHBqaGZpRDMzczRJM0E1YjBj?=
 =?utf-8?B?V1E1NS90YVBHYlhHUkRnZWxHOVFNV0FHbkhVRXFSZ0NWajU5Yk9ETTRtZlND?=
 =?utf-8?B?MTJwdkt4NG1xNUplTXdoYmxSNDh0UDNtR2dmcjBmMmdjeHc1aWM5MDBDaVVi?=
 =?utf-8?B?WGc9PQ==?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?cW92T05QbWY1ZFJ3NzJXZzVPTDRBbEF4TTkwOHBuWVNmeGVPZUYxWitHQ2FM?=
 =?utf-8?B?TXlOLzRoUFIrazd2YmcxeHFFdVNRSmFqV25KWkdEeUxDVDVIdWFvZ0NVZWZ6?=
 =?utf-8?B?bTZVMVQ0K3hPRjY0dVFaVnJ1Mzd6dXA1aTkzWUNJZC9ReWRqMW1Tb3FIeVdU?=
 =?utf-8?B?SXZuNFJFZDU4RGRHcy8wUjlxSjNjQ29OZ0Mwbmd0blAydDQyRG1iZEFiTXl5?=
 =?utf-8?B?QTM5bkoxZEVKZEcvdUhaWnd0V2RIQjEyUmVxbEdkWlVJemVMVzBVN1dVTndY?=
 =?utf-8?B?Wm12UkdZeFV3MzZ3VHBwOXA4em5WK29qdjg2Vk1iQys1SlZqL0JUVUlRdUtu?=
 =?utf-8?B?Yi9lSHVyK0F3SmMvRzZjK1kwYnA4aHg1UkV4NFJ5c3plK2xZcU91MXM4NGdo?=
 =?utf-8?B?REdJUzkxRTBsWEltZVp2T01xWEUxT3FYcmFxSUNxWGlMWThDUk0yVE5OdS9N?=
 =?utf-8?B?a2pHZzhQL3dySXZFNHVYRG5lRTFUdWtTb1p3UXJ2cThnWVBCZHAyVzJSSkxX?=
 =?utf-8?B?b2xzaWpBMlVHWEFkMXVHOExrbWxNeHRiT0NRZVN3OWlyb3FIbmhWUUNSTVQy?=
 =?utf-8?B?RlpQeCtwd0EvSkVQd3lncVo3TjMvOWxCeUdlTDgyTWNYSHdDY053S1VyWVVO?=
 =?utf-8?B?QnY5WlNvWlhQUGFsbDN5VCtoR1BFMUg4d1NQem1OeUJsRWl3YTVFdkw0VDJZ?=
 =?utf-8?B?Tno4eC9TT01ZUHFsb1pDTTBYWG8vY25FeHhNYk8yb3BBN0VDZTloMU9KSUQv?=
 =?utf-8?B?cU1oRFFTN3ptRGl4SUlHZW9tMXBJalJ1TzRQNVNGNTdCendDMFpvZ0FLQUNE?=
 =?utf-8?B?UmhUMFRVWHdpd3BqaE00WXorR04yWDhtNlB2UVZjeGxieGVHdEZEV0E0V1F4?=
 =?utf-8?B?aXVYdXljbDNIbnQ0ZVdLODJVNVlEUWZpUWduTm5taWpzOHBRczZOamo2RVdk?=
 =?utf-8?B?L3A5YkpDcTJUMTZMMGFPTnRERlNLRE1wVW1WZmpSU2czdnhnakk4NGg1WFlD?=
 =?utf-8?B?QTBxRDlvWVBTcVNCcG10VTVKYy9RZ1ZyZHZGVFVnSnZlSklDSlVvbUVxOXh4?=
 =?utf-8?B?VmV0NGpuU24yc3JYUTFPcUJKMTBVN0tkOHltV3FNanl4SEZ0TmtBZ2tNUXdz?=
 =?utf-8?B?T2p0cm56dkZMQWNEVHJvTUE5bGxCbG85Vk5iVCt1WGpIRXF3dGx2VEY5RWhQ?=
 =?utf-8?B?K0tjRkZ2SmJEMVpKTHBCNEF6UzFFTVlrTXdUamJ5Zjg4SHBYR09lYW5uNGlp?=
 =?utf-8?Q?aEH40hGdCAkvRGY?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4616b85b-167a-4783-e787-08dbd419a35d
X-MS-Exchange-CrossTenant-AuthSource: CH3PR10MB6810.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Oct 2023 22:44:38.4364
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: y5guIGuyd9qXz4OIyAtrof8HjB/X3SKgK/HJMq0m0+zyFKnhdfQHjR80S78n4Qie0X/Lf5qyT2931Bd/kou9Sw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR10MB7433
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.980,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2023-10-23_21,2023-10-19_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 malwarescore=0 mlxscore=0
 suspectscore=0 spamscore=0 adultscore=0 mlxlogscore=999 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2310170001
 definitions=main-2310230200
X-Proofpoint-GUID: 34fP7oTbK73ZCZnW-NVXtAow5d0HGzN4
X-Proofpoint-ORIG-GUID: 34fP7oTbK73ZCZnW-NVXtAow5d0HGzN4
X-Spam-Status: No, score=-0.8 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 23 Oct 2023 15:45:51 -0700 (PDT)

Threads of a process share address space and page tables that allows for
two key advantages:

1. Amount of memory required for PTEs to map physical pages stays low
even when large number of threads share the same pages since PTEs are
shared across threads.

2. Page protection attributes are shared across threads and a change
of attributes applies immediately to every thread without any overhead
of coordinating protection bit changes across threads.

These advantages no longer apply when unrelated processes share pages.
Some applications can require 1000s of processes that all access the
same set of data on shared pages. For instance, a database server may
map in a large chunk of database into memory to provide fast access to
data to the clients using buffer cache. Server may launch new processes
to provide services to new clients connecting to the shared database.
Each new process will map in the shared database pages. When the PTEs
for mapping in shared pages are not shared across processes, each
process will consume some memory to store these PTEs. On x86_64, each
page requires a PTE that is only 8 bytes long which is very small
compared to the 4K page size. When 2000 processes map the same page in
their address space, each one of them requires 8 bytes for its PTE and
together that adds up to 8K of memory just to hold the PTEs for one 4K
page. On a database server with 300GB SGA, a system crash was seen with
out-of-memory condition when 1500+ clients tried to share this SGA even
though the system had 512GB of memory. On this server, in the worst case
scenario of all 1500 processes mapping every page from SGA would have
required 878GB+ for just the PTEs. If these PTEs could be shared, amount
of memory saved is very significant.

When PTEs are not shared between processes, each process ends up with
its own set of protection bits for each shared page. Database servers
often need to change protection bits for pages as they manipulate and
update data in the database. When changing page protection for a shared
page, all PTEs across all processes that have mapped the shared page in
need to be updated to ensure data integrity. To accomplish this, the
process making the initial change to protection bits sends messages to
every process sharing that page. All processes then block any access to
that page, make the appropriate change to protection bits, and send a
confirmation back.  To ensure data consistency, access to shared page
can be resumed when all processes have acknowledged the change. This is
a disruptive and expensive coordination process. If PTEs were shared
across processes, a change to page protection for a shared PTE becomes
applicable to all processes instantly with no coordination required to
ensure consistency. Changing protection bits across all processes
sharing database pages is a common enough operation on Oracle databases
that the cost is significant and cost goes up with the number of clients.

This is a proposal to extend the same model of page table sharing for
threads across processes. This will allow processes to tap into the
same benefits that threads get from shared page tables,

Sharing page tables across processes opens their address spaces to each
other and thus must be done carefully. This proposal suggests sharing
PTEs across processes that trust each other and have explicitly agreed
to share page tables. The proposal is to add a new flag to mmap() call -
MAP_SHARED_PT.  This flag can be specified along with MAP_SHARED by a
process to hint to kernel that it wishes to share page table entries
for this file mapping mmap region with other processes. Any other process
that mmaps the same file with MAP_SHARED_PT flag can then share the same
page table entries. Besides specifying MAP_SHARED_PT flag, the processe
must map the files at a PMD aligned address with a size that is a
multiple of PMD size and at the same virtual addresses. NOTE: This
last requirement of same virtual addresses can possibly be relaxed if
that is the consensus.

When mmap() is called with MAP_SHARED_PT flag, a new host mm struct
is created to hold the shared page tables. Host mm struct is not
attached to a process. Start and size of host mm are set to the
start and size of the mmap region and a VMA covering this range is
also added to host mm struct. Existing page table entries from the
process that creates the mapping are copied over to the host mm
struct. All processes mapping this shared region are considered
guest processes. When a guest process mmap's the shared region, a vm
flag VM_SHARED_PT is added to the VMAs in guest process. Upon a page
fault, VMA is checked for the presence of VM_SHARED_PT flag. If the
flag is found, its corresponding PMD is updated with the PMD from
host mm struct so the PMD will point to the page tables in host mm
struct.  When a new PTE is created, it is created in the host mm struct
page tables and the PMD in guest mm points to the same PTEs.


--------------------------
Evolution of this proposal
--------------------------

The original proposal -
<https://lore.kernel.org/lkml/cover.1642526745.git.khalid.aziz@oracle.com/>,
was for an mshare() system call that a donor process calls to create
an empty mshare'd region. This shared region is pgdir aligned and
multiple of pgdir size. Each mshare'd region creates a corresponding
file under /sys/fs/mshare which can be read to get information on
the region.  Once an empty region has been created, any objects can
be mapped into this region and page tables for those objects will be
shared.  Snippet of the code that a donor process would run looks
like below:

         addr = mmap((void *)TB(2), GB(512), PROT_READ | PROT_WRITE,
                         MAP_SHARED | MAP_ANONYMOUS, 0, 0);
         if (addr == MAP_FAILED)
                 perror("ERROR: mmap failed");

         err = syscall(MSHARE_SYSCALL, "testregion", (void *)TB(2),
			GB(512), O_CREAT|O_RDWR|O_EXCL, 600);
         if (err < 0) {
                 perror("mshare() syscall failed");
                 exit(1);
         }

         strncpy(addr, "Some random shared text",
			sizeof("Some random shared text"));


Snippet of code that a consumer process would execute looks like:

         fd = open("testregion", O_RDONLY);
         if (fd < 0) {
                 perror("open failed");
                 exit(1);
         }

         if ((count = read(fd, &mshare_info, sizeof(mshare_info)) > 0))
                 printf("INFO: %ld bytes shared at addr %lx \n",
				mshare_info[1], mshare_info[0]);
         else
                 perror("read failed");

         close(fd);

         addr = (char *)mshare_info[0];
         err = syscall(MSHARE_SYSCALL, "testregion", (void *)mshare_info[0],
			mshare_info[1], O_RDWR, 600);
         if (err < 0) {
                 perror("mshare() syscall failed");
                 exit(1);
         }

         printf("Guest mmap at %px:\n", addr);
         printf("%s\n", addr);
	printf("\nDone\n");

         err = syscall(MSHARE_UNLINK_SYSCALL, "testregion");
         if (err < 0) {
                 perror("mshare_unlink() failed");
                 exit(1);
         }


This proposal evolved into completely file and mmap based API -
<https://lore.kernel.org/lkml/cover.1656531090.git.khalid.aziz@oracle.com/>.
This new API looks like below:

1. Mount msharefs on /sys/fs/mshare -
	mount -t msharefs msharefs /sys/fs/mshare

2. mshare regions have alignment and size requirements. Start
    address for the region must be aligned to an address boundary and
    be a multiple of fixed size. This alignment and size requirement
    can be obtained by reading the file /sys/fs/mshare/mshare_info
    which returns a number in text format. mshare regions must be
    aligned to this boundary and be a multiple of this size.

3. For the process creating mshare region:
	a. Create a file on /sys/fs/mshare, for example -
		fd = open("/sys/fs/mshare/shareme",
				O_RDWR|O_CREAT|O_EXCL, 0600);
	
	b. mmap this file to establish starting address and size -
		mmap((void *)TB(2), BUF_SIZE, PROT_READ | PROT_WRITE,
                         MAP_SHARED, fd, 0);

	c. Write and read to mshared region normally.

4. For processes attaching to mshare'd region:
	a. Open the file on msharefs, for example -
		fd = open("/sys/fs/mshare/shareme", O_RDWR);

	b. Get information about mshare'd region from the file:
		struct mshare_info {
			unsigned long start;
			unsigned long size;
		} m_info;

		read(fd, &m_info, sizeof(m_info));
	
	c. mmap the mshare'd region -
		mmap(m_info.start, m_info.size,
			PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0);

5. To delete the mshare region -
		unlink("/sys/fs/mshare/shareme");


Further discussions over mailing lists and LSF/MM resulted in eliminating
msharefs and making this entirely mmap based -
<https://lore.kernel.org/lkml/cover.1682453344.git.khalid.aziz@oracle.com/>.
With this change, if two processes map the same file with same
size, PMD aligned address, same virtual address and both specify
MAP_SHARED_PT flag, they start sharing PTEs for the file mapping.
These changes eliminate support for any arbitrary objects being
mapped in mshare'd region. The last implementation required sharing
minimum PMD sized chunks across processes. These changes were
significant enough to make this proposal distinct enough for me to
use a new name - ptshare.


----------
What next?
----------

There were some more discussions on this proposal while I was on
leave for a few months. There is enough interest in this feature to
continue to refine this. I will refine the code further but before
that I want to make sure we have a common understanding of what this
feature should do.

As a result of many discussions, a new distinct version of
original proposal has evolved. Which one do we agree to continue
forward with - (1) current version which restricts sharing to PMD sized
and aligned file mappings only, using just a new mmap flag
(MAP_SHARED_PT), or (2) original version that creates an empty page
table shared mshare region using msharefs and mmap for arbitrary
objects to be mapped into later?

Thanks,
Khalid