Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Message-ID: <fe797b58-bd46-754d-17d2-a19e7ce1bf40@oracle.com>
Date:   Mon, 11 Apr 2022 13:52:31 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.7.0
Subject: Re: [PATCH v1 00/14] Add support for shared PTEs across processes
Content-Language: en-US
To:     Matthew Wilcox <willy@infradead.org>
Cc:     akpm@linux-foundation.org, aneesh.kumar@linux.ibm.com,
        arnd@arndb.de, 21cnbao@gmail.com, corbet@lwn.net,
        dave.hansen@linux.intel.com, david@redhat.com,
        ebiederm@xmission.com, hagen@jauu.net, jack@suse.cz,
        keescook@chromium.org, kirill@shutemov.name, kucharsk@gmail.com,
        linkinjeon@kernel.org, linux-fsdevel@vger.kernel.org,
        linux-kernel@vger.kernel.org, linux-mm@kvack.org,
        longpeng2@huawei.com, luto@kernel.org, markhemm@googlemail.com,
        pcc@google.com, rppt@kernel.org, sieberf@amazon.com,
        sjpark@amazon.de, surenb@google.com, tst@schoebel-theuer.de,
        yzaikin@google.com
References: <cover.1649370874.git.khalid.aziz@oracle.com>
 <YlRnPstOywJzxUib@casper.infradead.org>
From:   Khalid Aziz <khalid.aziz@oracle.com>
In-Reply-To: <YlRnPstOywJzxUib@casper.infradead.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aDc3MlMxNU1qeUVzbnA1eE8wbmJrdmhOdVFJYUplUXk4NitsZU0wMnNBcVpr?=
 =?utf-8?B?NDlqdDVoeVVGK29VM2R5YWdPOVJQWXNVdTZSMEdqa0tRNE1ScjdKS3VTMXVs?=
 =?utf-8?B?dUdwcUY1OXQzTkQwcm84RS94cmh5cFUwV1ZEcmpmeWhma25mOGJXVjFBY3R3?=
 =?utf-8?B?azgvNEdDN2kvRjN1cmhMTEV4dEx1cGVSclhyU2ZJMThES0tWeUt2OEo3ZEt4?=
 =?utf-8?B?YUV5NEhTSzZtQU1OazN4Unp5SGdQRm8zb2VKL3d1L1pqVzRqRGR1SjB1VEFD?=
 =?utf-8?B?TFJmNCs2Q2ZkN2F6NGtNekl2NnVVSGVZL0xKeldPbmVBTldiaXdTWVYyUTlm?=
 =?utf-8?B?SHNma3Bva1ZRY0oxMTRwalBkTGgyTHhjMHk2ZlY1Nm1OUzEvYmNSRkdLV1Fy?=
 =?utf-8?B?ZjBBN2J0TVBWNCtxSWdOS0RjYjFpbDQ2cXp3Vmg1UEdudHdhTHB6a1VWdFJY?=
 =?utf-8?B?SkZZeHozaHpCaDh0OERBSWxnRzlyUTB5U2w5VmhjVm9iQk92YWFEUW9GSUFF?=
 =?utf-8?B?Rk5CL0xxd3NkUXRrdWpzRE9ZN0NKYmxmWTdaWVp6bFlkSmFoRElXVVRtSXZC?=
 =?utf-8?B?MHdSa0praHRHcVdTY3c0TnhYZXhCOEJRdmFMbVdLdzBpa21oUWZjMHVlZElM?=
 =?utf-8?B?Qm1ZVnJrWVVGSklEaGtLbDBPWlJDb2ovTzllZjQ1a3BJL1BmekJlUFJTcDh3?=
 =?utf-8?B?VVIvdnJ2dXY5cmUyM0x2S2hIemdnWXVYd0xSOWxxdW56YUl2OHB6RGlnRjUv?=
 =?utf-8?B?SVhPWXJNSitPbVp5cUlaeGVtVDVzc0lGVUJLWFpLdWVNVzUwOGNQU21CMEt2?=
 =?utf-8?B?TEVkaHVLZzJqdnBPTnZtTkRHNDBnaS9KdFYxazRGZmF0ZjRaSWFnZ1JOWGxq?=
 =?utf-8?B?ZXF2TnlzcW9mK1NoVy9vbURRTndnR3pnTFp6UUxjZjFCRm4yMFRYT2JYQmpM?=
 =?utf-8?B?dzI3dURWc2Q1OFNCc0VMck91Q2djMis1YXFzTit4MnhmdHhCbG9kL3h0YmRI?=
 =?utf-8?B?akNpSk9sQnNoOXRQeTh1Q3gyY2szajFlK2Q3RC9DQmo2amd4eXhxUnliN0Fs?=
 =?utf-8?B?NVlmMEtFNXFkVFljcmhiRFdTRFhIdGMyR3A1UFB0VWUra1RvRW5rVmdSYnZs?=
 =?utf-8?B?bk1NalJzVE9ZWG16VjZ0WnRBbFlId1lzeUF5L3dQYWpqQmIyNUpobHlIUkps?=
 =?utf-8?B?WWNxY3d6Sms4U2xBQzVPZnBpbjVsMUZUYXJYVTlaY0RhTU5GL1BKcCtvSWNT?=
 =?utf-8?B?cGxvVkxFdDI5TW5xb1NTZ0tmL1crL0hSOVQxV29pTVYxd1RnR25Ib0xXLzVI?=
 =?utf-8?B?NUZjbXBiQWlWMXBzQkJzamQzbXF3eHJSY2Z0TnRjZGJrM0h4dmlSQ3llbi9B?=
 =?utf-8?B?K1FOZFljTnVKMTgySjhRYjBjcnZZa2k2RmljYnZMbGg3ZGdyaEoxU00yZ1cz?=
 =?utf-8?B?UVdocCt4aWdiUkEyNjQ3TkJ3Rk1QSWQraDlnc3plOXZDN29VbXNZSVA0dlFY?=
 =?utf-8?B?anRLTDdpT1VuUTRTRTZCemhVSWJ0UHBHREJkSjlLcmdzdXVhNmN4WjZrOUVQ?=
 =?utf-8?B?OXRRdVI5VGl3WE82NlA2bTgrZC9SazFhc1BpZENYRFhVNEZEczhXUExtYzlw?=
 =?utf-8?B?Tk5UdWFkNGpCVlAvVFpUNUFxTlE0a3RsdElLb21DcnRQZlNhR1ZvTXI5eHMw?=
 =?utf-8?B?cjhKQ01qNUM5bnhSUHhwL1BISm93UG5LckdLcUdkSXBxN3lRMEtzNUJzSlZT?=
 =?utf-8?B?bUJ5TExVM1M4QTArL25XRDI1M290MW12VkU2VmU1WW1IZ2hKUHZkb1ZlRHpI?=
 =?utf-8?B?NFY5dHo2dWNjcWcwMzI1YjJsR2hyVEhaSjRJM3dpbFpTVlhiT2JlVGVyc3V1?=
 =?utf-8?B?TVBObUl0Q0l5d2FaSVdJOUgxbU8weFVMMURzZ1FPckhyMjQ3ZTY1U1Y2S1NR?=
 =?utf-8?B?S2REdEM1S3V0ZGNBNHpMQjVOZkxNOG9tZ3I0VTVGOHh5VXNaN3ZhK2xhMnc1?=
 =?utf-8?B?aXl4ZDNlQ1Z0TGR1cVNlZDdUSmRVOGZFNTNETUlIWUU5M3ZvRVQvQkRzRjZX?=
 =?utf-8?B?djBUVEMvMFBTT01rN0VrSDd0cUxqblR4TjBramJleUlFTVdRendjNXN5Sk1m?=
 =?utf-8?B?WHdnZktJMGxmLzE4dXFPZzRMQ2VGTFRoWUo4QUQ2bk5JS0VpQmFLWlNaVHdQ?=
 =?utf-8?B?c1dDcVRxckI2SHB0RkV2N05TSzdVSzQrUTltY2FiUDV6QjNMRTcwZzZrMExQ?=
 =?utf-8?B?T1F1Z3lhdUFLeXB2WFBwQU40eXJrTFpPdWVTbHRlN01YTis2d2FFNU43SlR5?=
 =?utf-8?B?dXVFUXlIVjJMaTVnVGsxeXRBWVhXZDhXSlkvdEFQd0FEUXlWa01DbTdoTUR5?=
 =?utf-8?Q?vS/2Az6ZVlt0b/II=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5e9d910b-f63a-4edf-6279-08da1bf4d418
X-MS-Exchange-CrossTenant-AuthSource: BN8PR10MB3220.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Apr 2022 19:52:37.3028
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8H9ZYs+OjDHNp4/rVIgsobbdgKI8AmVtaKLQ+smLxlaifuIJhAxNkNp2d0byDXEs33riQv5fc7klXcG4inuFcg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1001MB2159
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.425,18.0.858
 definitions=2022-04-11_08:2022-04-11,2022-04-11 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=988 malwarescore=0
 mlxscore=0 phishscore=0 suspectscore=0 spamscore=0 adultscore=0
 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2202240000 definitions=main-2204110110
X-Proofpoint-ORIG-GUID: YW4OHrEMqfoKlDqNCmFW0yIangeyi5jI
X-Proofpoint-GUID: YW4OHrEMqfoKlDqNCmFW0yIangeyi5jI
X-Spam-Status: No, score=-3.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,RDNS_NONE,SPF_HELO_NONE,
	T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 4/11/22 11:37, Matthew Wilcox wrote:
> On Mon, Apr 11, 2022 at 10:05:44AM -0600, Khalid Aziz wrote:
>> Page tables in kernel consume some of the memory and as long as number
>> of mappings being maintained is small enough, this space consumed by
>> page tables is not objectionable. When very few memory pages are
>> shared between processes, the number of page table entries (PTEs) to
>> maintain is mostly constrained by the number of pages of memory on the
>> system. As the number of shared pages and the number of times pages
>> are shared goes up, amount of memory consumed by page tables starts to
>> become significant.
> 
> All of this is true.  However, I've found a lot of people don't see this
> as compelling.  I've had more success explaining this from a different
> direction:
> 
> --- 8< ---
> 
> Linux supports processes which share all of their address space (threads)
> and processes that share none of their address space (tasks).  We propose
> a useful intermediate model where two or more cooperating processes
> can choose to share portions of their address space with each other.
> The shared portion is referred to by a file descriptor which processes
> can choose to attach to their own address space.
> 
> Modifications to the shared region affect all processes sharing
> that region, just as changes by one thread affect all threads in a
> multithreaded program.  This implies a certain level of trust between
> the different processes (ie malicious processes should not be allowed
> access to the mshared region).
> 
> --- 8< ---
> 
> Another argument that MM developers find compelling is that we can reduce
> some of the complexity in hugetlbfs where it has the ability to share
> page tables between processes.

This all sounds reasonable.

> 
> One objection that was raised is that the mechanism for starting the
> shared region is a bit clunky.  Did you investigate the proposed approach
> of creating an empty address space, attaching to it and using an fd-based
> mmap to modify its contents?

I want to make sure I understand this correctly. In the example I gave, the process creating mshare'd region maps in the 
address space first possibly using mmap(). It then calls mshare() to share this already-mapped region. Are you 
suggesting that the process be able to call mshare() before mapping in address range and then map things into that 
address range later? If yes, it is my intent to support that after the initial implementation as expansion of original 
concept.

> 
>> int mshare_unlink(char *name)
>>
>> A shared address range created by mshare() can be destroyed using
>> mshare_unlink() which removes the  shared named object. Once all
>> processes have unmapped the shared object, the shared address range
>> references are de-allocated and destroyed.
>>
>> mshare_unlink() returns 0 on success or -1 on error.
> 
> Can you explain why this is a syscall instead of being a library
> function which does
> 
> 	int dirfd = open("/sys/fs/mshare");
> 	err = unlinkat(dirfd, name, 0);
> 	close(dirfd);
> 	return err;

mshare_unlink can be simple unlink on the file in msharefs. API will be asymmetrical in that creating mshare'd region is 
a syscall while tearing it down is a file op. I don't mind saving a syscall slot. Would you prefer it that way?

> 
> Does msharefs support creating directories, so that we can use file
> permissions to limit who can see the sharable files?  Or is it strictly
> a single-level-deep hierarchy?
> 

For now msharefs is single-level-deep. It can be expanded to support directories to limit visibility of filenames. Would 
you prefer to see it support directories from the beginning or can that be a future expansion of this feature?

Thanks,
Khalid