Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
To:     Matthew Wilcox <willy@infradead.org>
Cc:     Khalid Aziz <khalid.aziz@oracle.com>,
        "Longpeng (Mike, Cloud Infrastructure Service Product Dept.)" 
        <longpeng2@huawei.com>, Steven Sistare <steven.sistare@oracle.com>,
        Anthony Yznaga <anthony.yznaga@oracle.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "Gonglei (Arei)" <arei.gonglei@huawei.com>
References: <43471cbb-67c6-f189-ef12-0f8302e81b06@oracle.com>
 <a1dbf12e-9949-109e-122c-ba7ba609801b@huawei.com>
 <YOubKmDwxMIvdAed@casper.infradead.org>
 <a94973ab83ce48bd85c91397f82d7915@huawei.com>
 <55720e1b39cff0a0f882d8610e7906dc80ea0a01.camel@oracle.com>
 <db2b7337-4c6b-4e4b-71d3-dc4940353498@redhat.com>
 <YRpVHnr55LpQQvTb@casper.infradead.org>
 <ca2d4ea4-e875-475a-6094-1ac58bc0b544@redhat.com>
 <YRpeHnP7QDNJRA8Y@casper.infradead.org>
 <88884f55-4991-11a9-d330-5d1ed9d5e688@redhat.com>
 <YRpo4EAJSkY7hI7Q@casper.infradead.org>
From:   David Hildenbrand <david@redhat.com>
Organization: Red Hat
Subject: Re: [RFC PATCH 0/5] madvise MADV_DOEXEC
Message-ID: <40bad572-501d-e4cf-80e3-9a8daa98dc7e@redhat.com>
Date:   Mon, 16 Aug 2021 16:10:28 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
In-Reply-To: <YRpo4EAJSkY7hI7Q@casper.infradead.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Precedence: bulk

>>> Until recently, the CPUs only having 4 1GB TLB entries.  I'm sure we
>>> still have customers using that generation of CPUs.  2MB pages perform
>>> better than 1GB pages on the previous generation of hardware, and I
>>> haven't seen numbers for the next generation yet.
>>
>> I read that somewhere else before, yet we have heavy 1 GiB page users,
>> especially in the context of VMs and DPDK.
> 
> I wonder if those users actually benchmarked.  Or whether the memory
> savings worked out so well for them that the loss of TLB performance
> didn't matter.

These applications are extremely performance sensitive (i.e., RT 
workloads), that's why I'm wondering. I recall that they are most 
certainly using more than 4 GiB memory in real applications.

E.g., the doc [1] even has a note that "For 64-bit applications, it is 
recommended to use 1 GB hugepages if the platform supports them."


[1] https://doc.dpdk.org/guides-16.04/linux_gsg/sys_reqs.html

> 
>> So, it only works for hugetlbfs in case uffd is not in place (-> no
>> per-process data in the page table) and we have an actual shared mappings.
>> When unsharing, we zap the PUD entry, which will result in allocating a
>> per-process page table on next fault.
> 
> I think uffd was a huge mistake.  It should have been a filesystem
> instead of a hack on the side of anonymous memory.

Yes it was. Especially, looking at all the special-casing, for example, 
even in mm/pagewalk.c.

> 
>> I will rephrase my previous statement "hugetlbfs just doesn't raise these
>> problems because we are special casing it all over the place already". For
>> example, not allowing to swap such pages. Disallowing MADV_DONTNEED. Special
>> hugetlbfs locking.
> 
> Sure, that's why I want to drag this feature out of "oh this is a
> hugetlb special case" and into "this is something Linux supports".

I would have understood the move to optimize SHMEM internally - similar 
to how we seem to optimize hugetlbfs SHMEM right now internally. 
(although sharing page tables for shmem can still be quite tricky)

I did not follow why we have to play games with MAP_PRIVATE, and having 
private anonymous pages shared between processes that don't COW, 
introducing new syscalls etc.

-- 
Thanks,

David / dhildenb