Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Song Liu <songliubraving@fb.com>
To:     Collin Fijalkovich <cfijalkovich@google.com>
CC:     Suren Baghdasaryan <surenb@google.com>,
        "hridya@google.com" <hridya@google.com>,
        "kaleshsingh@google.com" <kaleshsingh@google.com>,
        "hughd@google.com" <hughd@google.com>,
        "timmurray@google.com" <timmurray@google.com>,
        "william.kucharski@oracle.com" <william.kucharski@oracle.com>,
        "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
        "willy@infradead.org" <willy@infradead.org>,
        Alexander Viro <viro@zeniv.linux.org.uk>,
        "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: [PATCH v2] mm, thp: Relax the VM_DENYWRITE constraint on
 file-backed THPs
Thread-Topic: [PATCH v2] mm, thp: Relax the VM_DENYWRITE constraint on
 file-backed THPs
Thread-Index: AQHXKnknEhLdpDmfP0aTj6hRIe6caqq/wmuA
Date:   Thu, 22 Apr 2021 00:08:58 +0000
Message-ID: <16D17333-1BF1-412C-8EE9-95CF61F741B0@fb.com>
References: <20210406000930.3455850-1-cfijalkovich@google.com>
In-Reply-To: <20210406000930.3455850-1-cfijalkovich@google.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <D395478EFEFE084A9DA631861376CDC7@namprd15.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: fd0e5b8e-10c3-4441-10ec-08d90522d352
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Apr 2021 00:08:58.0624
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Xces6YhD2jUbMMQ2XH5wJCTZrD/43PUkvr120GlzKp559CtWCebIXg+oRKu2h18sO63x7cBRRCwnungv9P2Raw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY3PR15MB4897
X-OriginatorOrg: fb.com
X-Proofpoint-GUID: iGpGaTnDlo9Kqe0WQsRnDZMbi8Oydbqe
X-Proofpoint-ORIG-GUID: iGpGaTnDlo9Kqe0WQsRnDZMbi8Oydbqe
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.761
 definitions=2021-04-21_08:2021-04-21,2021-04-21 signatures=0
X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 clxscore=1011
 adultscore=0 phishscore=0 malwarescore=0 spamscore=0 mlxscore=0
 bulkscore=0 lowpriorityscore=0 suspectscore=0 impostorscore=0
 priorityscore=1501 mlxlogscore=999 classifier=spam adjust=0 reason=mlx
 scancount=1 engine=8.12.0-2104060000 definitions=main-2104210159
X-FB-Internal: deliver
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On Apr 5, 2021, at 5:09 PM, Collin Fijalkovich <cfijalkovich@google.com> =
wrote:
>=20
> Transparent huge pages are supported for read-only non-shmem files,
> but are only used for vmas with VM_DENYWRITE. This condition ensures that
> file THPs are protected from writes while an application is running
> (ETXTBSY).  Any existing file THPs are then dropped from the page cache
> when a file is opened for write in do_dentry_open(). Since sys_mmap
> ignores MAP_DENYWRITE, this constrains the use of file THPs to vmas
> produced by execve().
>=20
> Systems that make heavy use of shared libraries (e.g. Android) are unable
> to apply VM_DENYWRITE through the dynamic linker, preventing them from
> benefiting from the resultant reduced contention on the TLB.
>=20
> This patch reduces the constraint on file THPs allowing use with any
> executable mapping from a file not opened for write (see
> inode_is_open_for_write()). It also introduces additional conditions to
> ensure that files opened for write will never be backed by file THPs.
>=20
> Restricting the use of THPs to executable mappings eliminates the risk th=
at
> a read-only file later opened for write would encounter significant
> latencies due to page cache truncation.
>=20
> The ld linker flag '-z max-page-size=3D(hugepage size)' can be used to
> produce executables with the necessary layout. The dynamic linker must
> map these file's segments at a hugepage size aligned vma for the mapping =
to
> be backed with THPs.
>=20
> Comparison of the performance characteristics of 4KB and 2MB-backed
> libraries follows; the Android dex2oat tool was used to AOT compile an
> example application on a single ARM core.
>=20
> 4KB Pages:
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20
> count              event_name            # count / runtime
> 598,995,035,942    cpu-cycles            # 1.800861 GHz
> 81,195,620,851    raw-stall-frontend    # 244.112 M/sec
> 347,754,466,597    iTLB-loads            # 1.046 G/sec
>  2,970,248,900    iTLB-load-misses      # 0.854122% miss rate
>=20
> Total test time: 332.854998 seconds.
>=20
> 2MB Pages:
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>=20
> count              event_name            # count / runtime
> 592,872,663,047    cpu-cycles            # 1.800358 GHz
> 76,485,624,143    raw-stall-frontend    # 232.261 M/sec
> 350,478,413,710    iTLB-loads            # 1.064 G/sec
>    803,233,322    iTLB-load-misses      # 0.229182% miss rate
>=20
> Total test time: 329.826087 seconds
>=20
> A check of /proc/$(pidof dex2oat64)/smaps shows THPs in use:
>=20
> /apex/com.android.art/lib64/libart.so
> FilePmdMapped:      4096 kB
>=20
> /apex/com.android.art/lib64/libart-compiler.so
> FilePmdMapped:      2048 kB
>=20
> Signed-off-by: Collin Fijalkovich <cfijalkovich@google.com>

Acked-by: Song Liu <song@kernel.org>

> ---
> Changes v1 -> v2:
> * commit message 'non-shmem filesystems' -> 'non-shmem files'
> * Add performance testing data to commit message
>=20
> fs/open.c       | 13 +++++++++++--
> mm/khugepaged.c | 16 +++++++++++++++-
> 2 files changed, 26 insertions(+), 3 deletions(-)
>=20
> diff --git a/fs/open.c b/fs/open.c
> index e53af13b5835..f76e960d10ea 100644
> --- a/fs/open.c
> +++ b/fs/open.c
> @@ -852,8 +852,17 @@ static int do_dentry_open(struct file *f,
> 	 * XXX: Huge page cache doesn't support writing yet. Drop all page
> 	 * cache for this file before processing writes.
> 	 */
> -	if ((f->f_mode & FMODE_WRITE) && filemap_nr_thps(inode->i_mapping))
> -		truncate_pagecache(inode, 0);
> +	if (f->f_mode & FMODE_WRITE) {
> +		/*
> +		 * Paired with smp_mb() in collapse_file() to ensure nr_thps
> +		 * is up to date and the update to i_writecount by
> +		 * get_write_access() is visible. Ensures subsequent insertion
> +		 * of THPs into the page cache will fail.
> +		 */
> +		smp_mb();
> +		if (filemap_nr_thps(inode->i_mapping))
> +			truncate_pagecache(inode, 0);
> +	}
>=20
> 	return 0;
>=20
> diff --git a/mm/khugepaged.c b/mm/khugepaged.c
> index a7d6cb912b05..4c7cc877d5e3 100644
> --- a/mm/khugepaged.c
> +++ b/mm/khugepaged.c
> @@ -459,7 +459,8 @@ static bool hugepage_vma_check(struct vm_area_struct =
*vma,
>=20
> 	/* Read-only file mappings need to be aligned for THP to work. */
> 	if (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS) && vma->vm_file &&
> -	    (vm_flags & VM_DENYWRITE)) {
> +	    !inode_is_open_for_write(vma->vm_file->f_inode) &&
> +	    (vm_flags & VM_EXEC)) {
> 		return IS_ALIGNED((vma->vm_start >> PAGE_SHIFT) - vma->vm_pgoff,
> 				HPAGE_PMD_NR);
> 	}
> @@ -1872,6 +1873,19 @@ static void collapse_file(struct mm_struct *mm,
> 	else {
> 		__mod_lruvec_page_state(new_page, NR_FILE_THPS, nr);
> 		filemap_nr_thps_inc(mapping);
> +		/*
> +		 * Paired with smp_mb() in do_dentry_open() to ensure
> +		 * i_writecount is up to date and the update to nr_thps is
> +		 * visible. Ensures the page cache will be truncated if the
> +		 * file is opened writable.
> +		 */
> +		smp_mb();
> +		if (inode_is_open_for_write(mapping->host)) {
> +			result =3D SCAN_FAIL;
> +			__mod_lruvec_page_state(new_page, NR_FILE_THPS, -nr);
> +			filemap_nr_thps_dec(mapping);
> +			goto xa_locked;
> +		}
> 	}
>=20
> 	if (nr_none) {
> --=20
> 2.31.0.208.g409f899ff0-goog
>=20