Received-SPF: pass (google.com: domain of linux-kernel+bounces-131328-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Thu, 4 Apr 2024 08:46:52 -0300
From: Jason Gunthorpe <jgg@nvidia.com>
To: Christophe Leroy <christophe.leroy@csgroup.eu>
Cc: Andrew Morton <akpm@linux-foundation.org>, Peter Xu <peterx@redhat.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>,
	"linuxppc-dev@lists.ozlabs.org" <linuxppc-dev@lists.ozlabs.org>
Subject: Re: [RFC PATCH 1/8] mm: Provide pagesize to pmd_populate()
Message-ID: <20240404114652.GH1723999@nvidia.com>
References: <cover.1711377230.git.christophe.leroy@csgroup.eu>
 <54d78f1b7e7f1c671e40b7c0c637380bcb834326.1711377230.git.christophe.leroy@csgroup.eu>
 <20240325161919.GD6245@nvidia.com>
 <6e16e042-0143-4a52-b4b7-09cf0022bc3e@csgroup.eu>
 <20240326150118.GI6245@nvidia.com>
 <9703878c-c0b0-48ff-a356-d43e8f7391f3@csgroup.eu>
 <20240327165754.GM946323@nvidia.com>
 <dee0b89b-8daf-4003-b26f-f612b14012e0@csgroup.eu>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <dee0b89b-8daf-4003-b26f-f612b14012e0@csgroup.eu>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?h8seUJdKwMtAS1IeSuM0VDx1Ky8IF28eKjAYNE0xQLuSCa6dI9HDKbuk0p7x?=
 =?us-ascii?Q?kacRGlUqz8G6B9IUy/AE1UYSPEAXm89UxCfyfI/E63ffGxOBFDUb5qVdoKVk?=
 =?us-ascii?Q?BqHYNrU3eEiOQ/YCv2E7UeBKGpWmlZbIZw9UqS2Aol65kA1FH/d0gQ7GRcoW?=
 =?us-ascii?Q?jq3L6LIClJnwr4+Jtxr7BkgB8xvBPj4ynpk9nXjStzaD1uRpvTRU3pJmYJDO?=
 =?us-ascii?Q?cqUCIAeNL20FEpDNr8K+nzvIE4ZO0D6WFnC51E5lX7ouNE9L+MUbQIdT11BK?=
 =?us-ascii?Q?RvXjxdtKF4s77rphsXoTcfXb43pHgD4PknlYVBMZ2zhWI6YIuIzVhTmkbH0f?=
 =?us-ascii?Q?PaklWN4r2m23eM06RypdI35Dc5VaXFoETJRmgKcfch4A0lGrYGJWaDPc2vxa?=
 =?us-ascii?Q?zEOxeHpLyGUkD+lSG8qFJjurOK1QHeQOJCOs3hP6I9fz0zsuH1QovEdkRKms?=
 =?us-ascii?Q?ZEPJyvBZcrovhmrWfK0u93unTOwuUMYQm2jLC6HFlMaD15ylYTb8/0Pgg9k+?=
 =?us-ascii?Q?vaA/ER/HfsdHk9n/cWjGHdQTQKXQS5vJt1ZUFk96ZfKyLjnv2Obd1uGS3upp?=
 =?us-ascii?Q?AbEcXTb9uC2aKMZAe0BaEiAq8kVEd3Fi4gM8iybtIThKhxkkgYf8iAbxzxLn?=
 =?us-ascii?Q?WiNBZ0Ji0bCBTSoQnZyoDkTnzxeS+lCzBPiUIFNRuaPxlhmszasWCTSOANhU?=
 =?us-ascii?Q?8As7Wmmq+VNwmcgyKJ5fMsmugJOFP+atZ6oWQ5BMnDXWPjJM/chcZj6k0fOG?=
 =?us-ascii?Q?DRJaQsGNoBBC1pzScgNXBNoJdS6xiPen3Fc+tU6Fdzs4nPOl6cqH17tsAGW4?=
 =?us-ascii?Q?a6BrjnJubV5sW2fCnEUYqUdQea89tQPG76/dsZ/I5q+GhRtvTTNS4YS7I9dz?=
 =?us-ascii?Q?dOfEl6vASA8VJ98nZL31QhnFv+RXWpDbIbdxsy2QzUalQORHKc7V/Jkn+RdH?=
 =?us-ascii?Q?mng7P98SWRgDtEEhH3TO7+qARJYAx06KEpS/XKIOy9N//zRySGWAinfwfyNf?=
 =?us-ascii?Q?4R5SD4ED16YaUWpUgeLawm6YWNUCjGTkrutR7izY/p+0GDipLLsHVqHevZSH?=
 =?us-ascii?Q?7LIOEa/VNKjq1kudLMSOI0WjHoJogjwQBWrCo3g2kX4AXsxn2m1bZ7qk50zm?=
 =?us-ascii?Q?2+hof2JJ2jNHFNc/y0qnGdwePX+Oc+t9R4SOmFG32JwwgoWi8UC/GsVdIS67?=
 =?us-ascii?Q?ZJwThMLiICJ03gQUIx5Sy//kknaxmQD7ujCXgRS14ifshTQwla0SCZkf0nfT?=
 =?us-ascii?Q?NuknwE9zR0FUXr3n4AOeqlh9bQzbSl8j+ffTJzMkbtf0bZRTXWcqvas4wJgY?=
 =?us-ascii?Q?ydT/m5DcJiNT5x3PMVAWY5EmKzCjfVagX3x60kipXSL25jdey8B+D60Dk66A?=
 =?us-ascii?Q?z78ONkAsDp1TomO2zXmC8CMD1LPKfPnxC4M5BwroirPBwativxRJVjoB30w0?=
 =?us-ascii?Q?Y7s3r/td+eqH7Py55TxbnTVfCagH1icBhVHd73JOz9VZKwftG7bX4xTSGfX7?=
 =?us-ascii?Q?GncfPOPScARf+vXgPVlhAKeT4YM7kEKI90ultnkxTT3noM8m4y3F1XTpK7NB?=
 =?us-ascii?Q?WQ5CuxXPsii9u6NbFaPAr2SmfjDecdSKaQId07Pw?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b0d3243b-4ac3-4cce-bd86-08dc549cecc3
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Apr 2024 11:46:54.5446
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: R6caJb69MrWzhHBQAny8WVnrggL9DzUPmCGUEFtgFEBrl/ds/OxNTD2n01/AkwQy
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB8181

On Wed, Apr 03, 2024 at 06:24:38PM +0000, Christophe Leroy wrote:
> > If it is a software walker there might be value in just aligning to
> > the contig pte scheme in all levels and forgetting about the variable
> > size page table levels. That quarter page stuff is a PITA to manage
> > the memory allocation for on PPC anyhow..
> 
> Looking one step further, into nohash/32, I see a challenge: on that 
> platform, a PTE is 64 bits while a PGD/PMD entry is 32 bits. It is 
> therefore not possible as such to do PMD leaf or cont-PMD leaf.

Hmm, maybe not, I have a feeling you can hide this detail in the
pmd_offset routine if you pass in the PGD information too.

> - Double the size of PGD/PMD entries, but then we loose atomicity when 
> reading or writing an entry, could this be a problem ?

How does the 64 bit PTE work then? We have ignored this bug on x86 32
bit, but there is a general difficult race with 64 bit atomicity on 32
bit CPUs in the page tables.

Ideally you'd have 64 bit entries at the PMD level that encode the
page size the same as the PTE level. So you hit any level and you know
your size. This is less memory efficient (though every other arch
tolerates this) in general cases.

Can you throw away some bits of PA in the 32 bit entries to signal a
size?

> - Do as for the 8xx, ie go down to PTEs even for pages greater than 4M.

Aside from the memory waste, this is the most logical thing, go down
far enough that you can encode the desired page size in the PTE and
use the contig PTE scheme.

Jason