Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 23 Jun 2022 17:47:12 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Alex Williamson <alex.williamson@redhat.com>,
        Alistair Popple <apopple@nvidia.com>
Cc:     David Hildenbrand <david@redhat.com>, akpm@linux-foundation.org,
        minchan@kernel.org, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org, paulmck@kernel.org,
        jhubbard@nvidia.com, joaodias@google.com
Subject: Re: [PATCH] mm: Re-allow pinning of zero pfns
Message-ID: <20220623204712.GG4147@nvidia.com>
References: <165490039431.944052.12458624139225785964.stgit@omen>
 <bb120b4a-e6f6-de81-35f0-9803acf9b0be@redhat.com>
 <20220615155659.GA7684@nvidia.com>
 <cb7efb0f-5537-5ce4-7aec-bb10ea81d5de@redhat.com>
 <20220623142139.462a0841.alex.williamson@redhat.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220623142139.462a0841.alex.williamson@redhat.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?14UumVPcMZLUBCUMD3KcIUK+UM6XvUuiANBNSBIGdMLiwct0O5dkU9rEl0az?=
 =?us-ascii?Q?4jg8Zf8z6B0r+TCMRkpSwKMUAlQaBckyJfEQ+qV9/hSAqx2hSLIsQSRUolTN?=
 =?us-ascii?Q?JsrXwIGMNE3hQIPnawUWBwKmnqsjE2w8Gcp5yTkkXRX152uZzNNSBl1Sjcry?=
 =?us-ascii?Q?yKnE2MpkuPz1vL5n3XB2XACoLo0ZjFSPKbp7H38nS5SKAqR9khecoj0kLwTC?=
 =?us-ascii?Q?k8lJJ//JBM9GyAf+jQhwYE8rGI2WRzVpF8SH4Bg2d06uatKI2Q6ZojZszFK3?=
 =?us-ascii?Q?Spo+XCZJHuaobrCF8EeuqhN68BNN4bhHlJSX6Yis4vxfVM7MTc8Bggb90kz2?=
 =?us-ascii?Q?+KvrBmAs05wiPyJJififXLzx6JLU0/zBo1PGqakj0aapSwa2KxwIW54zprK7?=
 =?us-ascii?Q?/xMOiQ5XKsmrELirMwUkDoLotdwCB/m9TaHRA0O545kwXjzJ0nHpFiRigRu0?=
 =?us-ascii?Q?4JxUzJBS5skA4Ty5apdz1uekrfsDTkYw/KD5B9i9RxXD3ORUxAynPUjd3+yK?=
 =?us-ascii?Q?j5PcJDIqDRy7BdwiWVewxn6ONQIBSRKCP1hQiI4QN6hrlinn4UwgHVVKOSOH?=
 =?us-ascii?Q?sc7UgxnecQcGrqGoPjJx7lsdIr0+jJcjlb1GzSgTWBm9hDU0LyxPhqXVXsRb?=
 =?us-ascii?Q?UsZMwHTl2Cqc395g7/K+DWLwBdfGsDYiMqx/i1k8F+Ozb9/xYoj5jcVsduZ/?=
 =?us-ascii?Q?5FRzTnXRxM/Jh2WYlCQUmiPsYCX8iT32hyYOje9pnu2J97fGs9vOtaigOQ4U?=
 =?us-ascii?Q?cedbDlV81y0bs9bw9F5kIt24YWXKBYrpfEYqjCwh6zepX6nbeWUv0L9bD4Hu?=
 =?us-ascii?Q?uyhUTserrCijLsNYVPAhmowIbHvguxpxxvKUtjB7TtlYqlH2nNQlKvKp4Sjk?=
 =?us-ascii?Q?TT2szk+TjvGYBN4JesedwwT5RbopPeEu6AeEHmE4veFqexslOJ47HQK5lP0k?=
 =?us-ascii?Q?DVoqKDNNixbMPOEd54EIC7sxsZgZTwqODcdwVVG1vxeyqQuCxRnMSP1kJMqO?=
 =?us-ascii?Q?YzDQoZn+jxsbcsyh3h0XTw1PpuYzaDGmIVeWc9NSQhCsPyYd7IYG7xWG648o?=
 =?us-ascii?Q?AEf9XCbx1nfNbAkwb2Z/aT19xrSEhV41AAdi9kM98x8so6scZKKCn0bnZg0I?=
 =?us-ascii?Q?X1nOwFixXyx3GV79lh3Q83SpSvYfHWVwghx0XqdziwtgGBmbKEQvBCcuOfYv?=
 =?us-ascii?Q?BX191hBZjyiBdoWisrIJSRyXI97nkvdeHLLB2RuasThTwU4yu0H6QB90MjSY?=
 =?us-ascii?Q?7oWj2Aa9CJ/pqJMYsoRL8CwtFUywLELD8lWtSAl6/abultzpq/dxzvdRA/Dh?=
 =?us-ascii?Q?aQM+QO8ETV1Up4k7Nurt3fZEYgZwmdaXgODq1h2kGIFanAzakVL5onFDUbFX?=
 =?us-ascii?Q?BNfpvimiKZXgaVSgySvsQB90BTM5D4JyMsnpcPC/hjsgwAeu+JUJjbNwU15i?=
 =?us-ascii?Q?yMvlRLPRM6TKC+Tm9h9EgPeC2b2gltRgcyMYhUvTDmmW77RgP3MEy7Z8I88W?=
 =?us-ascii?Q?qjSMroHCUiiU6huB2JJuUKNUIVO5NBuSCYywI+m+vb8KL1GlRkA/DKPKI5PM?=
 =?us-ascii?Q?0wU8CZ3WTNp86ZE631xNTrYKeUT7SMXsqGqiV/eR?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2c1e9d49-cbcf-4980-dd65-08da55598cfe
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Jun 2022 20:47:13.3560
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: a0D/u7oz5pQc6p28Z2waHw4WKZdrkbrQD9+UkXm0oazueLMASkx4XpjuDQ83xZ5S
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6637
X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Jun 23, 2022 at 02:21:39PM -0600, Alex Williamson wrote:

> check_and_migrate_movable_pages() perpetually returns zero.  I believe
> this is because folio_is_pinnable() previously returned true, and now
> returns false.

Indeed, it is a bug that check_and_migrate_movable_pages() returns
0 when it didn't do anything. It should return an error code.

Hum.. Alistair, maybe you should look at this as well, I'm struggling
alot to understand how it is safe to drop the reference on the page
but hold a pointer to it on the movable_page_list - sure it was
isolated - but why does that mean it won't be concurrently unmapped
and freed?

Anyhow, it looks like the problem is the tortured logic in this
function, what do you think about this:

diff --git a/mm/gup.c b/mm/gup.c
index 5512644076246d..2ffcb3f4ff4a7b 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -1879,10 +1879,15 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages,
 					    unsigned int gup_flags)
 {
 	unsigned long isolation_error_count = 0, i;
+	struct migration_target_control mtc = {
+		.nid = NUMA_NO_NODE,
+		.gfp_mask = GFP_USER | __GFP_NOWARN,
+	};
 	struct folio *prev_folio = NULL;
 	LIST_HEAD(movable_page_list);
 	bool drain_allow = true;
-	int ret = 0;
+	int not_migrated;
+	int ret;
 
 	for (i = 0; i < nr_pages; i++) {
 		struct folio *folio = page_folio(pages[i]);
@@ -1919,16 +1924,13 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages,
 				    folio_nr_pages(folio));
 	}
 
-	if (!list_empty(&movable_page_list) || isolation_error_count)
-		goto unpin_pages;
-
 	/*
 	 * If list is empty, and no isolation errors, means that all pages are
-	 * in the correct zone.
+	 * in the correct zone, nothing to do.
 	 */
-	return nr_pages;
+	if (list_empty(&movable_page_list) && !isolation_error_count)
+		return nr_pages;
 
-unpin_pages:
 	if (gup_flags & FOLL_PIN) {
 		unpin_user_pages(pages, nr_pages);
 	} else {
@@ -1936,20 +1938,22 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages,
 			put_page(pages[i]);
 	}
 
-	if (!list_empty(&movable_page_list)) {
-		struct migration_target_control mtc = {
-			.nid = NUMA_NO_NODE,
-			.gfp_mask = GFP_USER | __GFP_NOWARN,
-		};
+	if (isolation_error_count) {
+		ret = -EINVAL;
+		goto err_putback;
+	}
 
-		ret = migrate_pages(&movable_page_list, alloc_migration_target,
-				    NULL, (unsigned long)&mtc, MIGRATE_SYNC,
-				    MR_LONGTERM_PIN, NULL);
-		if (ret > 0) /* number of pages not migrated */
-			ret = -ENOMEM;
+	not_migrated = migrate_pages(&movable_page_list, alloc_migration_target,
+				     NULL, (unsigned long)&mtc, MIGRATE_SYNC,
+				     MR_LONGTERM_PIN, NULL);
+	if (not_migrated > 0) {
+		ret = -ENOMEM;
+		goto err_putback;
 	}
+	return 0;
 
-	if (ret && !list_empty(&movable_page_list))
+err_putback:
+	if (!list_empty(&movable_page_list))
 		putback_movable_pages(&movable_page_list);
 	return ret;
 }

> If I generate an errno here, QEMU reports failing on the pc.rom memory
> region at 0xc0000.  Thanks,

Ah, a ROM region that is all zero'd makes some sense why it has gone
unnoticed as a bug.

Jason