Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Oleksii Moisieiev <Oleksii_Moisieiev@epam.com>
To:     "jgross@suse.com" <jgross@suse.com>
CC:     Oleksii Moisieiev <Oleksii_Moisieiev@epam.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        Oleksandr Tyshchenko <Oleksandr_Tyshchenko@epam.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Sumit Semwal <sumit.semwal@linaro.org>,
        =?iso-8859-1?Q?Christian_K=F6nig?= <christian.koenig@amd.com>,
        "linux-media@vger.kernel.org" <linux-media@vger.kernel.org>,
        "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
        "linaro-mm-sig@lists.linaro.org" <linaro-mm-sig@lists.linaro.org>
Subject: [PATCH v1 1/3] xen/grant-table: save page_count on map and use if
 during async unmapping
Thread-Topic: [PATCH v1 1/3] xen/grant-table: save page_count on map and use
 if during async unmapping
Thread-Index: AQHZHq/2VrVTMTTaekaw3MeMlNz4/w==
Date:   Mon, 2 Jan 2023 13:41:48 +0000
Message-ID: <e58e80d2856cb4656ec76409c2db75652865e2ed.1672666311.git.oleksii_moisieiev@epam.com>
References: <cover.1672666311.git.oleksii_moisieiev@epam.com>
In-Reply-To: <cover.1672666311.git.oleksii_moisieiev@epam.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?iso-8859-1?Q?dANsysGQdFemPfZPLN3R5/iueQyffU7Aefc222DGZ8engqUXxDYFqDhhWs?=
 =?iso-8859-1?Q?klXBmUqO8vdTsG9eOYChTyQ3+PL/+hhuseX0TkGSJKfjWL5yL1FxZI3DLL?=
 =?iso-8859-1?Q?LNWx8q2ucibwxBygufAWID7zKm+cikKSS2rG1pejj+wR1TIuu+K4jplG65?=
 =?iso-8859-1?Q?gFh/+hDBrmYEOHMFr+uTX/Lh8QwwaB/6qHCUjgRwIdlHc3a3UYM96ErdWU?=
 =?iso-8859-1?Q?QOvxwx+ofVp73MmjqxruCfkoSXCYr6T3sXHXGJRNK6L5zKsa8qmeIqt7Zb?=
 =?iso-8859-1?Q?ZwrLGQMJPVB0+P6bg4yghqhyj69aQHwy0x6O94ezPzNMmQYcIMdO+ekY3u?=
 =?iso-8859-1?Q?TKEWrbhJMy+Smzzu52UFjuCq5NTQdEoX1mhiEvnomzVaryP9ZqHooMRPTT?=
 =?iso-8859-1?Q?2p9XEorxFULahLRpfxbZUE28+dnNoLxy3DwrzLfP2YecjeBmwxewRXbxMl?=
 =?iso-8859-1?Q?Q3g+1N68v6lKIRYINDvORC2v7aSOodhMMxcd1Fto72kkDvuQgDrnY27u55?=
 =?iso-8859-1?Q?8OmEd4flK0bQ2jx2sFB5LzGQFN6jUcJIKVIoUyM0lSuHQY+/6PKUKfp/IT?=
 =?iso-8859-1?Q?hCgXV/xWkEELFj5S8nu+/znOyxSwEWeXHshjI5VUt45VhiCvHaHNFzkoQM?=
 =?iso-8859-1?Q?VCcKqsJpv+w8qy4aC+O/zHSn88lQWNbMgd8YYWfRgM75keJMuZbWA1lHQr?=
 =?iso-8859-1?Q?DOnSVB7uyVzulo/R3rWzsTR6oXTE7k3smf49EChQUJRzono5yFwXGDKCfZ?=
 =?iso-8859-1?Q?Sw4/+83oz2cTfxyPYnd/AFfj+M5himSq/t282B5M00TU4Xz3b8YgeDs3Qg?=
 =?iso-8859-1?Q?J3ZBI/2oaAh5nWD9Y+7758VlM/ANYR9Aj46qDx+FOZzJN0cE/xwggKnVDT?=
 =?iso-8859-1?Q?0lX616JttIsRpZ0MYz0cDn+Z388IrOdy9tqgiGe3wvEZFujLz3XRxnyVlz?=
 =?iso-8859-1?Q?xRHIhUGyj6ii4r+jvD2KA8nvMV1mcxtbHkcKlzG9O5dn7+w1/pvVF8f8tI?=
 =?iso-8859-1?Q?nCJ/1SG18OlFiPoUQkKNXYi0gcjEdnxlIQ9Jp+Bt+NKwn9zr1VtjovfjPh?=
 =?iso-8859-1?Q?Rzun/CuzGk/fh2uuaVEadwIcntMwSGXrroTPLOiB17mDlUqd86F+lC/615?=
 =?iso-8859-1?Q?wEzM4vFGJo1gN5F/PmMfolS1kGUcXjVYdYC0wnTFtFlfW1BEZE9EW8MWmT?=
 =?iso-8859-1?Q?dQ/4u6JIBk/Kn8b9cXEOqmg2mcP6yGcIfARxp9CgDU9v9Zuwqe4Ut9v0iF?=
 =?iso-8859-1?Q?Q8QR4DIedxMgyTZPc6wfygq0+Prcu3CzYyAahoY7BAAROmXxKXvIuHxlx+?=
 =?iso-8859-1?Q?tWyMROlWn5GapUZEH34JKeVSD0H9TdnrT7yvsTncF5KCNdKVCtpOuoM+L1?=
 =?iso-8859-1?Q?PgL62+5zHNaB7PqzKmXwB0nwr9yOPIgblcWs1df9/tVt1syoCdf0YNcayZ?=
 =?iso-8859-1?Q?6706N9LqTL7QejzPhha14Lqym3usffPv8FSd80TdfyRJ5wv0ICxWXp8D/Y?=
 =?iso-8859-1?Q?gIqGHet/mIPyvNR10HTpJVUT1rXskt0g3Dc5EPWVM637dWaMp0SfhwOfT0?=
 =?iso-8859-1?Q?zyufy/nezBU2LaSsW2RaKNfl2Gp4tzoSSO3uoJzsHiIDy0pwY1tGcqeB1H?=
 =?iso-8859-1?Q?f/0OgvjdnGNOcLr6n/E1H018gdiz50V/9uefCUfa6ecGjGARXGxbNAOw?=
 =?iso-8859-1?Q?=3D=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: epam.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PA4PR03MB7136.eurprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 01aafa7a-9d3f-4d4e-013e-08daecc718ea
X-MS-Exchange-CrossTenant-originalarrivaltime: 02 Jan 2023 13:41:48.6825
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b41b72d0-4e9f-4c26-8a69-f949f367c91d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: kI1NPR1PtH+s/DzX1W50xaAwDmqfN4R/xQrpV0d+2vI3yUPaymI20bn90qI3J/4UWEJxnW+SL0clikRBIlzGek+KvB6SWXedvjDmoLuBIUU=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DB9PR03MB9709
X-Proofpoint-GUID: mk6-_2x5bNgRKHNXlG71jmOCVvk0mvbE
X-Proofpoint-ORIG-GUID: mk6-_2x5bNgRKHNXlG71jmOCVvk0mvbE
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2023-01-02_08,2022-12-30_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=902 phishscore=0
 lowpriorityscore=0 bulkscore=0 impostorscore=0 spamscore=0
 priorityscore=1501 adultscore=0 suspectscore=0 mlxscore=0 malwarescore=0
 clxscore=1015 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2212070000 definitions=main-2301020124
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_NONE,
        SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Save the reference count of the page before mapping and use this
value in gntdev_unmap_refs_async() call.
This is the enhancement of the
commit 3f9f1c67572f5e5e6dc84216d48d1480f3c4fcf6 ("xen/grant-table: add a
mechanism to safely unmap pages that are in use").
Safe unmapping mechanism defers page that may being use (ref count > 1).

This is needed to allow to map/unmap pages, which have more than 1
reference. For example, DRM_IOCTL_MODE_CREATE_DUMB creates dma buffer
with page_count =3D 2, which unmap call would be deferred while buffer
exists because ref count will never equals 1.
This means the buffer remains mapped during DRM_IOCTL_MODE_DESTROY_DUMB
call which causes an error:

Unable to handle kernel paging request at virtual address <addr>
....
Call trace:
  check_move_unevictable_folios+0xb8/0x4d0
  check_move_unevictable_pages+0x8c/0x110
  drm_gem_put_pages+0x118/0x198
  drm_gem_shmem_put_pages_locked+0x4c/0x70
  drm_gem_shmem_unpin+0x30/0x50
  virtio_gpu_cleanup_object+0x84/0x130
  virtio_gpu_cmd_unref_cb+0x18/0x2c
  virtio_gpu_dequeue_ctrl_func+0x124/0x290
  process_one_work+0x1d0/0x320
  worker_thread+0x14c/0x444
  kthread+0x10c/0x110

This enhancement allows to provide the expected page_count during
map call so refs could be unmapped properly without unneeded defers.

Signed-off-by: Oleksii Moisieiev <oleksii_moisieiev@epam.com>
---
 drivers/xen/grant-table.c | 16 +++++++++++++++-
 include/xen/grant_table.h |  3 +++
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/drivers/xen/grant-table.c b/drivers/xen/grant-table.c
index e1ec725c2819..d6576c8b6f0f 100644
--- a/drivers/xen/grant-table.c
+++ b/drivers/xen/grant-table.c
@@ -1241,11 +1241,23 @@ int gnttab_map_refs(struct gnttab_map_grant_ref *ma=
p_ops,
 		case GNTST_okay:
 		{
 			struct xen_page_foreign *foreign;
+			int page_cnt;
=20
 			SetPageForeign(pages[i]);
 			foreign =3D xen_page_foreign(pages[i]);
 			foreign->domid =3D map_ops[i].dom;
 			foreign->gref =3D map_ops[i].ref;
+			page_cnt =3D page_count(pages[i]);
+			if (page_cnt > FOREIGN_MAX_PAGE_COUNT) {
+				/* foreign structure can't hold more than FOREIGN_MAX_PAGE_COUNT.
+				 * That's why we save page_count =3D 1 so safe unmap mechanism will
+				 * defer unmapping until all users stops using this page and let
+				 * caller handle page users.
+				 */
+				pr_warn_ratelimited("page have too many users. Will wait for 0 on umap=
\n");
+				foreign->private =3D 1;
+			} else
+				foreign->private =3D page_cnt;
 			break;
 		}
=20
@@ -1308,9 +1320,11 @@ static void __gnttab_unmap_refs_async(struct gntab_u=
nmap_queue_data* item)
 {
 	int ret;
 	int pc;
+	struct xen_page_foreign *foreign;
=20
 	for (pc =3D 0; pc < item->count; pc++) {
-		if (page_count(item->pages[pc]) > 1) {
+		foreign =3D xen_page_foreign(item->pages[pc]);
+		if (page_count(item->pages[pc]) > foreign->private) {
 			unsigned long delay =3D GNTTAB_UNMAP_REFS_DELAY * (item->age + 1);
 			schedule_delayed_work(&item->gnttab_work,
 					      msecs_to_jiffies(delay));
diff --git a/include/xen/grant_table.h b/include/xen/grant_table.h
index e279be353e3f..8e220edf44ab 100644
--- a/include/xen/grant_table.h
+++ b/include/xen/grant_table.h
@@ -49,6 +49,7 @@
 #include <linux/mm_types.h>
 #include <linux/page-flags.h>
 #include <linux/kernel.h>
+#include <linux/limits.h>
=20
 /*
  * Technically there's no reliably invalid grant reference or grant handle=
,
@@ -274,9 +275,11 @@ int gnttab_unmap_refs_sync(struct gntab_unmap_queue_da=
ta *item);
 void gnttab_batch_map(struct gnttab_map_grant_ref *batch, unsigned count);
 void gnttab_batch_copy(struct gnttab_copy *batch, unsigned count);
=20
+#define FOREIGN_MAX_PAGE_COUNT       U16_MAX
=20
 struct xen_page_foreign {
 	domid_t domid;
+	uint16_t private;
 	grant_ref_t gref;
 };
=20
--=20
2.25.1