Received: by 2002:a05:7412:8d10:b0:f3:1519:9f41 with SMTP id bj16csp99391rdb; Mon, 4 Dec 2023 22:46:55 -0800 (PST) X-Google-Smtp-Source: AGHT+IFHHT2hIQKsKZe2sgMFiTmQ+RPHSfy5woYdbnAsMHZvHbHvZxIhwruD8Epc501gFOwOtHpR X-Received: by 2002:a17:90a:f289:b0:286:6cc0:caea with SMTP id fs9-20020a17090af28900b002866cc0caeamr729451pjb.97.1701758815509; Mon, 04 Dec 2023 22:46:55 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1701758815; cv=pass; d=google.com; s=arc-20160816; b=DW/M5rL7M5HulFzN8xPskwgu41hX2XlZmPGHgPjeLx9JCC1wOQ6AONJbkDCXmdpuS0 rEW1iNNOY4Ga+4jcD3Iz0jYcvQx55TcYGALwd23ehxXcb74t/s1IINDP2FkYMnIgLtjy Y7A6Z+H1/A87YY/rwqyKhrZhwRqdQkiP/o0r+ucsk/exls9c3WymNK0dDO4pv+iQVPA9 tJxRAUOr5v5GBzU2GraKmvrq3BIJYSRCRpntG+u/7ooEoCBgy/EyVHo38SkE5UtrnSN5 wWDJJ9bZndNzPvcUSJfz1YiV3r5zpAtU3yDkFTJBq2Fxvgl89QIBqTjmRTIJeNmUNy6d xPHw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:content-transfer-encoding :content-language:accept-language:in-reply-to:references:message-id :date:thread-index:thread-topic:subject:cc:to:from:dkim-signature; bh=jvdPS+9uWXI+EJq/k3ETApJeS6Dt7opu7SPqwybY868=; fh=gd2k6n0Q8NcD/1rGbhRxWwRlpENJQxmEgy58RjAqAI8=; b=QVh7K3UW1TzWDYaMPmyXC/LMXj5ZorwZNMZ1S+BQanwFDMp1NM//k9qqvsmMSkh68n j92343d74FoOEPdIw/q23YhztmKw3EqsjBva3OibcR2xUMso9AItS3Ho1WC5Eczu3WMc Jml7TyabtmZQelq0fDXieQebfc86ZbxBoqoEEzEuzAUA0xbX2vhzLh6BwMLgN4agY0pm EJE/n4AOK12Vns70vSmdqEyzC6szakl8SaBMgo3P992SK7UYHS/P4vHsOAjholG9TH1Z Rt/NjL9Qwod33TblFDTO4jSFeRKKbaUPKkNfeaNAgUY1y6NRSxcuHtXU70v388wI4Ms/ kc7g== ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=R1+o5MUC; arc=pass (i=1 spf=pass spfdomain=intel.com dkim=pass dkdomain=intel.com dmarc=pass fromdomain=intel.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id o19-20020a17090ac71300b00286a297bfd4si3225213pjt.127.2023.12.04.22.46.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 04 Dec 2023 22:46:55 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=R1+o5MUC; arc=pass (i=1 spf=pass spfdomain=intel.com dkim=pass dkdomain=intel.com dmarc=pass fromdomain=intel.com); spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 923638079C94; Mon, 4 Dec 2023 22:45:43 -0800 (PST) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.11 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1344314AbjLEGpX (ORCPT + 99 others); Tue, 5 Dec 2023 01:45:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36562 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229611AbjLEGpV (ORCPT ); Tue, 5 Dec 2023 01:45:21 -0500 Received: from mgamail.intel.com (mgamail.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 445B9D3; Mon, 4 Dec 2023 22:45:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1701758727; x=1733294727; h=from:to:cc:subject:date:message-id:references: in-reply-to:content-transfer-encoding:mime-version; bh=vnO/l0kmGjggmnjR6H5CzA0tIAUCMFeTA1ep0/eUD0M=; b=R1+o5MUCOu4/7LUiaCspEUf6jIavx//FKoCbQ3StE2G3gnQk8+nHsres zhjrsBocfPbQYmn84TrDYOIwRl/4kx8Ppg/9Cip6XjenqHe0TeC0me5U5 vlxaaMef4aXRVqkNtvj4FNEeo8b0MbyjbKK54+dPjBny0uXdb8WH3hR9S UqmO583xogLr0t9D3qz4k/HefjIrGifqw6+/VKEaDP1fyN2f+5tsTjGHr eVcuoI994rgxCJHRrL4EQWU5wN7SUCqJ0cTU0yrCM708dApHw3cKq6sTK SGeTZuz10Aet6Iqip1qHqE6DVj20zh6cIsMEUtY3FziO0qkCoLd14dhhS g==; X-IronPort-AV: E=McAfee;i="6600,9927,10914"; a="396647797" X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="396647797" Received: from orviesa002.jf.intel.com ([10.64.159.142]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Dec 2023 22:45:26 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.04,251,1695711600"; d="scan'208";a="12227964" Received: from fmsmsx603.amr.corp.intel.com ([10.18.126.83]) by orviesa002.jf.intel.com with ESMTP/TLS/AES256-GCM-SHA384; 04 Dec 2023 22:45:27 -0800 Received: from fmsmsx611.amr.corp.intel.com (10.18.126.91) by fmsmsx603.amr.corp.intel.com (10.18.126.83) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 4 Dec 2023 22:45:25 -0800 Received: from fmsmsx610.amr.corp.intel.com (10.18.126.90) by fmsmsx611.amr.corp.intel.com (10.18.126.91) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Mon, 4 Dec 2023 22:45:25 -0800 Received: from fmsedg601.ED.cps.intel.com (10.1.192.135) by fmsmsx610.amr.corp.intel.com (10.18.126.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35 via Frontend Transport; Mon, 4 Dec 2023 22:45:25 -0800 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (104.47.59.168) by edgegateway.intel.com (192.55.55.70) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Mon, 4 Dec 2023 22:45:24 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VoUiMY2G8GsCaOkn0C7FEUeLGPteA/fLsJySID2doiJInwIUqbXQobviET7E0EDCDH2kTMsK5IB0NVY52tW70/KZjJHrMD5Ch3otporEnmuybLobTJNXpPRYaC85U66EqwH7G4odZvTIZMg9h6tRsREgWWUxQRIak/IdFZCbhCIJmBs4ZAu9xRdrQZL08U+UHHDfFb+u+TvPc71TpSUCxyo6fLrdrnXQIgPdOtFJ7rFRNlseA94gZUsgbwbqOW3yq7mLEZmpAinx/6GeVPf9xUF+yFoGbBB4i2I4/gkfrkqFT6rWDoMd2hbRlXQS5CS8IYhZNGu6D9QxrNwtzm81lg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jvdPS+9uWXI+EJq/k3ETApJeS6Dt7opu7SPqwybY868=; b=dL0R+QCvK+L0OW5kxx8+jRHvR88lRZss0ysOy9dObBXBMuExl8Uq9mo//h1VrBfhQhc1dEODI1aC97MrDOLeMyRcvL7CiC838pB9Q7fPvRh9S2WyNNGACR+J/izrTww/t8XpmJyC2zgi6cmKtiM1CXT3EGtnlL1Remi7RDvo4x+y72RpRayI1F3ickIdLzhKjoNLCwtGoKOu2tRg/3S68uW4zO/gd/PNXVuiypp+iBxzK+IkO6PW+S2xdUcGYVM71HG4BTupTiJRSdn8sn5JaS+6/PwzEgm5eQRUa4D5vaPBa1kNTEbkw1deQRvhYlJ/XNpC6sUUoSzjlD81PSYpGA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none Received: from BN9PR11MB5276.namprd11.prod.outlook.com (2603:10b6:408:135::18) by SA1PR11MB8349.namprd11.prod.outlook.com (2603:10b6:806:383::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.34; Tue, 5 Dec 2023 06:45:23 +0000 Received: from BN9PR11MB5276.namprd11.prod.outlook.com ([fe80::e7a4:a757:2f2e:f96a]) by BN9PR11MB5276.namprd11.prod.outlook.com ([fe80::e7a4:a757:2f2e:f96a%3]) with mapi id 15.20.7046.034; Tue, 5 Dec 2023 06:45:23 +0000 From: "Tian, Kevin" To: "Zhao, Yan Y" , Sean Christopherson CC: Jason Gunthorpe , "iommu@lists.linux.dev" , "kvm@vger.kernel.org" , "linux-kernel@vger.kernel.org" , "alex.williamson@redhat.com" , "pbonzini@redhat.com" , "joro@8bytes.org" , "will@kernel.org" , "robin.murphy@arm.com" , "baolu.lu@linux.intel.com" , "dwmw2@infradead.org" , "Liu, Yi L" Subject: RE: [RFC PATCH 00/42] Sharing KVM TDP to IOMMU Thread-Topic: [RFC PATCH 00/42] Sharing KVM TDP to IOMMU Thread-Index: AQHaJQO3kBR+d+LKkUe3K6mDC1fRXbCZPU8AgAAZOoCAAJUZAIAAU9Pw Date: Tue, 5 Dec 2023 06:45:22 +0000 Message-ID: References: <20231202091211.13376-1-yan.y.zhao@intel.com> <20231204150800.GD1493156@nvidia.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=intel.com; x-ms-publictraffictype: Email x-ms-traffictypediagnostic: BN9PR11MB5276:EE_|SA1PR11MB8349:EE_ x-ms-office365-filtering-correlation-id: 2edbabb4-8a31-478a-e8e9-08dbf55dc16e x-ms-exchange-senderadcheck: 1 x-ms-exchange-antispam-relay: 0 x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: zssJa+eLKAI/jl43piheGixSZFbELSycj7RyOlMW2eutbK+noivu8wjkr85TUEwKzkAGJ6HsD3+oW6Qh0aWo7g10dr5zhCxhLWvIcHQZH94FEtGyzDy29Sz42D46S/qA70ni7sJbh/Gp42N0Sg/1V7JS9+tPyOuAYSWNe9q/FBKBZ32QNtwxQa8U6JlO2jjSFRD0rGyS9P42pXdjKgpsVIeBTaoRWkDHTKQycrL8e5FSNQ/KbcuvmbxucRS6asE7t+aTi9IL2a638rS8rVjrUEATm6gtqN2sCkO7xzYhI7dKo53erpN+xbTZlNH5mzxAeyqNWnBth1TIkQCYwK7788c2lRVOz8gGZ0EhVWZrprE2TyvGqINGPGSkA20T8pL8l6UltmKfPZ+zIvkIzzYaCnwKXrz6N70QSiyP3YNrlQsnv9bce1baWEej4iXvVqC9rYtWwiuqs2lfPVWL59hnqY9wq6i7LjKl/BvNcPSGU9RlfIciEfLNy2j1eeWKwat73uD6NIdHKkblURsN8j1BZ8IS7+xeljuF4YjDebnrzs8JPGhxfASqByJ0rhVM6B7NWgKrpKlo92kmbzJFFg3Cw9mfJZo2f1XjpVFsnOjN4yM= x-forefront-antispam-report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BN9PR11MB5276.namprd11.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366004)(136003)(396003)(39860400002)(346002)(376002)(230922051799003)(64100799003)(451199024)(186009)(1800799012)(5660300002)(52536014)(7416002)(86362001)(4326008)(8676002)(8936002)(2906002)(38070700009)(41300700001)(33656002)(9686003)(6506007)(82960400001)(478600001)(26005)(7696005)(71200400001)(38100700002)(55016003)(122000001)(110136005)(316002)(54906003)(64756008)(66446008)(66476007)(66556008)(76116006)(66946007);DIR:OUT;SFP:1102; x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?+Gi4t1NDP0rLPUEX91hHAKDwh5LC4Pw8VYJHnIgb3cOUam5miKooPIzAvHaC?= =?us-ascii?Q?JuavHrzY7G0deIZXZsTeYwXOpOMBcmmK7CXftaPpe8O+w4TTx01m27/m1FdW?= =?us-ascii?Q?YOzXc/2TNoagFUQWBtpKh8aoUv6mYFHp3GJ4FeFTcDXl8QECVKeZcsIcB4GG?= =?us-ascii?Q?GL7GlKTzKDPY/76BDTdGgH685eXl6vJAr/hZ5au1dW8vTlb8XGrZg2jMA0/L?= =?us-ascii?Q?X1x17kWz2pklVlK1RzRGTfzok9B19CkIkAm40ugIbF07+lFU1mifpCvY65p0?= =?us-ascii?Q?PCPlVpp/Knb0sZ2vLoK4AAZX/46wEP/N2L0JztyPEZ5QoIvwkD9A0vT266v6?= =?us-ascii?Q?9CJGgTGTx94xMYpyejC6G9jEuWfOx7cNATbQz8GU8dBPbPNn2u0LBOVfmx3b?= =?us-ascii?Q?rA+VVMCvU3j5CVNIMp0phWZj17eaE0hd2rliXTtp+gBMhsKBMY3OTEODhRDm?= =?us-ascii?Q?7DixgIPAS6YLZqtAarTJqvTkgKJC5D64cAqwOatVIBtqIya+dBYOkXJo6hKA?= =?us-ascii?Q?TiwsYXfZHAdU5n7iugT5674JzO6NRbVfUYSNuMDeoKp7kWPHGOLRWb+bwPw8?= =?us-ascii?Q?OY1C+aFU4CsNr/2JI/TMjR4MFvaeSUBwKQow2fMJKDT/ao29OKHDTBloL7lI?= =?us-ascii?Q?FlaNpbuydrz7sqbsjh6+roM/e3zDWuqjY5rwRQjpCu24GPGRnbm3IptDoQCB?= =?us-ascii?Q?tOzYrNP0KrRSrGTp42rr6zK/aXzspJrSV/ucmMW7Gy9GX2SGj7rhdGFye2RS?= =?us-ascii?Q?W6XPEGjzyAsRXtcFSbzUTCisCbNVC72D3YNhHZcu0yhxELq6JNvYHL55AfGh?= =?us-ascii?Q?nS1GuSGIOSmQBfeCjRbl//Fy3jkHRmH9vHOmv26Prgj6BGMX/5SMAPAi88n2?= =?us-ascii?Q?CYWq139umPskgiHSeeNIbFGdB1aSzEzitIMUVlyWbzuvX+QxfDL3SzQMELlg?= =?us-ascii?Q?1ksGprTefKONcnkL6ockuQKg2RsARYvO9NrTya0xf7s4V3QYouqGliqfbeio?= =?us-ascii?Q?TXlLuT3Defax69vYivGefSay19MPkT7tAWJN1GYmfL9/jKC19W8DkUIVAQbx?= =?us-ascii?Q?GkmZiluQ2YfOHlvexW+9HAg7t9Zy3vKnP2UI+VKVCE3CavlTsAZ/wy7pfBHO?= =?us-ascii?Q?s6twc1lDkIaYgOhmtdzdvfr9wATyNirrzVcQ19EgXAV7mwzfXVC18TPZK1QS?= =?us-ascii?Q?aJMBlciPG0yGQzWC5P9bGmKYK1UeMCdXPwONZJ9FEr7xEEuIeR5QABtOqN0W?= =?us-ascii?Q?2PoDXyRMvPwArkI85B6xGEtC3qoTalnWVkLx6ot4/YVkzJ0z2FiHmt6iNgs6?= =?us-ascii?Q?W5ntOSQNk4nuO8Xe0VYxb73dbUSCHRQZYlB2ArhT+24/wHDxqJI/E57YVzSi?= =?us-ascii?Q?aSsB4qFiTAZgzDB2E2DUyr6ZMTipp+BRLGWdSDryX4RAh+L/qlnXXjUNyoIG?= =?us-ascii?Q?cbzKADZpxr1v3Ed0SwHLQPhgOtPsaNTXikE0+ymDSwzN45Y2uHSxpO8hBSp3?= =?us-ascii?Q?VjtbI7BlB8XdMRgNZstDi+J5DiZq3JEAMYy9Cs84hNixFo+r+p1fj3MUJput?= =?us-ascii?Q?MLgWCny17hCPlzs4MP4vT5mdUCo+EoEPH6VWFMOL?= Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2edbabb4-8a31-478a-e8e9-08dbf55dc16e X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Dec 2023 06:45:22.8596 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: BYe+psa/TvlBGhGp4YEPtYO/EYqFUrIbn8AjS2NN0bueHut55yudN/3VqqevDxHULrOLbRpHdUzowWwW6KvKdw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR11MB8349 X-OriginatorOrg: intel.com X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 04 Dec 2023 22:45:43 -0800 (PST) > From: Zhao, Yan Y > Sent: Tuesday, December 5, 2023 9:32 AM >=20 > On Mon, Dec 04, 2023 at 08:38:17AM -0800, Sean Christopherson wrote: > > The number of possible TDP page tables used for nested VMs is well > bounded, but > > since devices obviously can't be nested VMs, I won't bother trying to > explain the > > the various possibilities (nested NPT on AMD is downright ridiculous). > In future, if possible, I wonder if we can export an TDP for nested VM to= o. > E.g. in scenarios where TDP is partitioned, and one piece is for L2 VM. > Maybe we can specify that and tell KVM the very piece of TDP to export. >=20 nesting is tricky. The reason why the sharing (w/o nesting) is logically ok is that both IOMMU and KVM page tables are for the same GPA address space created by the host. for nested VM together with vIOMMU, the same sharing story holds if the stage-2 page table in both sides still translates GPA. It implies vIOMMU is enabled in nested translation mode and L0 KVM doesn't expose vEPT to L1 VMM (which then uses shadow instead).=20 things become tricky when vIOMMU is working in a shadowing mode or when L0 KVM exposes vEPT to L1 VMM. In either case the stage-2 page table of L0 IOMMU/KVM actually translates a guest address space then sharing becomes problematic (on figuring out whether both refers to the same guest address space while that fact might change at any time).