Date:   Wed, 15 Mar 2023 12:14:29 +0800
From:   Huang Rui <ray.huang@amd.com>
To:     Jan Beulich <jbeulich@suse.com>,
        Stefano Stabellini <sstabellini@kernel.org>,
        "Koenig, Christian" <Christian.Koenig@amd.com>
Cc:     Juergen Gross <jgross@suse.com>,
        "Huang, Honglei1" <Honglei1.Huang@amd.com>,
        "amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>,
        "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Hildebrand, Stewart" <Stewart.Hildebrand@amd.com>,
        Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>,
        "Chen, Jiqian" <Jiqian.Chen@amd.com>,
        Xenia Ragiadakou <burzalodowa@gmail.com>,
        "Deucher, Alexander" <Alexander.Deucher@amd.com>,
        "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        "Zhang, Julia" <Julia.Zhang@amd.com>,
        Roger Pau =?utf-8?B?TW9ubu+/vQ==?= <roger.pau@citrix.com>
Subject: Re: [RFC PATCH 1/5] x86/xen: disable swiotlb for xen pvh
Message-ID: <ZBFGJUE0ahA4R9gr@amd.com>
References: <20230312120157.452859-1-ray.huang@amd.com>
 <20230312120157.452859-2-ray.huang@amd.com>
 <ea0e3852-87ba-984b-4010-5eeac3d6c507@suse.com>
 <alpine.DEB.2.22.394.2303141747350.863724@ubuntu-linux-20-04-desktop>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <alpine.DEB.2.22.394.2303141747350.863724@ubuntu-linux-20-04-desktop>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?NifREuM7fqiWloKmSGHWuP1iOJZdSVHeOOeCP28pnosmFWeRWIgTyLJvqU0X?=
 =?us-ascii?Q?wufo7b/LpMKPoS+HsVw//uKLlqG/BZ59LRBGsIFk7dhgLDfjTJLH2EM1XUwj?=
 =?us-ascii?Q?D/gMMIFt5xWOD+Nv0LTJrUnTObbv7vMf9fI9oYCyUD7JXLE19fIhbBsmRlEB?=
 =?us-ascii?Q?nZcB/U7oelJ12tuAWkDEo3HuuXA6hRK0DXeu3CMPCwz/nBenmul+NLlPa2my?=
 =?us-ascii?Q?MAADW3LNPGhs37APrAO5sw2uWCRVXezd0ii5gQv2BZRIjijW5PI5+Gsgna+H?=
 =?us-ascii?Q?Z7kwRQ2+olq9TM89T9NBpheRZjibEF3ooehzZGrdIjvEUeFdKbOFKulkB7DQ?=
 =?us-ascii?Q?B3sUVyK2kFdvm71P1VyojF3gg+sBEukP5/bbDvbSuTGQAq9jMvduQV4XAoFF?=
 =?us-ascii?Q?Qkt+kf8nbC9N76w5s6NY0U45gPwFDQ/oY7nuoQIDeTPwXxSXC/PINyLGzBEC?=
 =?us-ascii?Q?Z7QVoDyU1kzs+/z2KhNxy1N7KHKwbhm+XISa50OWamKT2dwjEbz8dPtYnPaF?=
 =?us-ascii?Q?AEeRsGJgfBbWemm5B8lAglZZByYIltbNZDWvlVnx6b3Be4lnHnKUJYWDtnDU?=
 =?us-ascii?Q?s5U62dFFYWJxBs+5nuvm6Im3BCcf2iUGM9usXbH0pGx1N1jPy1dAXY9CWlGM?=
 =?us-ascii?Q?X/1g5otfo94h5GUGZw/V7Yj/RNeDrre9be7rwRVdB4QtHfHbN1lTZ/4qu9Yh?=
 =?us-ascii?Q?RxdSGF/x54gEUbEThDd8NcaK4xhYrcY6bs9/PH/w7vm5UBmF6fGWwGcBYvZN?=
 =?us-ascii?Q?u7F6WUFxXoVTCDBjks3KWn1nGYyTu1CTD8bIJ//GatqQBWvYJ9AOat5m+JiR?=
 =?us-ascii?Q?I1KlCF2BlNydps5M2/xLGEfn/aVnqmULOL3fAgvJ28ZwaBnL6C27B6MJlCKZ?=
 =?us-ascii?Q?QOAlvUAEUmQ++dggyRg4cqV2lq77wgsKToA4+UPnHhuXpJM2cZTbQ8mU3Tmw?=
 =?us-ascii?Q?mipsgp7HFFDtisUqVcKIKw5gbEhP3YDRg3YAgi/80JLhkDixAT5cBqPHonJO?=
 =?us-ascii?Q?OmgwUxs/jru0W8l7OtIN1+XW+Hk+M+dt3VSh0ZmNnfNBnqAU4qJA49EeCnNd?=
 =?us-ascii?Q?by5f7tmrMVALMTyczjhrdOz754fzeK2g7qa5IAnZy54s4+TpWvw329Phu7UN?=
 =?us-ascii?Q?7WBOhbNWsh/3NLg3rauOdilw/yESKoLj9PWsBzVdCZHjTV4kfzQVYedSkR2n?=
 =?us-ascii?Q?sIkoK2cN4L4AuUpOpOxPUSuXoS2uqSqmmWAH2SGMWIvuncQS51iVwr6oCKzm?=
 =?us-ascii?Q?L828kk4pvtK0rL+avU8lkdD/1ii8ww/iIQZsXZkuIdQrkPs1h1VHOGEo3V0I?=
 =?us-ascii?Q?9ssDYjLYZUig0CbFt6eTjuJba6X/Gnf34ESyD6+56L9FWTvcepAf76axYtj/?=
 =?us-ascii?Q?VMI1TdExH6Xr/fPd98P1Dj/7MtbrwuaedXNOmJEyp/thdGmR4OG6NajW05d7?=
 =?us-ascii?Q?CKBvsJsHac5mlSrpH6E9noIpgA0JcGKc5By11EDsEZevP8Gx4gMyd5BCb2wH?=
 =?us-ascii?Q?KsyYNXLyuHiwoKs9ok4ckR2et4c4+Gho/G+RyihhI4Ko0pHtkXG2FtvmUwSC?=
 =?us-ascii?Q?l19I1YblrWMWH6YRfXyYXRYL3jd+tv0vvaJN295w?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 294de6ca-5332-4c24-7867-08db250bd4a4
X-MS-Exchange-CrossTenant-AuthSource: DM5PR12MB2504.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Mar 2023 04:14:54.7661
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BcQBx1bew9fj0EvUx0Vk6NtDjrBWYxVxqru6tvnOuDM/5YWEeVtKxeg3UXjcek1zVRGrYub+TnQoRVIG6bzT/Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH0PR12MB8507
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Mar 15, 2023 at 08:52:30AM +0800, Stefano Stabellini wrote:
> On Mon, 13 Mar 2023, Jan Beulich wrote:
> > On 12.03.2023 13:01, Huang Rui wrote:
> > > Xen PVH is the paravirtualized mode and takes advantage of hardware
> > > virtualization support when possible. It will using the hardware IOMMU
> > > support instead of xen-swiotlb, so disable swiotlb if current domain is
> > > Xen PVH.
> > 
> > But the kernel has no way (yet) to drive the IOMMU, so how can it get
> > away without resorting to swiotlb in certain cases (like I/O to an
> > address-restricted device)?
> 
> I think Ray meant that, thanks to the IOMMU setup by Xen, there is no
> need for swiotlb-xen in Dom0. Address translations are done by the IOMMU
> so we can use guest physical addresses instead of machine addresses for
> DMA. This is a similar case to Dom0 on ARM when the IOMMU is available
> (see include/xen/arm/swiotlb-xen.h:xen_swiotlb_detect, the corresponding
> case is XENFEAT_not_direct_mapped).

Hi Jan, sorry to late reply. We are using the native kernel amdgpu and ttm
driver on Dom0, amdgpu/ttm would like to use IOMMU to allocate coherent
buffers for userptr that map the user space memory to gpu access, however,
swiotlb doesn't support this. In other words, with swiotlb, we only can
handle the buffer page by page.

Thanks,
Ray

> 
> Jurgen, what do you think? Would you rather make xen_swiotlb_detect
> common between ARM and x86?