Received: by 2002:a05:6520:4d:b0:139:a872:a4c9 with SMTP id i13csp761527lkm; Wed, 22 Sep 2021 14:21:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw7tNzPNOpyu9SRYBMreqArf7yT8JkLhIa6aDxFpXVdx92YtTANDc33RXc2R6CpyWEBHy6z X-Received: by 2002:a17:906:b74f:: with SMTP id fx15mr1418220ejb.299.1632345662973; Wed, 22 Sep 2021 14:21:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1632345662; cv=none; d=google.com; s=arc-20160816; b=rBYfyQ6WqB96jUA4dPG/sJz/p6MV+NHhabTLUmYSva2Zbsqnn+r7kQI6LNrFpe/YQZ xvjmu3ZTO9dQzFbi/gY6T2vfWEQmPdgewK3aPtPbO4ROVvay15kUUGmLbShYFmIC2a7v w7JCpxoKr2twCTfIgIb9YYvHknx+pvb2Mnzji43Z26A8m8FlggYwAgvp4BrN/JL4I8WP I/9hWAEfncQ3EeUQy6KAjV0jwzT+MzrGPyGUq+EFpXD9cU7NaNi0Rq99F0XuV5RW6DbF E8EDjW3sy4x0xMLopWfOwS2Bpr3EKhSaiPDAdiEUSoa216pmC9az8YWtSmi6qugBT/xJ 77gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :organization:references:in-reply-to:message-id:subject:cc:to:from :date:dkim-signature; bh=/gYLvFBwTWP63mHQyIOLQDvO3hmF+2iVnEein9qF53Y=; b=Pw+8Oc4KV+tdwXU6ehECxfo4PrZSKbXqgXGZiedVhQ0ttWTIuZ+DrSWInGRgXfItj/ dFD4mqKtptfR1WIrTP9IXWSYKX7vSXx22XSruw+EU5Z5Zc9HEBmVJmuLpHa7F5DTM37h +rYuDwlC6T/QI+jBlCZZy1i+gqRH4Agu/3Rw+4Mek+24fYIKufW4N8OC24msPBrJU/gR xNpx5exIOxrTvOuhll3R4BI1C2EgOSozVtlSjwuFesNU/XK5fXiyhg9wYwH3u8sfcaaM e/WXRH+Md0G5XXq/UacA82wjsdDmED3klC0aka/6QRv8nWvdPJgQoG1tvl7vpLdo29HF 63Hw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="G9NZG6L/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id b2si3782557ejd.610.2021.09.22.14.20.37; Wed, 22 Sep 2021 14:21:02 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b="G9NZG6L/"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237988AbhIVVSs (ORCPT + 99 others); Wed, 22 Sep 2021 17:18:48 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:52226 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237964AbhIVVSr (ORCPT ); Wed, 22 Sep 2021 17:18:47 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1632345437; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/gYLvFBwTWP63mHQyIOLQDvO3hmF+2iVnEein9qF53Y=; b=G9NZG6L/GhdGFp2d4yc9CQ4GNDzxJvhr/lIijCMfUKX2htWSvOl/974bv6FPF/qUoKqqlo uohIIlFQLsFlE6HbqRPVsVxMoP5+LYNCuXOY8RF0K1frItrU0+qPfDxMVjkFP6nv/opKGH 3W6JDmDfI8YUc8NCcEmZUPL4XAAsRUI= Received: from mail-oo1-f70.google.com (mail-oo1-f70.google.com [209.85.161.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-407-1pzwaxnkOomU5x3V873fFg-1; Wed, 22 Sep 2021 17:17:15 -0400 X-MC-Unique: 1pzwaxnkOomU5x3V873fFg-1 Received: by mail-oo1-f70.google.com with SMTP id w21-20020a4ae9f5000000b0029116e62638so2501366ooc.4 for ; Wed, 22 Sep 2021 14:17:15 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:organization:mime-version:content-transfer-encoding; bh=/gYLvFBwTWP63mHQyIOLQDvO3hmF+2iVnEein9qF53Y=; b=dlXORHuYdn76rnGyG8sC5Oat1N+iai58HFgBH88nRr49Lsbfm7Pdjflbmjb4QYTGEb GUc09vF6c7dIeOLeyaXh76PKic6aOhWQ+vZQe20wkWfrWKYkGqllvQ/QRI3cK8fOS13Q eXCxmxWfcGe4hdM2NOXhaJxWAw2fB8JgguGevC0SyOahgi9tyTaBj7qZOve2FehbntoA nAoMU51evrAKmyGmK/c1oS2Mu9BjgKALIKXSEoqYzb0T6607WHMvu6aJHrVFaC+EK3nQ alEPTpDgZ7mDx48p61NM9FtBw9um45YsQEWMyfbugNwl8fGNQq1lStSgsFrykvcWMg2G FtIQ== X-Gm-Message-State: AOAM531pgyDEbcY+3nlpgbBhZjNkZLfvlepHGKCqRWPeOt0H5ziVxj9B fYDorrg6HSKaK74aSFyuh6dS/qY9Om0yOMs46zrFR+KougF1RDALXi9B+s132zXAXqfjuMNJOiI a81lIbenWmFDILXW1e4rrMp2V X-Received: by 2002:a05:6830:2486:: with SMTP id u6mr1135219ots.93.1632345435217; Wed, 22 Sep 2021 14:17:15 -0700 (PDT) X-Received: by 2002:a05:6830:2486:: with SMTP id u6mr1135191ots.93.1632345435007; Wed, 22 Sep 2021 14:17:15 -0700 (PDT) Received: from redhat.com ([198.99.80.109]) by smtp.gmail.com with ESMTPSA id j4sm802381oia.56.2021.09.22.14.17.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 22 Sep 2021 14:17:14 -0700 (PDT) Date: Wed, 22 Sep 2021 15:17:12 -0600 From: Alex Williamson To: "Tian, Kevin" Cc: Jason Gunthorpe , "Liu, Yi L" , "hch@lst.de" , "jasowang@redhat.com" , "joro@8bytes.org" , "jean-philippe@linaro.org" , "parav@mellanox.com" , "lkml@metux.net" , "pbonzini@redhat.com" , "lushenming@huawei.com" , "eric.auger@redhat.com" , "corbet@lwn.net" , "Raj, Ashok" , "yi.l.liu@linux.intel.com" , "Tian, Jun J" , "Wu, Hao" , "Jiang, Dave" , "jacob.jun.pan@linux.intel.com" , "kwankhede@nvidia.com" , "robin.murphy@arm.com" , "kvm@vger.kernel.org" , "iommu@lists.linux-foundation.org" , "dwmw2@infradead.org" , "linux-kernel@vger.kernel.org" , "baolu.lu@linux.intel.com" , "david@gibson.dropbear.id.au" , "nicolinc@nvidia.com" Subject: Re: [RFC 05/20] vfio/pci: Register device to /dev/vfio/devices Message-ID: <20210922151712.20162912.alex.williamson@redhat.com> In-Reply-To: References: <20210919063848.1476776-1-yi.l.liu@intel.com> <20210919063848.1476776-6-yi.l.liu@intel.com> <20210921164001.GR327412@nvidia.com> <20210921150929.5977702c.alex.williamson@redhat.com> Organization: Red Hat X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.33; x86_64-redhat-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 22 Sep 2021 01:19:08 +0000 "Tian, Kevin" wrote: > > From: Alex Williamson > > Sent: Wednesday, September 22, 2021 5:09 AM > > > > On Tue, 21 Sep 2021 13:40:01 -0300 > > Jason Gunthorpe wrote: > > > > > On Sun, Sep 19, 2021 at 02:38:33PM +0800, Liu Yi L wrote: > > > > This patch exposes the device-centric interface for vfio-pci devices. To > > > > be compatiable with existing users, vfio-pci exposes both legacy group > > > > interface and device-centric interface. > > > > > > > > As explained in last patch, this change doesn't apply to devices which > > > > cannot be forced to snoop cache by their upstream iommu. Such devices > > > > are still expected to be opened via the legacy group interface. > > > > This doesn't make much sense to me. The previous patch indicates > > there's work to be done in updating the kvm-vfio contract to understand > > DMA coherency, so you're trying to limit use cases to those where the > > IOMMU enforces coherency, but there's QEMU work to be done to support > > the iommufd uAPI at all. Isn't part of that work to understand how KVM > > will be told about non-coherent devices rather than "meh, skip it in the > > kernel"? Also let's not forget that vfio is not only for KVM. > > The policy here is that VFIO will not expose such devices (no enforce-snoop) > in the new device hierarchy at all. In this case QEMU will fall back to the > group interface automatically and then rely on the existing contract to connect > vfio and QEMU. It doesn't need to care about the whatever new contract > until such devices are exposed in the new interface. > > yes, vfio is not only for KVM. But here it's more a task split based on staging > consideration. imo it's not necessary to further split task into supporting > non-snoop device for userspace driver and then for kvm. Patch 10 introduces an iommufd interface for QEMU to learn whether the IOMMU enforces DMA coherency, at that point QEMU could revert to the legacy interface, or register the iommufd with KVM, or otherwise establish non-coherent DMA with KVM as necessary. We're adding cruft to the kernel here to enforce an unnecessary limitation. If there are reasons the kernel can't support the device interface, that's a valid reason not to present the interface, but this seems like picking a specific gap that userspace is already able to detect from this series at the expense of other use cases. Thanks, Alex