Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: 3b1QHOvsMmYJn5mOOexMW56PPOOUkPHucWXRYwid7EYdq6eWYa+sQi3Bb2NGNlalAz8lCWPUIw
 rhWGC8tuKaBA==
IronPort-SDR: ZtusT+LbjUh3T7aLJ0BJzDneXMJBVk6Py9g05B8GDXhwqs263nx5UkX9LeBP8T0mqPgWJOW1xI
 qwnmL6hT126Q==
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Jason Gunthorpe <jgg@nvidia.com>,
        "Alex Williamson (alex.williamson@redhat.com)" 
        <alex.williamson@redhat.com>,
        "Jean-Philippe Brucker" <jean-philippe@linaro.org>,
        David Gibson <david@gibson.dropbear.id.au>,
        Jason Wang <jasowang@redhat.com>,
        "parav@mellanox.com" <parav@mellanox.com>,
        "Enrico Weigelt, metux IT consult" <lkml@metux.net>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Shenming Lu <lushenming@huawei.com>
CC:     Eric Auger <eric.auger@redhat.com>,
        Jonathan Corbet <corbet@lwn.net>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "Tian, Kevin" <kevin.tian@intel.com>,
        "Liu, Yi L" <yi.l.liu@intel.com>, "Wu, Hao" <hao.wu@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        Jacob Pan <jacob.jun.pan@linux.intel.com>,
        "Kirti Wankhede" <kwankhede@nvidia.com>,
        Robin Murphy <robin.murphy@arm.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "David Woodhouse" <dwmw2@infradead.org>,
        Joerg Roedel <joro@8bytes.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Lu Baolu <baolu.lu@linux.intel.com>
Subject: Plan for /dev/ioasid RFC v2
Thread-Topic: Plan for /dev/ioasid RFC v2
Thread-Index: AddbO/WEUAFl3MPnRsG8exiH8bwEag==
Date:   Mon, 7 Jun 2021 02:58:18 +0000
Message-ID: <MWHPR11MB188699D0B9C10EB51686C4138C389@MWHPR11MB1886.namprd11.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
dlp-version: 11.5.1.3
dlp-product: dlpe-windows
dlp-reaction: no-action
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 8b2d35cc-7b9e-4d1a-1c2e-08d929601ae2
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Jun 2021 02:58:18.6874
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: f7pNzEYkRZIAON0mIkVjxbiyV7Ye89bTuUw1X8cJfe6vorTukPmzgt2g5CiGSb4Qq3LMvNk+7zplGuF/ygRmDA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB4882
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi, all,

We plan to work on v2 now, given many good comments already received
and substantial changes envisioned. This is a very complex topic with
many sub-threads being discussed. To ensure that I didn't miss valuable=20
suggestions (and also keep everyone on the same page), here I'd like to=20
provide a list of planned changes in my mind. Please let me know if=20
anything important is lost.  :)

--

(Remaining opens in v1)

-   Protocol between kvm/vfio/ioasid for wbinvd/no-snoop. I'll see how
    much can be refined based on discussion progress when v2 is out;

-   Device-centric (Jason) vs. group-centric (David) uAPI. David is not ful=
ly
    convinced yet. Based on discussion v2 will continue to have ioasid uAPI
    being device-centric (but it's fine for vfio to be group-centric). A ne=
w
    section will be added to elaborate this part;

-   PASID virtualization (section 4) has not been thoroughly discussed yet.=
=20
    Jason gave some suggestion on how to categorize intended usages.=20
    I will rephrase this section and hope more discussions can be held for=
=20
    it in v2;

(Adopted suggestions)

-   (Jason) Rename /dev/ioasid to /dev/iommu (so does uAPI e.g. IOASID
    _XXX to IOMMU_XXX). One suggestion (Jason) was to also rename=20
    RID+PASID to SID+SSID. But given the familiarity of the former, I will=
=20
    still use RID+PASID in v2 to ease the discussoin;

-   (Jason) v1 prevents one device from binding to multiple ioasid_fd's. Th=
is=20
    will be fixed in v2;

-   (Jean/Jason) No need to track guest I/O page tables on ARM/AMD. When=20
    a pasid table is bound, it becomes a container for all guest I/O page t=
ables;

-   (Jean/Jason) Accordingly a device label is required so iotlb invalidati=
on=20
    and fault handling can both support per-device operation. Per Jean's=20
    suggestion, this label will come from userspace (when VFIO_BIND_
    IOASID_FD);

-   (Jason) Addition of device label allows per-device capability/format=20
    check before IOASIDs are created. This leads to another major uAPI=20
    change in v2 - specify format info when creating an IOASID (mapping=20
    protocol, nesting, coherent, etc.). User is expected to check per-devic=
e=20
    format and then set proper format for IOASID upon to-be-attached=20
    device;

-   (Jason/David) No restriction on map/unmap vs. bind/invalidate. They
    can be used in either parent or child;

-   (David) Change IOASID_GET_INFO to report permitted range instead of
    reserved IOVA ranges. This works better for PPC;

-   (Jason) For helper functions, expect to have explicit bus-type wrappers
    e.g. ioasid_pci_device_attach;

(Not adopted)

-   (Parav) Make page pinning a syscall;
-   (Jason. W/Enrico) one I/O page table per fd;
-   (David) Replace IOASID_REGISTER_MEMORY through another ioasid
    nesting (sort of passthrough mode). Need more thinking. v2 will not=20
    change this part;

Thanks
Kevin