Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: /JmLL9sqTLiAhREfHzFN6QwMtdSobY3K8bXYFSG3qMLMC6lb5g9pc7KvFkUu/qiIe+80mo3zzU
 WbZoEDEHaA3g==
IronPort-SDR: a0ir13e/BzJaNciPsna0Ci/RFWDgpE43oyuAxYXsdxZHNDLRUYNkw4XXhYTxz8HrfbiwgOkB87
 1mevmKaqRWRg==
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Alex Williamson <alex.williamson@redhat.com>
CC:     Jason Gunthorpe <jgg@nvidia.com>, Joerg Roedel <joro@8bytes.org>,
        Jean-Philippe Brucker <jean-philippe@linaro.org>,
        David Gibson <david@gibson.dropbear.id.au>,
        Jason Wang <jasowang@redhat.com>,
        "parav@mellanox.com" <parav@mellanox.com>,
        "Enrico Weigelt, metux IT consult" <lkml@metux.net>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Shenming Lu <lushenming@huawei.com>,
        Eric Auger <eric.auger@redhat.com>,
        Jonathan Corbet <corbet@lwn.net>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "Liu, Yi L" <yi.l.liu@intel.com>, "Wu, Hao" <hao.wu@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        Jacob Pan <jacob.jun.pan@linux.intel.com>,
        "Kirti Wankhede" <kwankhede@nvidia.com>,
        Robin Murphy <robin.murphy@arm.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "David Woodhouse" <dwmw2@infradead.org>,
        LKML <linux-kernel@vger.kernel.org>,
        "Lu Baolu" <baolu.lu@linux.intel.com>
Subject: RE: Plan for /dev/ioasid RFC v2
Thread-Topic: Plan for /dev/ioasid RFC v2
Thread-Index: AddbO/WEUAFl3MPnRsG8exiH8bwEagB7l+uAAACIfoAAAdwYAAADDw6AAAHKgwAAANd4AAAAacwAAAT4QwAAK587AAAR+AnQACzmaQAAbNgBMAADwf4AACz/UeA=
Date:   Tue, 15 Jun 2021 01:05:33 +0000
Message-ID: <MWHPR11MB1886F94BDF1CCBC9AC8253A58C309@MWHPR11MB1886.namprd11.prod.outlook.com>
References: <MWHPR11MB188699D0B9C10EB51686C4138C389@MWHPR11MB1886.namprd11.prod.outlook.com>
        <YMCy48Xnt/aphfh3@8bytes.org>   <20210609123919.GA1002214@nvidia.com>
        <YMDC8tOMvw4FtSek@8bytes.org>   <20210609150009.GE1002214@nvidia.com>
        <YMDjfmJKUDSrbZbo@8bytes.org>
        <20210609101532.452851eb.alex.williamson@redhat.com>
        <20210609102722.5abf62e1.alex.williamson@redhat.com>
        <20210609184940.GH1002214@nvidia.com>
        <20210610093842.6b9a4e5b.alex.williamson@redhat.com>
        <BN6PR11MB187579A2F88C77ED2131CEF08C349@BN6PR11MB1875.namprd11.prod.outlook.com>
        <20210611153850.7c402f0b.alex.williamson@redhat.com>
        <MWHPR11MB1886C2A0A8AA3000EBD5F8E18C319@MWHPR11MB1886.namprd11.prod.outlook.com>
 <20210613212258.6f2a2dac.alex.williamson@redhat.com>
In-Reply-To: <20210613212258.6f2a2dac.alex.williamson@redhat.com>
Accept-Language: en-US
Content-Language: en-US
dlp-version: 11.5.1.3
dlp-product: dlpe-windows
dlp-reaction: no-action
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?ZD/oSuz0+62Zbm6CRmm6kbcCdexv/MJb8ZnfJ8qF0tYGcfW2Vnmooh6/s1fr?=
 =?us-ascii?Q?rrGSIQFTJmmdRR7tqiaH90BcLex9hTZpgh26+k2pqWKjWv8kspu14PRNdKaP?=
 =?us-ascii?Q?s/xOj89Ntwl0uDDCT1VVDXQj1P+lVrcjraWz/Bsc+45KjDG1kFSKbqbu/2Lq?=
 =?us-ascii?Q?17Vtcc0h0NiqEWxwCnVaHpstWw0gLUWmPWYu5a0AmWysVKwPdCiSHAwbsLli?=
 =?us-ascii?Q?eEkZeq3evH3/HuQ1IiCGsuMi5M3RDdNCNG+sdK4x5O8PUFhWr3cjZCP3+onI?=
 =?us-ascii?Q?FK4LLt4Ll1L4AN2ydfulMuneWQiLPq6Ub/oGjqq2tIUB1Toe3G9zE7t5/Cun?=
 =?us-ascii?Q?BgYoG4m+6buTFZPieCG+ks5zbJPYrGRtfB4Ldb8zTCmm7B/84lblEmaGWZG2?=
 =?us-ascii?Q?DSDtdYEBb9c1zT8dS3Zfrp5QM/KRPZLyHNQ+511iJOyobD57+IYkUdhchnFk?=
 =?us-ascii?Q?iA7BkcbfXNFJWYq9aYz1lZEaCfWnquCYXiDpn+R7mDvmAse/ZtajLWomhX+J?=
 =?us-ascii?Q?RBquKYauoW9JCP5qkLXo04yIBXAf6Yca4SNcKUMfKT39NCiDr2sVGA1zWsRP?=
 =?us-ascii?Q?bH2ZfbWUAn3YkNYqNw1q23fkEEn9P5wvCCQs+qklY5fwg2kao7djLbwkU67I?=
 =?us-ascii?Q?8GgoY4nky9gHWKCrJTsha2ps5vqdSvnVIVsTpEzAONobngHo9oLEyvHANE7K?=
 =?us-ascii?Q?qUtGBGFPBxP5KT13y/rlJyMsdCE2MaXoWictesAvl3SxbzqJ3O98RWohpOqD?=
 =?us-ascii?Q?NwG1McmXlWDfvnQoyY2FqkEEETxM6N1mKkikJfMxD9GWrOk042toO25dBZQa?=
 =?us-ascii?Q?MoX1BYtdY94fAwDPNsWq1SgkBNgXc7Y8t2n6zSkD37LITjINDGOWY5H710UV?=
 =?us-ascii?Q?nxnC5W1AivmF2XrN1bewG0FrDVmGf+M/wCj45lHbyzY9TzzChCIK38wqu1nU?=
 =?us-ascii?Q?1VYw3MAlezWyZpI28kPxpQprqOUfwIT2kmInH01gHJIhaqBv8lj4g+vhDZVo?=
 =?us-ascii?Q?9QKkBdDtYo4plqLFKf10PnYGNgmrxMau5FUYSynojgle04XCigIiV5Xsobem?=
 =?us-ascii?Q?7KYmjmor6DYxfpGfAa3iu3oml41yQdM3fyg16GUpHy5603XYVpYCkoeeom/R?=
 =?us-ascii?Q?GMNYlAy7/WrUE8wkh1u04eJNhHkVG1hguX71f32d1o4asjcAgbJIyEhe/MZV?=
 =?us-ascii?Q?JywbHkptR4sAuuAbxGRqOM69/ciXXJBdD7GQ294vYa2pGZk0DsXO7tchwNBr?=
 =?us-ascii?Q?ocLCNY7LnkfmLrh1w8CAo9NiswY/RHAn6ECxLmC2aBtz5SQd3iNShSE2G3IE?=
 =?us-ascii?Q?nBOj5TtoB4Vnj25W4Xb2jPfm?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: MWHPR11MB1886.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1de190a9-deb3-411f-7e7c-08d92f99adc6
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Jun 2021 01:05:34.0534
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 3oFKSb4ro8Ww9f/sXleiwVn2nLuQwbEtGPtqwLu/aq0eSRyQH1d0+CZ6ELQmHFCVfYPeUoS9q0+yRckMlal5Ig==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB5010
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Monday, June 14, 2021 11:23 AM
>=20
[...]
> > In the meantime, I'm thinking about another way whether group
> > security can be enforced in the iommu layer to relax the uAPI design.
> > If a device can be always blocked from accessing memory in the
> > IOMMU before it's bound to a driver or more specifically before
> > the driver moves it to a new security context, then there is no need
> > for VFIO to track whether IOASIDfd has taken over ownership of
> > the DMA context for all devices within a group.
>=20
> But we know we don't have IOMMU level isolation between devices in the
> same group, so I don't see how this helps us.
>=20
> > But as you said this cannot be achieved via existing default domain
> > approach. So far a device is always attached to a domain:
> >
> > - DOMAIN_IDENTITY: a default domain without DMA protection
> > - DOMAIN_DMA: a default domain with DMA protection via DMA
> >   API and iommu core
> > - DOMAIN_UNMANAGED: a driver-created domain which is not
> >    managed by iommu core.
> >
> > The special sequence in current vfio group design is to mitigate
> > the 1st case, i.e. if a device is left in passthrough mode before
> > bound to VFIO it's definitely insecure to allow user to access it.
> > Then the sequence ensures that the user access is granted on it
> > only after all devices within a group switch to a security context.
> >
> > Now if the new proposed scheme can be supported, a device
> > is always in a security context (block-dma) before it's switched
> > to a new security context and existing domain types should be
> > applied only in the new context when the device starts to do
> > DMAs. For VFIO case this switch happens explicitly when attaching
> > the device to an IOASID. For kernel driver it's implicit e.g. could
> > happen when the 1st DMA API call is received.
> >
> > If this works I didn't see the need for vfio to keep the sequence.
> > VFIO still keeps group fd to claim ownership of all devices in a
> > group. Once it's done, vfio doesn't need to track the device attach
> > status and user access can be always granted regardless of
> > how the attach status changes. Moving a device from IOASID1
> > to IOASID2 involves detaching from IOASID1 (back to blocked
> > dma context) and then reattaching to IOASID2 (switch to a
> > new security context).
> >
> > Following this direction even IOASIDfd doesn't need to verify
> > the group attach upon such guarantee from the iommu layer.
> > The devices within a group can be in different security contexts,
> > e.g. with some devices attached to GPA IOASID while others not
> > attached. In this way vfio userspace could choose to not attach
> > every device of a group to sustain the current semantics.
>=20
> It seems like this entirely misses the point of groups with multiple
> devices.  If we had IOMMU level isolation between all devices, we'd
> never have multi-device groups.  Thanks,
>=20

If multiple devices in a group are all in a block-DMA state when the
group is attached to vfio, why does vfio need to know whether they
have all been switched to a new security context via IOASIDfd before
it grants user access to a device in a group? Yes, there is no isolation
between devices within a group, but from iommu p.o.v they are all
blocked from touching the rest of the system thus having user access
them won't cause any security problem. Then it's just user's call about=20
how it tolerates lacking of isolation within that group:

1) User could attach all devices in the group to a single IOASID;
2) User could attach some devices in the group to a single IOASID,
     leaving other devices still in block-DMA state;
3) User could attach some devices in the group to IOASID1 and others
     to IOASID2, e.g. when the group is created due to !ACS and the two
     address spaces are carefully tweaked to not cause undesired p2p
     traffic;

In any point in above use cases, the devices within a group are always
in a security context which isolates them from the rest of the system
(though no isolation in-between).

Thanks
Kevin