Received: by 2002:a05:6a10:a852:0:0:0:0 with SMTP id d18csp889089pxy; Wed, 5 May 2021 17:04:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJws1e0FEgvxId6gQh2FZ2CPjKlrVgvXL8YX6F7KXcoTvKpsAJJBnBaqqPB7Fs7pFwRPvsXH X-Received: by 2002:a05:6402:5107:: with SMTP id m7mr1710378edd.75.1620259495837; Wed, 05 May 2021 17:04:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1620259495; cv=none; d=google.com; s=arc-20160816; b=Jc3zmJgLbeJlhIt8Yt92y0u+kUUXNYDacn/KEKRwX7AzZ5cOLWmkWWvHyeYRMLYqDO ARKDTJh/6AqVOHjU+1Q+L02jWYtvTm42/8RTG6YCYTud0yC64Ja8FRPqcFs4EvxOf/q7 pAp0lRj9kdu7BHb0VG09A7/S0QR1ujgxY8zul1uxwqdY9xVlSQNN+BDuvt4L4/einHxc wlUWPbPuRb2uXz+bT8YkifXvqySU+Iau8MNJp5Y3xJiFUGuoLtUY6ZLtI3UBWXmxtxMn 5yV8FpqxiBwfbxHChehzjQRKnh/afyvqrjDLW5X/oO/vkjTLsp5RkRojoKEnHwej908O sGFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:user-agent:in-reply-to:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :ironport-sdr:ironport-sdr; bh=/lnrWM/2E5asLWH3ei7wRuUY2IOjVIsYGbWx6ijZbVU=; b=IUezjcEkkdV4ZwZz+LwyrTwiFukgqBBcXPCC0uc7VtSm/KZSI9DBxQglHf3e3ch9uK 00v2N+OJ7x3Ig5QW/zvV8ZHDJB02+okCJz7YMRPtm6NNWtBCTmKNUlau8AErUQyVyGu8 Qu8ED5IYBI4HlY9bhQ3xZ7/xN3JZjYTt77+/ThDXfEXKniTE1BhqP/EjUY/ROml0NENd QMNHILvegX5PdGG+t2wTbVcjgqIk7EoQ8QlidC78YMby4qLLBKm+CRiu4kVNMFvpsM8e Hvk1X3MbMDBxdxGRAvuJ1yk6VeC2WHrFB6Sr4HBcsZ9G33HtaiIfCl37fVEt7vK0YKYN M0RQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id vr6si714955ejb.739.2021.05.05.17.04.28; Wed, 05 May 2021 17:04:55 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231200AbhEEXYT (ORCPT + 99 others); Wed, 5 May 2021 19:24:19 -0400 Received: from mga01.intel.com ([192.55.52.88]:26124 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231158AbhEEXYS (ORCPT ); Wed, 5 May 2021 19:24:18 -0400 IronPort-SDR: anQ4sq13xt/nbKAl/yW065bk70+iVadgXjWKMgYE7VYMIBiMBFFsU1uP7S8awLm5mocu1B5+G4 eSVpuUnfNiDQ== X-IronPort-AV: E=McAfee;i="6200,9189,9975"; a="219201884" X-IronPort-AV: E=Sophos;i="5.82,276,1613462400"; d="scan'208";a="219201884" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2021 16:23:21 -0700 IronPort-SDR: y6H1briiY6X5e8Ct0n+c9ZWb74lP1ORzh6BLftnomIRyzmNIFA8xgMXubeYF4fCTApEPrsBn/l R9AN4Z2u5GLA== X-IronPort-AV: E=Sophos;i="5.82,276,1613462400"; d="scan'208";a="532177206" Received: from otc-nc-03.jf.intel.com (HELO otc-nc-03) ([10.54.39.36]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 May 2021 16:23:20 -0700 Date: Wed, 5 May 2021 16:23:19 -0700 From: "Raj, Ashok" To: Jason Gunthorpe Cc: Jacob Pan , "Tian, Kevin" , Alex Williamson , "Liu, Yi L" , Auger Eric , Jean-Philippe Brucker , LKML , Joerg Roedel , Lu Baolu , David Woodhouse , "iommu@lists.linux-foundation.org" , "cgroups@vger.kernel.org" , Tejun Heo , Li Zefan , Johannes Weiner , Jean-Philippe Brucker , Jonathan Corbet , "Wu, Hao" , "Jiang, Dave" , Ashok Raj Subject: Re: [PATCH V4 05/18] iommu/ioasid: Redefine IOASID set and allocation APIs Message-ID: <20210505232319.GA5087@otc-nc-03> References: <20210426123817.GQ1370958@nvidia.com> <20210504084148.4f61d0b5@jacob-builder> <20210504180050.GB1370958@nvidia.com> <20210504151154.02908c63@jacob-builder> <20210504231530.GE1370958@nvidia.com> <20210505102259.044cafdf@jacob-builder> <20210505180023.GJ1370958@nvidia.com> <20210505130446.3ee2fccd@jacob-builder> <20210505222120.GM1370958@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210505222120.GM1370958@nvidia.com> User-Agent: Mutt/1.5.24 (2015-08-30) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, May 05, 2021 at 07:21:20PM -0300, Jason Gunthorpe wrote: > On Wed, May 05, 2021 at 01:04:46PM -0700, Jacob Pan wrote: > > Hi Jason, > > > > On Wed, 5 May 2021 15:00:23 -0300, Jason Gunthorpe wrote: > > > > > On Wed, May 05, 2021 at 10:22:59AM -0700, Jacob Pan wrote: > > > > > > > Global and pluggable are for slightly separate reasons. > > > > - We need global PASID on VT-d in that we need to support shared > > > > workqueues (SWQ). E.g. One SWQ can be wrapped into two mdevs then > > > > assigned to two VMs. Each VM uses its private guest PASID to submit > > > > work but each guest PASID must be translated to a global (system-wide) > > > > host PASID to avoid conflict. Also, since PASID table storage is per > > > > PF, if two mdevs of the same PF are assigned to different VMs, the > > > > PASIDs must be unique. > > > > > > From a protocol perspective each RID has a unique PASID table, and > > > RIDs can have overlapping PASIDs. > > > > > True, per RID or per PF as I was referring to. > > > > > Since your SWQ is connected to a single RID the requirement that > > > PASIDs are unique to the RID ensures they are sufficiently unique. > > > > > True, but one process can submit work to multiple mdevs from different > > RIDs/PFs. One process uses one PASID and PASID translation table is per VM. > > The same PASID is used for all the PASID tables of each RID. > > If the model is "assign this PASID to this RID" then yes, there is a > big problem keeping everything straight that can only be solved with a > global table. > > But if the model is "give me a PASID for this RID" then it isn't such > a problem. Correct, since we have usage with ENQCMD, its more like - Give me a PASID1 (not attached to any RID) - Bind/attach PASID1 with RID1 - Bind/attach PASID1 with RID2 and ENQCMD isn't just for Intel, with the DMWr spec in PCI, it brings it to all devices as long as routing is supported by interim switches and such. > > Basically trying to enforce a uniform PASID for an IOASID across all > RIDs attached to it is not such a nice choice. > > > > That is fine, but all this stuff should be inside the Intel vIOMMU > > > driver not made into a global resource of the entire iommu subsystem. > > > > > Intel vIOMMU has to use a generic uAPI to allocate PASID so the generic > > code need to have this option. I guess you are saying we should also have a > > per RID allocation option in addition to global? > > There always has to be a RID involvement for the PASID, for security, > this issue really boils down to where the PASID lives. We do have a RID involvement with PASID always for security. Every RID has its own PASID table, but the PASID name space is global. So if you have RID1 associated with PASID1, another RID2 doesn't have the PASID1 in its PASID table. Until when the app binds PASID1 with RID2 as well. Then you have PASID1 plumbed in the PASID table for RID2. Is this what you refer to for security? > > If you need the PASID attached to the IOASID then it has to be global > because the IOASID can be attached to any RID and must keep the same > PASID. > > If the PASID is learned when the IOASID is attached to a RID then the > PASID is more flexible and isn't attached to the IOASID. > > Honestly I'm a little leary to bake into a UAPI a specific HW choice > that Intel made here. Like I mentioned, this isn't just Intel going forward. The specs are public in PCIe. I just can't comment which other vendors are adopting it. > > I would advise making the "attach a global PASID to this IOASID" > operation explicit and opt into for case that actually need it. > > Which implies the API to the iommu driver should be more like: > > 'assign an IOASID to this RID and return the PASID' > 'reserve a PASID from every RID' I don't think this has any decent change of success. Its rather round about way to get a global PASID namespace. > 'assign an IOASID to this RID and use this specific PASID' This seems a bit complicated. Another way to specify this. - IOASID is a logical construct to specify a page table. - You can bind a global PASID to an IOASID We aren't loosing any security by using a global PASID name space. Until the application asks for it, that is not bound to any other RID without an explicit request. -- Cheers, Ashok