Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 216.228.117.160 as permitted sender) receiver=protection.outlook.com;
 client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C
Date:   Tue, 27 Jun 2023 10:06:52 -0700
From:   Nicolin Chen <nicolinc@nvidia.com>
To:     Robin Murphy <robin.murphy@arm.com>
CC:     <will@kernel.org>, <jgg@nvidia.com>, <joro@8bytes.org>,
        <shameerali.kolothum.thodi@huawei.com>, <yangyicong@hisilicon.com>,
        <jean-philippe@linaro.org>, <linux-arm-kernel@lists.infradead.org>,
        <iommu@lists.linux.dev>, <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v1] iommu/arm-smmu-v3: Allow default substream bypass
 with a pasid support
Message-ID: <ZJsXLFtH8WkRK41w@Asurada-Nvidia>
References: <20230627033326.5236-1-nicolinc@nvidia.com>
 <826cbb86-ac7d-c40b-f7e3-51681cda50b8@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <826cbb86-ac7d-c40b-f7e3-51681cda50b8@arm.com>
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Jun 2023 17:07:06.3209
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 86c8e8b3-f14b-40c3-1025-08db7730ef9d
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT093.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7977
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=no autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Robin,

On Tue, Jun 27, 2023 at 10:00:18AM +0100, Robin Murphy wrote:
> On 2023-06-27 04:33, Nicolin Chen wrote:
> > When an iommu_domain is set to IOMMU_DOMAIN_IDENTITY, the driver would
> > skip the allocation of a CD table and set the CONFIG field of the STE
> > to STRTAB_STE_0_CFG_BYPASS. This works well for devices that only have
> > one substream, i.e. PASID disabled.
> > 
> > However, there could be a use case, for a pasid capable device, that
> > allows bypassing the translation at the default substream while still
> > enabling the pasid feature, which means the driver should not skip the
> > allocation of a CD table nor simply bypass the CONFIG field. Instead,
> > the S1DSS field should be set to STRTAB_STE_1_S1DSS_BYPASS and the
> > SHCFG field should be set to STRTAB_STE_1_SHCFG_INCOMING.
> > 
> > Add s1dss and shcfg in struct arm_smmu_s1_cfg, to allow configurations
> > in the finalise() to support that use case. Then, set them accordingly
> > depending on the iommu_domain->type and the master->ssid_bits.
> > 
> > Also, add STRTAB_STE_1_SHCFG_NONSHAREABLE of the default configuration
> > to distinguish from STRTAB_STE_1_SHCFG_INCOMING of the bypass one.
> 
> Why? The "default configuration" is that the S1 shareability attribute
> is determined by the S1 translation itself, so the incoming value is
> irrelevant.

That was for a consistency since the driver set the SHCFG field
to 0x0 (STRTAB_STE_1_SHCFG_NONSHAREABLE). I was not quite sure,
in a long run, if leaving an uncleared s1_cfg->shcfg potentially
can result in an unexpected behavior if it's passed in the STE.
Yet, we could be seemingly sure that the !IOMMU_DOMAIN_IDENTITY
means the S1 translation must be enabled and so the SHCFG would
be irrelevant?

If so, I make make it:

+	if (smmu_domain->domain.type == IOMMU_DOMAIN_IDENTITY) {
+		cfg->s1dss = STRTAB_STE_1_S1DSS_BYPASS;
+		cfg->shcfg = STRTAB_STE_1_SHCFG_INCOMING;
+	} else {
+		cfg->s1dss = STRTAB_STE_1_S1DSS_SSID0;
+	}

> > @@ -2198,7 +2206,11 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
> >       struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
> >       struct arm_smmu_device *smmu = smmu_domain->smmu;
> > 
> > -     if (domain->type == IOMMU_DOMAIN_IDENTITY) {
> > +     /*
> > +      * A master with a pasid capability might need a CD table, so only set
> > +      * ARM_SMMU_DOMAIN_BYPASS if IOMMU_DOMAIN_IDENTITY and non-pasid master
> > +      */
> > +     if (domain->type == IOMMU_DOMAIN_IDENTITY && !master->ssid_bits) {
> >               smmu_domain->stage = ARM_SMMU_DOMAIN_BYPASS;
> >               return 0;
> >       }
> 
> This means we'll now go on to allocate a pagetable for an identity
> domain, which doesn't seem ideal :/

Do you suggest to bypass alloc_io_pgtable_ops()? That would zero
out the TCR fields in the CD. Not sure if it'd work seamlessly,
but I can give it a try.

Thanks
Nic