Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 9 Aug 2023 12:08:02 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Will Deacon <will@kernel.org>
Cc:     Michael Shavit <mshavit@google.com>, iommu@lists.linux.dev,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org,
        robin.murphy@arm.com, nicolinc@nvidia.com, jean-philippe@linaro.org
Subject: Re: [PATCH v5 2/9] iommu/arm-smmu-v3: Replace s1_cfg with cdtab_cfg
Message-ID: <ZNOr0ggoO9kXHJWl@nvidia.com>
References: <20230808171446.2187795-1-mshavit@google.com>
 <20230809011204.v5.2.I1ef1ed19d7786c8176a0d05820c869e650c8d68f@changeid>
 <20230809134941.GA4226@willie-the-truck>
 <ZNObxeogswAYyDQ5@nvidia.com>
 <20230809145542.GB4472@willie-the-truck>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230809145542.GB4472@willie-the-truck>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Pe15e7YEJ/c7dAMbZpT/sQxsGMzlCqvDZUHjvz6d66EXTY9jT+p5EcVf3JEi?=
 =?us-ascii?Q?1+IewHXH4ozqQcOJXLTSqgPPAxySzTbLje6suB8ihePE30JUOcpdM1Uu0qDQ?=
 =?us-ascii?Q?zTZv/MJ9LZwqxRQvBP1orBTow5ZfJ5NEXSe2dak7Huo2gjRT1Jg2YYMMwkZ2?=
 =?us-ascii?Q?TOdbkPiaNZMUne+v0ti5cnbKwy+SB9psxc/7kdwdNi6Oenrio0OpOkCb/PQJ?=
 =?us-ascii?Q?P5zz8vRKLP5QzgHYsRdmN+gEs/jEwknHwz4kUZ3yrksl3P0f4G95uPyfkUF0?=
 =?us-ascii?Q?Ne5M1LzgQta3o5etojnKqaypgG7AqGTOhFYC2r509UQ7GNE7MLZ3oWtsYQGb?=
 =?us-ascii?Q?BnA0hb0oXIuST8zLt01CDfmnkKVUSzjID21cwcMkOpLdSpbpFRAqYYbl3Nfp?=
 =?us-ascii?Q?kgdMroqtdr90mct7xDoR8Y2uE7VyytJwCRKEzLBIDIXqVwuPPq6ClTP+mdOP?=
 =?us-ascii?Q?nwtGJeAeebBNBtVFG+NDCSY5rD3Fd3Oyqm68tM7Zl2A5CcJ3Z3RDv8jLHvi8?=
 =?us-ascii?Q?ak0jipUaOA0ToMQRqRE7BsuLdRwRCyQpQTMOEnMs6oQLPzsvJv2VUkyHxfPM?=
 =?us-ascii?Q?ED+sl0tn76WQJ13wK8a8veZstmK3QF9rUHaFfIxY0GAPnkHyHogdFTk82cPR?=
 =?us-ascii?Q?wNL6WMcV0L3XuwjQNqnaBFRnRD69mKp0AH4T9iJWt2RzG9VXZdHUQTLdjfIp?=
 =?us-ascii?Q?kDZKmYrpVd/oYEB+xlzf94K3XaezIVS//4uehiETokZ+v8kimmwB5UD59n5Y?=
 =?us-ascii?Q?m+ClE+RYxs29bZUmdv1+QfMjMHrb9p4OVntRx39ckPbb4qc8BUU5g6MRRjkD?=
 =?us-ascii?Q?Ay1S4EmfQNOl2CLt3otA9clbNm7lGcB1DJEhm2CMHZj7UW2JRZQF5Q0/fGcM?=
 =?us-ascii?Q?4VU8EQrt8txdTVcVVtnrnaOENWMnN+Rw8doWjlCXz6jB0FK4iUzsL5B0E9C1?=
 =?us-ascii?Q?TCgLFkQdLrfzfWDlxLm+EDFiWojtryNB5dDzJAMFgxenER1o797dAXZ6083F?=
 =?us-ascii?Q?77XqRPY5Qq5hzeRQwjYwsDJf9FON6mI/rnJ0Dz2roLvKl5playfO3DzTyflX?=
 =?us-ascii?Q?RtmvHWWfd8TwHOu5ODLqD5Hrp8lce4/Jyr74RvA7V0wRTkz/WdURTQQ4vBFj?=
 =?us-ascii?Q?K70HMHC+YfFJIddRtUWioboxZm/VWwKXOyOX1j762jZ0PzcTcqIpdzWsdqYO?=
 =?us-ascii?Q?VRUr7GqwuQeowP4tMkJ6+KsPwZb98tPGSBq9T7k08KmxswEb/ICCy6j0m86k?=
 =?us-ascii?Q?3uZNtbE+mEvsIb4jTS0PzjNCklMzSp3H/U2pt3muYEP4j/Yj2e0J85h4wVu0?=
 =?us-ascii?Q?vn7/HvuvDDEBRiT/zp/lPyMjFmB6QgMQGgdN/RoRernE6rLlJs/E5aoju+UT?=
 =?us-ascii?Q?3Peu8PgCfxslQ52blzJxUSm6KiYoGBZEPK65QPAgJJmw8ROYAD7wZgojcOd3?=
 =?us-ascii?Q?bvl5XzFTxe35l8ZMj5+PkSxXuTU3n1OMOvTpgrwMZHIsS/F9Co402lxe+tGo?=
 =?us-ascii?Q?0x13KGGQiJV+r4eWE6LJtmkLT6ngl1vz2JoUrV6leXkdEu37/LZ2K+Jk9d0y?=
 =?us-ascii?Q?/lPNUlM6uyOHCnnNmirKiiEMULaaLh/qP/B2fuzS?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b08b491c-545b-415e-8db9-08db98ea6db6
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2023 15:08:03.5030
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Tnj3IXQbT4TytpNPCgaH21QpNWhwTlgrKjCd0OtvRg9UbwP0kUDNvFcKgu9BdHAT
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5120
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Aug 09, 2023 at 03:55:43PM +0100, Will Deacon wrote:
> On Wed, Aug 09, 2023 at 10:59:33AM -0300, Jason Gunthorpe wrote:
> > On Wed, Aug 09, 2023 at 02:49:41PM +0100, Will Deacon wrote:
> > 
> > > > @@ -1360,10 +1357,14 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid,
> > > >  		    !master->stall_enabled)
> > > >  			dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD);
> > > >  
> > > > -		val |= (s1_cfg->cdcfg.cdtab_dma & STRTAB_STE_0_S1CTXPTR_MASK) |
> > > > -			FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) |
> > > > -			FIELD_PREP(STRTAB_STE_0_S1CDMAX, s1_cfg->s1cdmax) |
> > > > -			FIELD_PREP(STRTAB_STE_0_S1FMT, s1_cfg->s1fmt);
> > > > +		val |= (cd_table->cdtab_dma & STRTAB_STE_0_S1CTXPTR_MASK) |
> > > > +		       FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) |
> > > > +		       FIELD_PREP(STRTAB_STE_0_S1CDMAX,
> > > > +				  cd_table->max_cds_bits) |
> > > > +		       FIELD_PREP(STRTAB_STE_0_S1FMT,
> > > > +				  cd_table->l1_desc ?
> > > > +					  STRTAB_STE_0_S1FMT_64K_L2 :
> > > > +					  STRTAB_STE_0_S1FMT_LINEAR);
> > > 
> > > magically know that we're using 64k tables.
> > > 
> > > Why is this an improvement to the driver?
> > 
> > Put the above in a function 
> > 
> > arm_smmu_get_cd_ste(struct arm_smmu_ctx_desc_cfg *cdtab, void *ste)
> > 
> > And it makes more sense.
> 
> Sorry, but I'm not seeing it :/
> 
> > We don't need the driver to precompute the "s1_cfg" parameters and
> > store them in a redundant struct along side the ctx_desc_cfg when we
> > can compute those same values on the fly with no cost.
> 
> But the computation isn't happening -- the STRTAB_STE_0_S1FMT_64K_L2
> constant is hardcoded here. 

So it would be hard coded in arm_smmu_get_cd_ste() because that
reflects the current state of CD table code.

> If we want to use 4k leaf tables in some cases, how would you add
> that? Such a change shouldn't need the low-level strtab creation
> code to change.

You would modify arm_smmu_ctx_desc_cfg to teach it about the different
format. It would gain some (enum?) member that specifies the CD table
layout and geometry. arm_smmu_get_cd_ste() will interpret that member
and generate the correct STE for the specifc cd table.

It is a standard OOP sort of practice that the object self-describes
and has accessors to export its internal definition. In this case the
STE bits are part of/derived from the internal definition of the CD
table.

Jason