Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 24 Jun 2022 10:39:00 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     nobuhiro1.iwamatsu@toshiba.co.jp
Cc:     baolu.lu@linux.intel.com, joro@8bytes.org, will@kernel.org,
        robh+dt@kernel.org, devicetree@vger.kernel.org,
        linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org,
        yuji2.ishikawa@toshiba.co.jp, linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH 1/3] iommu: Add Visconti5 IOMMU driver
Message-ID: <20220624133900.GN4147@nvidia.com>
References: <20220525013147.2215355-1-nobuhiro1.iwamatsu@toshiba.co.jp>
 <20220525013147.2215355-2-nobuhiro1.iwamatsu@toshiba.co.jp>
 <63369db0-cf7f-aa53-bf9f-de2b0b2289ac@linux.intel.com>
 <20220525182644.GF1343366@nvidia.com>
 <TYWPR01MB9420F74E27D8513CD2C2D27592B09@TYWPR01MB9420.jpnprd01.prod.outlook.com>
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <TYWPR01MB9420F74E27D8513CD2C2D27592B09@TYWPR01MB9420.jpnprd01.prod.outlook.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?a05LUGIrZFZncnJwR1Z1Y1JSK0NvMDc1b1N3YVJBSGpzTUVUVFJBdWVBQ1Jm?=
 =?utf-8?B?bEhUSDFJR25HTUFlelhIOVFULzBRZDBRZzZsUVJoTHRaZFIwa3ZCM0tncWlo?=
 =?utf-8?B?QmczMFN1ZkRkK3ZSaFFRQ09IZi82TkVrbHFhNUpyZVFwUWlCbWxiek5JQVp2?=
 =?utf-8?B?N3pDYTdwY1l1Q3FiWWdmWWE0SEtQUHE3UXR5bEl4MWN2cDV0S1ZGNUVhbHJG?=
 =?utf-8?B?UzB6TTlNUkFubkFyRDR2a29nRUR6T0s4eTJQV0dZUXZpSHJGRDZOc1J1N3p5?=
 =?utf-8?B?QTVMallQZnIxMDB3Rm44dzIvbUs2R09Yd2U2UkRmbXovNG9ZQW9aR1lFakVI?=
 =?utf-8?B?MWthTndkdGlGNXV5OVJJdGZHYnRlVStXWUxDRTBFMzM1Y01ZT292UHdPdTRH?=
 =?utf-8?B?WWRHMVN1Vmg4a3FwZFMxVzVQNEVHZUVTVW82c21LaGZFSWNhK2Fma2VQM1h3?=
 =?utf-8?B?cmNvTWhrTExqN2NYYVk4VVJwUTZFSGlHNC9GVklQQUZiZ1dmS2NheU1Pclk1?=
 =?utf-8?B?LytNVUwzQ0NHSGd2c3ZVOE53ZHZNVk43Q1B6V2k1VWJwVFR4ampkbFMyenJ3?=
 =?utf-8?B?aUF1NzRqWlNFTnliUGFjNUE4VmM4c0VuV00rTVJJOXlRb09Fd28ydHBxQld6?=
 =?utf-8?B?RG13WThxdjd1S2RVT1lHV0lEazNpeGhGM0lINGJiMU5xS2t1Uk5qZzBjYTJj?=
 =?utf-8?B?MDFYWVdER2RGL3p4Wjl3bVE0a1hoRlhLSnI0L3FwV1hnMHlOcWY4d3hLQngx?=
 =?utf-8?B?RFAwN0s0NnZNMUVzS3NLVlpRYU0yNk1sNFUwa3JpYTI0UVl6Rit1WFgwazdE?=
 =?utf-8?B?RkEvSWIzb3VMNUh3ZnUrc3dadmNEdnAvR01OandrL2NhUjNIZzJBZlI5VEJp?=
 =?utf-8?B?TDJOeHIrVTFRaGl2M3Ywak5PdnlDdlhraFlpaWdrSVNsS0dqYStXQzNjT001?=
 =?utf-8?B?ZGkyQzdZRXF4YllzRnk1ZjB2K2dNOFFNTVQ2L2V5SlM3RnlaL2g5Z3BRTml3?=
 =?utf-8?B?bFptajRUVHUrWXJENUtpYmFobFVmWEhYSGdHNUVRQzBCZUVLcHRNVkhpYm9i?=
 =?utf-8?B?WmxtSk1td1VuT0djZDdNWUViT3d2S1NGR1pxRDZHbThCYitkNGxBckxYQTI0?=
 =?utf-8?B?Z2tGdlkyRks5T1JrV0thQ0h6ZGVOeXJHaEpyR3h0Ly9sdXNFY05iczUvUXls?=
 =?utf-8?B?YkpyNFhCVEJNL2pXNHJxcEpYY3RkdnJIajhaMFlPdXFyMm9iaHlNak4wdUNr?=
 =?utf-8?B?OFJPdkd4cmFWVk8ySS9oajNJa1UxTmpPekgvSk1WTVF4WnhVZDU3dEkxUEE3?=
 =?utf-8?B?eDYySXBXeDhVUWd5NUwyNTY4enlHSDF0bTlRQ3pDS0J1WmVKZ1Q4ZFFURXVU?=
 =?utf-8?B?c1VzWDMxejk3N2owTnZrNFh6YVFRYXNSTE00bmJhSDVVQ2hLWjVaUTcyY2t0?=
 =?utf-8?B?VjlsaG52T2xBNU5sZ2huakg3cHVJUGFObVRvdGxCOE0wTmlieTR2QmMvUUFZ?=
 =?utf-8?B?ZjFYMjlCZ1c4M3JLa2VlbFAyd0tzKzYrZ2RmYU1qQmd0SUVSWFc5MXltcFYz?=
 =?utf-8?B?dGlTV0RxdFpFOVJ6bUMwYlk0OUMvemRLRmIzK2lKNzVVUmlmSUZwdGRRb29U?=
 =?utf-8?B?eU4zZ3Y1V3Q2M0N1clQvZnZxeE9zclhlb0g3aWdpRVJDZVpSU3ZFYm41K0hK?=
 =?utf-8?B?RUxDNEg5Q0V5T1p6c1pGVEZKNmlJZm1rdzEybjM5WkdiVUgrZE9QN2lyeU51?=
 =?utf-8?B?ajhMR0hxSmNVQnNjam9kWVl3K250czZZclNHenBPSGZvMTRCSjFCU1hwalpm?=
 =?utf-8?B?S0JuMnJDanZUVi9ZUThwOGVWMWt6TTBidHlPdUZqNE5NcTVKeFNpMkJOQUFh?=
 =?utf-8?B?OTRBN21BZGdIRmF0ZU9RRXNGdXVJN1J2eCtSaDJKa0FSSkxCNnViUHorTXdw?=
 =?utf-8?B?UnZUOXhadDhYM0FUeS9Td0dHdVRjWTczK2U1WUhyc0VRUnRJT2pEenpPN2li?=
 =?utf-8?B?bDlVcXJLZDZEWFFveDRRNGpvdC9KcU5MZzc4Lyt2bjYybEZrYnFjUjFzVnp2?=
 =?utf-8?B?cXpCenNhUjFzZUhiT3pqb1NqY0VWS1NMRDVoSjRjaEdKaFVuNVl5VUg4WEJI?=
 =?utf-8?Q?41NHhE0RXbkl61xxE3Ht+Grww?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ca0ad465-9239-4653-9e01-08da55e6e5ca
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jun 2022 13:39:01.4477
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bILEbS/3yVcqfPlcosz1W5e1w+YmP2yGSRzqXG8pq4JmCe0aJnV0QdiR83iMLTMM
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4173
X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, Jun 20, 2022 at 05:49:13AM +0000, nobuhiro1.iwamatsu@toshiba.co.jp wrote:
> Hi,
> 
> Thanks for your review.
> 
> > -----Original Message-----
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Thursday, May 26, 2022 3:27 AM
> > To: Baolu Lu <baolu.lu@linux.intel.com>
> > Cc: iwamatsu nobuhiro(岩松 信洋 □ＳＷＣ◯ＡＣＴ)
> > <nobuhiro1.iwamatsu@toshiba.co.jp>; Joerg Roedel <joro@8bytes.org>; Will
> > Deacon <will@kernel.org>; Rob Herring <robh+dt@kernel.org>;
> > devicetree@vger.kernel.org; linux-kernel@vger.kernel.org;
> > iommu@lists.linux-foundation.org; ishikawa yuji(石川 悠司 ○ＲＤＣ□ＡＩＴ
> > Ｃ○ＥＡ開) <yuji2.ishikawa@toshiba.co.jp>;
> > linux-arm-kernel@lists.infradead.org
> > Subject: Re: [PATCH 1/3] iommu: Add Visconti5 IOMMU driver
> > 
> > On Wed, May 25, 2022 at 02:26:37PM +0800, Baolu Lu wrote:
> > > On 2022/5/25 09:31, Nobuhiro Iwamatsu wrote:
> > > > +static const struct iommu_ops visconti_atu_ops = {
> > > > +	.domain_alloc = visconti_atu_domain_alloc,
> > > > +	.probe_device = visconti_atu_probe_device,
> > > > +	.release_device = visconti_atu_release_device,
> > > > +	.device_group = generic_device_group,
> > > > +	.of_xlate = visconti_atu_of_xlate,
> > > > +	.pgsize_bitmap = ATU_IOMMU_PGSIZE_BITMAP,
> > > > +	.default_domain_ops = &(const struct iommu_domain_ops) {
> > > > +		.attach_dev = visconti_atu_attach_device,
> > > > +		.detach_dev = visconti_atu_detach_device,
> > >
> > > The detach_dev callback is about to be deprecated. The new drivers
> > > should implement the default domain and blocking domain instead.
> > 
> > Yes please, new drivers need to use default_domains.
> > 
> > It is very strange that visconti_atu_detach_device() does nothing.  It is not
> > required that a domain is fully unmapped before being destructed, I think
> > detach should set ATU_AT_EN to 0.
> 
> I see, I rethink implementation.
> 
> > 
> > What behavior does the HW have when ATU_AT_ENTRY_EN == 0? If DMA is
> > rejected then this driver should have a IOMMU_DOMAIN_BLOCKING and
> > return that from ops->def_domain_type().
> 
> If ATU_AT_ENTRY_EN is 0, nothing happens. It does not work with IOMMU,
> it works with the memory space set in device tree.

So that would be an assignment to DOMAIN_IDENTITY

> > I'm feeling like these "special" drivers need some kind of handshake with their
> > only users because they don't work with things like VFIO..
> 
> Since the devices that utilize this IOMMU function are fixed, I do
> not think that a special handshake is required.  Could you you tell
> me where you thought you needed a handshake?

In this case the iommu driver is so limited that it will not work with
VFIO - it is only ment to be used with the fixed drivers that are
paired with it.

Ideally we'd prevent VFIO from connecting and only allow drivers that
know the limitations of the IOMMU to use the unmanaged domain.

Jason