Received-SPF: pass (google.com: domain of linux-kernel+bounces-166465-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Thu, 2 May 2024 09:41:03 -0300
From: Jason Gunthorpe <jgg@nvidia.com>
To: Nicolin Chen <nicolinc@nvidia.com>
Cc: will@kernel.org, robin.murphy@arm.com, joro@8bytes.org,
	thierry.reding@gmail.com, vdumpa@nvidia.com, jonathanh@nvidia.com,
	linux-kernel@vger.kernel.org, iommu@lists.linux.dev,
	linux-arm-kernel@lists.infradead.org, linux-tegra@vger.kernel.org
Subject: Re: [PATCH v6 5/6] iommu/arm-smmu-v3: Add in-kernel support for
 NVIDIA Tegra241 (Grace) CMDQV
Message-ID: <20240502124103.GA3341011@nvidia.com>
References: <cover.1714451595.git.nicolinc@nvidia.com>
 <63414546b1eafdf8032ac1b95ea514da6d206d63.1714451595.git.nicolinc@nvidia.com>
 <20240430163545.GS941030@nvidia.com>
 <ZjEztwhd5AN1FTCk@Asurada-Nvidia>
 <20240501130042.GC941030@nvidia.com>
 <ZjJ/S1bawYkfs1I4@Asurada-Nvidia>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZjJ/S1bawYkfs1I4@Asurada-Nvidia>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?LlqlVI9emcXs+wZivqymU4KrLicfyR43lnMuv79Vud0gO+DJbW7TUxLP1NoR?=
 =?us-ascii?Q?prTfwq7a+o5BO1iJA5jKB5klDpM9NrPECJKh7pvkDG4bhFMrtZx5XlnlfJC2?=
 =?us-ascii?Q?U1fz6ZgdzevfCdYdfCvuQV+YG2eJVJswGBLpd1+mepxDpoV2sLeGXxAByxYA?=
 =?us-ascii?Q?68kS5HKmCgCQfv7BFEZfhQllNyS21osb3h4B9Bn8kIqfeAXTKQ2ixe+6xDN9?=
 =?us-ascii?Q?KTh6ABVIAhPI5VgvJMsMseQfpypEtAVpl2cji9nsaCI1Xf7Sr11v823+no+F?=
 =?us-ascii?Q?lfHH/ppt2+NrwpGfzyFzH9JqeX79iRCVyj92rekC7hY69F8ChX48ZU3OIo9l?=
 =?us-ascii?Q?wKr8ej89jtglzDnmzZn32Z6KfxfEHl/J3Z+sL6KDg/icKNkeEOCNELLiPBlZ?=
 =?us-ascii?Q?3SPDtRncL7JGBy+H2vHxmDCtittI24e5XLrH4wGwEnpDTc0hdv5juAY/sNAf?=
 =?us-ascii?Q?Up9YUQ5y2PPzQAW3jdXquA3/IVV4GRwn3Z4o74BifUEapd1YNba9056UZC7j?=
 =?us-ascii?Q?ZddeQ96oIhhVfb9n9BeqzYGBnXqVzOHkcL9rEPXXBMnumShlOd1fRPFfnT++?=
 =?us-ascii?Q?w73t6IFBeONSgTGTYFz7vMVXLztGbdXw7i+vu95ZnM4Ts6VF0QI8m9skgmJd?=
 =?us-ascii?Q?5A+rERmVjrMOB5dpZRV55vVWFfECZYLXZE3totxi8h6HxrlzSsarXklJCM/L?=
 =?us-ascii?Q?nUMLTZ9xGATGmi1nxuixJ2CASDlqGWbgpZIwAd3HBPSGWyHqy0QgUV+DxCVZ?=
 =?us-ascii?Q?M2J2KS6PliBwZHQHFEFkyPA/Z54S5prayBct4zhmt4urj4C4mvnopdh+ptH2?=
 =?us-ascii?Q?OOwjLJZ3flyAxilzBvkBDzX1BPLLkomy55pM7RaFBwF7XqzWVrcJnpSJWZCA?=
 =?us-ascii?Q?bPPKDfSTIco6WxxbiTvHjJRkF1t57yjrEZjJ9g1/hgknEJtobsKSogLzGzMm?=
 =?us-ascii?Q?0faU4eBLlp4iHYXqvKMIQaVvFoiW7N6lTWOf6cN1yGyGBI0qrImzRlnOddkv?=
 =?us-ascii?Q?9jIPrURJ2n3QWSGBHwHondI3fFOJuGQ4LQqPKbor93p3vmrYDfvW4ZXfw/EN?=
 =?us-ascii?Q?lJjO1Rv8GLcwcPjOSWfwIXS2Kdu83Bhz6xhdDenQBff9bmL8m4uH4qdEp2dU?=
 =?us-ascii?Q?fcu7OUwMifBdUEVoIDhN6yJfb8jQVlO6BJRhZggyHZl8Zm5tx4WeOQlrNWwP?=
 =?us-ascii?Q?uU6XBePSwzMcPQzJyU26iw/KyZ2wNHfvdsYNWQEQ2/oBnhFRuhWlpQcPX6vq?=
 =?us-ascii?Q?QmcWlllcHwWmQRWRUphIqEnX/6/FYDw3QJDkYsEc/YCp3fTG0FEMinS8Z/Ip?=
 =?us-ascii?Q?i3ctb0QV5CrWchKmLufDEI9VHcXLSgfPizJJXQbnqY0BNuxpUv+c3S019DRi?=
 =?us-ascii?Q?/26hDb1Fl9YaYyxBnW1kq+rqH8jGaULtuwZI6OcWoQitzqypNQuLCS709AsT?=
 =?us-ascii?Q?WIuwDFPZOXVoiPKgSRYyomAWKu5yPl/5X1/fCb0I6d0YSu/wHISfKeJrECaU?=
 =?us-ascii?Q?+U+dm+f+yX38nElnvIIzEmtXkEtcanpaZe+KZ3rkj1oSoJzjyibrXjkmdSoa?=
 =?us-ascii?Q?KebM20sbZmb2E8CScZQXDzyka3tF9IhCNufMvdHC?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b45d34d5-e3ed-4c78-3708-08dc6aa52250
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2024 12:41:05.9108
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: rjxqldUZ3yhYAttK4WE8WRXb7yxC2kJvjZFJ6aQqMHFHbKaQw7rboEJ6DtF+B+yN
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4485

On Wed, May 01, 2024 at 10:43:39AM -0700, Nicolin Chen wrote:
> > It doesn't fix any real race, I'm not sure what this is supposed to be
> > doing. The cmdq becomes broken and you get an ISR, so before the ISR
> > it will still post but get stuck, during the ISR it will avoid
> > posting, and after it will go back to posting?
> > 
> > Why? Just always post to the Q and let the ISR fix it?
> 
> Yes, we could do so. I was thinking of the worst case by giving
> the guest OS a chance to continue (though in a slower mode), if
> something unrecoverable happens to the VINTF/VCMDQ part.

Does that happn? The stuck vcmdq will have stuck entries on it no
matter what, can we actually fully recover from that? Ie re-issue the
commands on another queue?

> > So just don't use it. There is no value if the places where it should
> > work automatically are not functioning.
> 
> I thought devm could work when rmmod too, not only when the probe
> fails..

It is limited to cases when the probing driver of the passed struct
device unbinds, including probe failure.

Jason