Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33;
Date:   Tue, 21 Nov 2023 19:46:30 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Jakub Kicinski <kuba@kernel.org>
Cc:     Saeed Mahameed <saeed@kernel.org>, Arnd Bergmann <arnd@arndb.de>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Leon Romanovsky <leonro@nvidia.com>,
        Jiri Pirko <jiri@nvidia.com>, Leonid Bloch <lbloch@nvidia.com>,
        Itay Avraham <itayavr@nvidia.com>,
        linux-kernel@vger.kernel.org, Saeed Mahameed <saeedm@nvidia.com>
Subject: Re: [PATCH V3 5/5] misc: mlx5ctl: Add umem reg/unreg ioctl
Message-ID: <20231121234630.GJ6083@nvidia.com>
References: <20231121070619.9836-1-saeed@kernel.org>
 <20231121070619.9836-6-saeed@kernel.org>
 <20231121124456.7a6388c6@kernel.org>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20231121124456.7a6388c6@kernel.org>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?tPOULwuh4LnXup1uFeoN5WGE3nQ4ux6MXXM/Dx8/2uVtpEBLYFnnVZbtH5fB?=
 =?us-ascii?Q?XHSNHywyZcR+NB2vPNvObDaHP2NFH4wPHdszl3iNgRzw3ytd9cYREaB8bgeY?=
 =?us-ascii?Q?zFjKdMszt39rUqoS8yerOywiASRSphy30nOj5ca5MS4nAltbCodxXmKnNDGR?=
 =?us-ascii?Q?Z8teK5axx1Z3VGXCPlCtJD+SOgJ/cGXb/CaCgqCsQm8uk3oY5rHr7400n+WC?=
 =?us-ascii?Q?bWQK82KpmGrEAEEcT4yUau40ltL2yXpRuEu0uqOwflQuBq882HylNQFOSeGr?=
 =?us-ascii?Q?rTowmVMo/quhlAbe665Zrs2R0UTNowF1kCEoLFeNhg9C21RMRzKlpM13nDNm?=
 =?us-ascii?Q?xXii/kVvkilRj5b2AuwETNTs6zgFQx0yfXf1aYcyY8beFOawoBmpUHBHuPoh?=
 =?us-ascii?Q?m1Oc/sJQJPgpmKGsStSaC0BGwzrccO68fcvU5QPvoRofSM11hGvt6EwR6EnV?=
 =?us-ascii?Q?e2eVa/PWU2ZCu0q/nGO9xlZIkRgzJyJSHnqnfN03iqXEU+Y4tPP4hfM3Hna5?=
 =?us-ascii?Q?oqZTRunBrSNYYgLt+P1iOm8TIxaVLo3ff6JOQlT8/2mrrjzxbbWcwJM3qI9L?=
 =?us-ascii?Q?feotJyWDl5oERbMAyk5JnQtZlyhW7P+LXRu318fjWkbAy5J/tkexss5Payfl?=
 =?us-ascii?Q?W5XORe8o9PBMCipTm1D0j8OgZoxdEOsO1N+U5PRzqJvni3lS8rUrpRl0nn99?=
 =?us-ascii?Q?8yRY5VXW4vNshejsYvAVZx234sEKIM/gMLkEGMnyxUKLuVhPoJebQ4oxtkrE?=
 =?us-ascii?Q?kLigfi3oJXnFh8jnaC9LKYjwLoe2loIHO4npJTeuplK+h/GGMwT0bH3pBLM+?=
 =?us-ascii?Q?q9SVNh5CbZe6iaIU902Zi/1m6hqKAkObAfRli9GCVTDsxB5z/v2EgXua6LO6?=
 =?us-ascii?Q?+9tplgBR9PP60zXx6W/B3ES/P853BsN1stUTJGFJ0o5SZJ613eTpIiTLbem8?=
 =?us-ascii?Q?xEVBqz7DHh+MdphsFELcsuV8GHWw0SqUuHLfQgsCT4T73wX/MTt5BlBY/ffu?=
 =?us-ascii?Q?NrFBqrXWUzOzqpH80IrqIx7Y+baZQgNRm4vzruWJC+ur/7teeGnvcaAySHGK?=
 =?us-ascii?Q?wVYWkeIL54aafR5m6Nk30wgbljXQH+k4NI16YT5W10LwIEuXcdlCwUbK71Tg?=
 =?us-ascii?Q?KDujY2l5/1sXam1N+IQsjb5TVGsLxFFBjwliJagu4K63D1XXhdqmL2sFismE?=
 =?us-ascii?Q?Itzp++ESV0GSkqNBgo2xdb7Nib+YdAc+pADeqcz0h3jH2XhUqmCtWslKj1a/?=
 =?us-ascii?Q?pLe3/1kAZkjMuVg+Akf0nF94tnm9ncRm240gThGSIEi9oQFuVcvH4CUF/qiw?=
 =?us-ascii?Q?0lBlpXqn0JbCxeR0fiAIZY7wh+LEhCNubT03HVIgno9FBTYK55Ot7tBY+lfW?=
 =?us-ascii?Q?UkJKmoidCf/F5qGTSOqb1BTUKcM7eo5+imYIzxdTGS18jyqD69Oj0gq7UVfF?=
 =?us-ascii?Q?qbk2/WPLdtjy+K6xRyutagsF0yxEpeHAIIL0/h8vKGJDl1G5RzwauHokez91?=
 =?us-ascii?Q?6hG/pqCNVl1zGfnYjYamDteVHhWP4RZ+JSa00TTzyjflQs6MmJAe5LsJvVQc?=
 =?us-ascii?Q?4JENPd6bMmbZP6KyJPw+5Tl1S9gl7JdkHf/FUed9?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b2f49ff2-e82e-4ff6-02db-08dbeaec16d3
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Nov 2023 23:46:32.0674
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XPPwVdo+ST8Xesn+Q+N7qBp8hwq5XEpZBmnf4ohbvJR4kxsvpT3KzsaTLIreJ5eL
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4895
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Tue, 21 Nov 2023 15:46:49 -0800 (PST)

On Tue, Nov 21, 2023 at 12:44:56PM -0800, Jakub Kicinski wrote:
> On Mon, 20 Nov 2023 23:06:19 -0800 Saeed Mahameed wrote:
> > high frequency diagnostic counters
> 
> So is it a debug driver or not a debug driver?

In the part you decided not to quote Saeed explained how the main
purpose of the generic DMA to userspace mechanism is to transfer FW
trace, FW memory copies and other large data dumps.

The thing with generic stuff is you can use it for lots of things if
you are so inclined. Saeed gave many examples. I think you took it in
the wrong way as I am not aware of any plan for actual high speed
netdev relavent counters in a performance monitor application. It
isn't that kind of "high speed".

The main interest is for micro-architectural debugging
information. The kind that are opaque unless you can reference the RTL
to understand what it means. It is "high speed" in the sense that
executing a FW command per register/counter would be offensively slow
compared to executing a FW command to bulk DMA a cluster of
micro-architecture registers/etc in the device.

The design is so generic because it is a debug interface that we want
to be always available and always fully functional. Mellanox ships new
FW and new chips at a rapid rate, we do not want to be changing the
kernel driver every time we do anything. That will never get
backported into production kernels across all our supported customers
fast enough. Debug features that a field support engineer cannot
access simply do not exist.

Debugs are challenging. mlx5 is the most popular datacenter NIC in the
world. We have so many insane problems, you wouldn't belive it. I just
spent 8 months leading a debug that turned out to be a qemu defect
(thanks Paolo for all the help!!). This debug data and flexibility is
critical to making these hugely complex systems work.

Jason