Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6;
Date:   Mon, 4 Dec 2023 15:37:56 -0600
From:   Aron Silverton <aron.silverton@oracle.com>
To:     Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc:     Saeed Mahameed <saeed@kernel.org>,
        Jakub Kicinski <kuba@kernel.org>,
        Jason Gunthorpe <jgg@nvidia.com>,
        David Ahern <dsahern@kernel.org>,
        Arnd Bergmann <arnd@arndb.de>,
        Leon Romanovsky <leonro@nvidia.com>,
        Jiri Pirko <jiri@nvidia.com>, Leonid Bloch <lbloch@nvidia.com>,
        Itay Avraham <itayavr@nvidia.com>,
        linux-kernel@vger.kernel.org, Saeed Mahameed <saeedm@nvidia.com>
Subject: Re: [PATCH V3 2/5] misc: mlx5ctl: Add mlx5ctl misc driver
Message-ID: <oxtcvxwbj2hzv4lxnxubo3hoxn7diyzhm2oj3tsw2toxbc3og4@ddglhm6r3oa5>
References: <ZWTtTjgBrNxpd9IO@x130>
 <20231127160719.4a8b2ad1@kernel.org>
 <20231128044628.GA8901@u2004-local>
 <20231128065321.53d4d5bb@kernel.org>
 <20231128162413.GP436702@nvidia.com>
 <20231128084421.6321b9b2@kernel.org>
 <20231128175224.GR436702@nvidia.com>
 <20231128103304.25c2c642@kernel.org>
 <ZWZJGF7moDM_k6TU@x130>
 <2023112922-lyricist-unclip-8e78@gregkh>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <2023112922-lyricist-unclip-8e78@gregkh>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Qkop49tpKzw/O84NaQ70fu3rDRgk2oXloeeP4uf6sBKtJceW4/H6D05pJjFX?=
 =?us-ascii?Q?h69mdk9krQfDfODzOzVYZfUYH/J5W15jpuyUtVRx+cU6xGGk686GkEFnidyn?=
 =?us-ascii?Q?91KXnJHdderCwfANNhi46KMTWkeJGO8Vi7V2wfJGqmhFfX6Ok5dI1Qmst3bD?=
 =?us-ascii?Q?+g1pKrN5/ACmO6cA4hdUoFDsZGdb1srZFBJy03fjgeH/QX3MSCraNDR32Tsw?=
 =?us-ascii?Q?99hBWYXcDmqMLmidL5/+3e+BGilaKI5MwqTWrhR5XriWIz92irVoSilABNHR?=
 =?us-ascii?Q?ABE6DZQlrFYJ22NXpQ8mL8XVaokmdJ+phgYKA0ye12cCJXKhsJ2ntDKmYjs/?=
 =?us-ascii?Q?zIcpz/2aDIg1K8XatyTyg24rNU7qXrETEiO04qryaNBkjoTH6KbLo4HMuUFj?=
 =?us-ascii?Q?y1Bwi+ZZYe8VhLU0n9MdokHQv3eOTPKXpgGJDpesjkuk8twKCcEPkr9pdPZl?=
 =?us-ascii?Q?OXxAHY15cadMs7GCG4b7niho7zERMmsni+OIQxVhzcO7jahI6RclS3jMmP48?=
 =?us-ascii?Q?T9LhVH74hmRwRdSpOuJsrUG7hc/46AtLx+AVRIKyJm+/+6aQmO9d0Jf9y4FF?=
 =?us-ascii?Q?ttyezqVUGuw61NnErC82hszOMKJ0+jKfNqAW+XhqPaPyk+5S4D5aPybAQtSe?=
 =?us-ascii?Q?BO4ZPMAgIvXZrxjXCx3aIV69e2DA5bCtZ6bY3rx1R9SnbJCQkHCBHgbzeYY5?=
 =?us-ascii?Q?3SeNjf+MyXRuYqPauWnU5YtjlWEvb43jXlkfCidgkcMtX8ysKOk0BoWv+3xF?=
 =?us-ascii?Q?IR4xM8uxM82Gt8rmKZJG/k/oMThFVvxanISf2TMaiVg7qYjmcqxnAq3mRqPA?=
 =?us-ascii?Q?Oqxsaopr7D0Baz3FSchKA5O70RLZNGwdBfhkzh4Cf+a0wXJGT2O8e7cSMqje?=
 =?us-ascii?Q?EXKwKz+9CWpj6xMgPLVACYBo5g4tIXs3sj2sFR9PfaLv5eNQANZaQq8p6AJY?=
 =?us-ascii?Q?AhKxFJQtfpVYD/WlBgeQIqG5VeTfL1KsaIiOqMpECItGnpy63+PMpO39SXQb?=
 =?us-ascii?Q?54YsshEjd/Bc9YQV9msl2nzE1le3pMZYNzwW0yxV/0W9KuVdMLWs6J5uINdY?=
 =?us-ascii?Q?75kmJqj74wR4o31ZMJGnpzsXAQko5UvhhyQXA3PxQu+mgnrLZPnopyCgI7wS?=
 =?us-ascii?Q?eCoZoMT9bbEvgRIkYJ3/EEmCtdaXLG+uvFrb/NcHjmTWAf0cFmfljQpnzZ/C?=
 =?us-ascii?Q?UtEo5qYlxrqAPszElxapkIAkPbZ5YTT/k/n5dHq1oUmnjCZpqzcZd7Pozxxj?=
 =?us-ascii?Q?/4MmhHPgxoZLbANeCNdRFaMUsubXBthw/4QK7IGU4DbpRsQBpLyleZ6qZ/E1?=
 =?us-ascii?Q?5Z/BlSOFXritWqDwYWe5YMhudfq47Tv3crQoiM1AmMjX9SaPrnXit2fs1RVA?=
 =?us-ascii?Q?IrDDQyO1jSinIQlb4qoTGw7ogpVhIMLqMvXBRDXU2XOmMYqxdAdTDCsAwH8t?=
 =?us-ascii?Q?Gewyt9YBeg3SFsKsKqVeWpisxqyOwKo/oal07zH7T5EicSzZaJr7XVBu79QG?=
 =?us-ascii?Q?Sjz1HvzGQiZlTA6VZL/aWCjeptQ8zSiTjJnWC1dkjX4cQ9Bup3ERaZLxfESs?=
 =?us-ascii?Q?Xf4yTCd63DzP0/n8K5b52i8F7X6u51GpDj/PPdm77Lmw92Pf7MGX2WLuy9dp?=
 =?us-ascii?Q?+g=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: c3lB13cdp7evpjjIu8TNRc8b2odJhKj4ea3MI4tAz1mBcojzFXKHSipIc8IRo4GWFwRvnJ+qDgQ5XjPvWzP9LNCiUvtxBNVg6gT1CImeMJD+JiN7gh/Dh2NXTGopqrL9WBZ4EO6gTqI4jHcoebTljUP/Vms4OzF4Oy4HPRaa1gN2R/tE4xbr9igDKvt2EXPplq7bGCZVhUJEl0JJtEZjUihkM9kdHzHRaJaej+EmX1PxwrAcTNmg1iMVGPjsVnhtLgp1mBfVCarM6waZD3w//v71SQcBuMF/F+hFfF/0+YOTdy4aemagT/0CFDd1T/Qad3RCpB7Et/s0iXHj2/rZWoQojZeztUeXrz+wLNlCqwxN/jtC7lA/xnN9Fqv/878ELZ03eXnvLMnrw2ZEsvtVpHZy2olTJ8ZtN6PIiUUD79BZtbPywMGTp/X02Hy+NKZKZETm0ZvsnrHzo+HDZ5swzbq/81Gg0JqDjoYUjKyWLdSMcskjvXlZqZnLOdvCodflEFhAMKlSEPLQsJD1FFFTlsTD3chBGn5k0HJO74hEtzFnnwFPQj6k6LilJy3SYbQOjIeqALtXSbQOza1haP+wna5uxfrwxnK4AC2eVdfmIIbJbJbrBST6X+MmUITHY5ncapWX5rX5PIEJVY7pyTpeDVQN801Qc48gVLGyn7n4lueETBzhjcIw2BZNI6PWX46708UR4N4KaDEQYAN3hSDIPOs28xDiFm2fkhZhBSES3KhJNMpzSJklvJesIv+JeKA0j9D9qoR6kQ9uXPVNtY9PpR5nL2lft2o03w0DHe1YRSiYeNNSspwKJAoK/ovwFpG0ghSH0FXJ0Vu+bqMs1mRw1lXwWZm16J2j/fGFzGhEUmJlWQ3q6PtaZXwC5f/urMe9+8RsD9Hg/fyAEsu/i9V/Rw==
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 18d1cd0e-b0c9-4901-5ecf-08dbf51149e1
X-MS-Exchange-CrossTenant-AuthSource: CH2PR10MB4309.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Dec 2023 21:38:01.2496
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KMCcyBuSp4dltS70LefmcCsPgVSLSyExfDYX6EnV+LHVs7ZUHIRo+Jr0QJuNR7MJljX4Jtqq7MoA0f3Vhd5ncaF9Z9g9N1zn3ZJ7s09SZ6A=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB4446
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.997,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2023-12-04_19,2023-12-04_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0
 mlxlogscore=875 adultscore=0 phishscore=0 malwarescore=0 spamscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311060000 definitions=main-2312040168
X-Proofpoint-ORIG-GUID: T6BIiz1bMPPH_3wLWxr4dt2W2QDdGV6J
X-Proofpoint-GUID: T6BIiz1bMPPH_3wLWxr4dt2W2QDdGV6J
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
	DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,
	SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Mon, 04 Dec 2023 13:38:37 -0800 (PST)

Hi,

On Wed, Nov 29, 2023 at 09:08:03AM +0000, Greg Kroah-Hartman wrote:
> On Tue, Nov 28, 2023 at 12:10:00PM -0800, Saeed Mahameed wrote:
> > On 28 Nov 10:33, Jakub Kicinski wrote:
> > > On Tue, 28 Nov 2023 13:52:24 -0400 Jason Gunthorpe wrote:
> > > > > The question at LPC was about making devlink params completely
> > > > > transparent to the kernel. Basically added directly from FW.
> > > > > That what I was not happy about.
> > > > 
> > > > It is creating a back-porting nightmare for all the enterprise
> > > > distributions.
> > > 
> > > We don't care about enterprise distros, Jason, or stable kernel APIs.
> > > 
> > 
> > Oh, I missed this one, so you don't care about users?
> 
> Ok, time out please.  This isn't going anywhere fast except to make
> everyone else mad.
> 
> To Jakub's point, no, we don't care about enterprise distros, they are a
> consumer of our releases and while some of them pay the salaries of our
> developers, they really don't have much influence over our development
> rules as they are just so far behind in releases that their releases
> look nothing like what we do in places (i.e. Linux "like" just like many
> Android systems.)
> 
> If the enterprise distros pop in here and give us their opinions of the
> patchset, I would GREATLY appreciate it, as having more people review
> code at this point in time would be most helpful instead of having to
> hear about how the interfaces are broken 2 years from now.

We will be happy to test and review v4 of this series.

Fully interactive debugging has become essential to getting to the root
cause of complex issues that arise between the types of devices being
discussed and their interactions with various software layers. Turning
knobs and dumping registers just isn't sufficient, and I wish we'd had
this capability long ago.

Our customers have already benefited from the interactive debugging
capability that these patches provide, but the full potential won't be
realized until this is upstream.

> 
> And I think that's what is driving your work now, the "enterprise"
> distros finally picked up the "lock down the kernel from random PCI
> device access in userspace" which caused you to have to drop your
> userspace implementation to create a real kernel driver, correct?
> 
> And as for stable kernel apis, you all know that's just not a thing, and
> has nothing to do with users EXCEPT it benefits users because it keeps
> kernel code smaller and faster overall, that's well documented and users
> appreciate it.
> 
> > Users often pay to distros for support, and distros always turn to vendors
> > for debug situations, in fact one of the high stakeholders for this is an
> > enterprise distro..
> 
> Hey, I was right, an enterprise distro wants this driver, great, can you
> get them to review it as well?  I'm tired of being the only one to
> review patches like this and could use the help if they are going to
> rely on this (why do they pass that work to me?).  They should be the
> ones helping us catch basic things like the reference count issues I
> pointed out, as they can test the code, I can't :)

Guilty as charged? But, I'm sure we are not the only ones. I'm sorry that
we were not able to spot this and provide a review earlier, but we are
happy to do so for v4.

> 
> But, let's step back a bit further.
> 
> It seems the network device normal interface for this is using devlink.
> And drivers are allowed to have driver-specific devlink interfaces, as
> you know, your driver has lots of them today.  Why not just add more?
> What's wrong with 600+ more interfaces being added as that way they
> would be well documented and fit in with the existing infrastructure
> that you have today.
> 
> Is the issue that the firmware doesn't guarantee that these interfaces
> will be documented or stable or even known at this point in time?  If
> so, how are your going to have a good userspace tool for this?  What am
> I missing that requires a totally-new-and-custom user/kernel api from
> what all other network drivers are using today?
> 
> thanks,
> 
> greg k-h

Aron