Received-SPF: pass (google.com: domain of linux-kernel+bounces-135699-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Date: Mon, 8 Apr 2024 13:41:14 -0300
From: Jason Gunthorpe <jgg@nvidia.com>
To: Jakub Kicinski <kuba@kernel.org>
Cc: Edward Cree <ecree.xilinx@gmail.com>, David Ahern <dsahern@kernel.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Christoph Hellwig <hch@infradead.org>,
	Saeed Mahameed <saeed@kernel.org>, Arnd Bergmann <arnd@arndb.de>,
	Leon Romanovsky <leonro@nvidia.com>, Jiri Pirko <jiri@nvidia.com>,
	Leonid Bloch <lbloch@nvidia.com>, Itay Avraham <itayavr@nvidia.com>,
	Saeed Mahameed <saeedm@nvidia.com>,
	Aron Silverton <aron.silverton@oracle.com>,
	linux-kernel@vger.kernel.org,
	"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
	Andy Gospodarek <andrew.gospodarek@broadcom.com>
Subject: Re: [PATCH V4 0/5] mlx5 ConnectX control misc driver
Message-ID: <20240408164114.GP5383@nvidia.com>
References: <1cd2a70c-17b8-4421-b70b-3c0199a84a6a@kernel.org>
 <0ea32dd4-f408-5870-77eb-f18899f1ad44@gmail.com>
 <20240402184055.GP946323@nvidia.com>
 <83025203-fefb-d828-724d-259e5df7c1b2@gmail.com>
 <20240404183305.GM1723999@nvidia.com>
 <20240404125339.14695f27@kernel.org>
 <20240404204454.GO1723999@nvidia.com>
 <20240404143407.64b44a88@kernel.org>
 <20240405111306.GB5383@nvidia.com>
 <20240405083827.78cc1b20@kernel.org>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240405083827.78cc1b20@kernel.org>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?tkw29uIAomV8FcdYNRzbD5US0S1GIPVNTjcmGqxV5kcVJI6NOqBxQe6IPeC6?=
 =?us-ascii?Q?GUz6gSuMGF9Oplqwp/MJfU8pPPEZ+p7jRNn/yR+ugch5RhjYYcXZFa1nAk/L?=
 =?us-ascii?Q?0HUos6d+IMiH22J1c1CId75IgnPUWpNnun2tSTTADQan9lE7cd3ltudOR7JI?=
 =?us-ascii?Q?B58p0PAmlJ70NEPy15RbS9KXImd6IesU8Bb//XbnRa9s2fNmIYPEO7XGPAsZ?=
 =?us-ascii?Q?vF8Pjb+eGdjqyW0O5Vb3oErepGiCzreis9FTtac5RvnPvgcFG7uo/rHQjZ6u?=
 =?us-ascii?Q?g3YiKCm2XwPXgVX8b4Lix7uRjPKr8hAlmxMHpmf0DHG017fPmBBQVPS1H3dJ?=
 =?us-ascii?Q?FDaMkli8LqEjzT+PNIQJEM5i5vsRdVCmeQYjukLUF3+W0BiZbJvxDcWqvH/q?=
 =?us-ascii?Q?TUsH3eeGzUPI0MNns/rL9R+w9vgkKafJYMNn0bro2N8bChWt6WDJS5cKeix+?=
 =?us-ascii?Q?mRILaWTuKnyVzsLE/0i3cAUo+wAor7Xfx/JSET/mKnWbnFs2Y9/aO9MLS8I3?=
 =?us-ascii?Q?Gm+lr7kHJaEh7S+RViHueKGLFbYdUZvmrW/87xmlfdP2dYeXDVbnAF13iFBl?=
 =?us-ascii?Q?KQjCYYKlITBfmqvcN9iiEBAm2OZYq7e/xcKZwn7I3pbsYkvZxtRWEB2mcDs6?=
 =?us-ascii?Q?fsmI8o0KN6dsVas0nozrZ7Ln0OLnPYT0rziC1M2AoId8RYUIkYJe6IE7+BMd?=
 =?us-ascii?Q?hOf9kCdo//KcZpXTZrI7lt6syrt7YEOUlDhG9yM+OXHU/PSEWvbaROPX/46X?=
 =?us-ascii?Q?jUMHAd3JYRTW5WbqHJmd+4gXKgdA9soEFQ2gvBAHOg6Judw+ZqSL0oc5Rp6R?=
 =?us-ascii?Q?fYOv07qseNgwm5qXAp+mDSAzEr3e4cpzoeAtWhLLs1CJmLLK7/WzJF1pA7qZ?=
 =?us-ascii?Q?9gQ64yapQIyoKy8uuIUFiEqihBUZl0FS/1H+UU4/fe3Ls+mDIAMvvivPxFAS?=
 =?us-ascii?Q?p67R6UoGHstvjgwh01fSspwMAZeUXVdVNoAFKf675Z+do2NO/QCe+4VMtVX2?=
 =?us-ascii?Q?0nz5rRlSmfzHEASxtZh4bNp8wvsUBCtt1U41MshgIrxQQ4mqArrk8mkg7Qdg?=
 =?us-ascii?Q?Irvczsz4RLTYLD2CCATbYWXWZ2/9JRqnKadHT7+0env8HRyf3SrQla1uzWr1?=
 =?us-ascii?Q?zXS4qbbFym6I2cmqu0mj/9NzHC2Qk35UZDZ6jINWjIDr+UnZ2PouQTtY2pZN?=
 =?us-ascii?Q?VmdAsuETu5Vx/jbHtJowhnCXXWKcNfeRAkuFvJqisQdlqYx8F5QG40r602aE?=
 =?us-ascii?Q?wv7aomgxuWep9QbMs07bowMU5EFfeTvH5d+8Eqp9JUzB0TwWnlnjtXz/0H4d?=
 =?us-ascii?Q?gAenzFFTnCMbezut/8UgZPzaQmP4uUrfmWyHTCVxUyiWmxj3Yr2yrCEJt5f4?=
 =?us-ascii?Q?Fei4shg0zmNHQDvqVIbKfBx/CpvBA3+KQ1k6DoB9umirddJQf/ZrWUCHwWsz?=
 =?us-ascii?Q?itZxG8Fewo7GxarGWmKZJHK9C1MHOMVRryobdTCjeOEu4fRxCcEjs0OpIztk?=
 =?us-ascii?Q?213VhxIK2n6nR8F8oVzVDyumIL2fXniJ+UBr2mZhqu0fIQiFEwoJMzvPUxS4?=
 =?us-ascii?Q?2LNSonDOTTwYDGphLkw=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cd60da2c-9b8f-4758-8982-08dc57eab52e
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Apr 2024 16:41:15.5122
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: BHU988g3xfnaCtNHHcYdd+BwsK3Q8a1AYxR65SujLots/+JPI5Y/RORph0tZJdyd
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4230

On Fri, Apr 05, 2024 at 08:38:27AM -0700, Jakub Kicinski wrote:
> On Fri, 5 Apr 2024 08:13:06 -0300 Jason Gunthorpe wrote:
> > As I answered to Anderew, a lot is functional behavior not so much
> > "tunables". The same way many BIOS settings are not all tunables but
> > have functional impacts to the machine. Like enable SRIOV, for
> > instance.
> 
> Thanks, SRIOV is a great example:
> https://docs.kernel.org/next/networking/devlink/devlink-params.html#id2
> Literally the first devlink param on the list.

It is too basic to be really usable, unfortunately - recall my earlier
remarks that a site needs to configure everything, not just the 2 SRIOV
related values in that take. Those are nice generic ones, but others
are not.

> The only "tunables" I'm aware of were for the OCP Yosemite platform,
> which is an interesting beast with 4 hosts plugged into one NIC,
> and constrained PCIe BW. Which is why I said the "tunables" are really
> about the server platform not being off the shelf. Updating NIC FW
> to fix server compatibility is hardly unusual.

I don't think it is appropriate to go into details of all the stuff
that happens in the commercial relationship between Meta and
NVIDIA.. There is lots of history there.

My main point is, for others reading these threads, that taking a COTS
device like mlx5 and essentially forking it for a single user's
special requirements is pretty much standard operating procedure
now. Some people enjoy this with custom devices and custom FW, some
people have to run standard FW and customize it at provisioning time.

Edward accused this all of being "hacks", but I strongly
disagree. Having device specific customization and parameters that
make sense for the device architecture is not a hack. Functional
changes are not the same was weird performance tunables.

> > So please don't use that as a justification to pull up the ladder so
> > nobody else can enjoy even a semi-customized device.
> 
> So in this thread I'm pulling up the ladder and in the fbnic one I'm
> not (as I hope you'd agree)? One could hopefully be forgiven for
> wondering to what extent your assessment of my intentions is colored
> by whether they align with your particular goals :(

I'm not sure what your position is on fbnic TBH. I said mine, I think
it would be fine to merge that series - but I'm pretty moderate in my
views of what should be accepted to Linux for ideological reasons.

Jason