Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 20 Jul 2022 20:49:24 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     longli@microsoft.com
Cc:     "K. Y. Srinivasan" <kys@microsoft.com>,
        Haiyang Zhang <haiyangz@microsoft.com>,
        Stephen Hemminger <sthemmin@microsoft.com>,
        Wei Liu <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
        "David S. Miller" <davem@davemloft.net>,
        Jakub Kicinski <kuba@kernel.org>,
        Paolo Abeni <pabeni@redhat.com>,
        Leon Romanovsky <leon@kernel.org>, edumazet@google.com,
        shiraz.saleem@intel.com, Ajay Sharma <sharmaajay@microsoft.com>,
        linux-hyperv@vger.kernel.org, netdev@vger.kernel.org,
        linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org
Subject: Re: [Patch v4 12/12] RDMA/mana_ib: Add a driver for Microsoft Azure
 Network Adapter
Message-ID: <20220720234924.GA406750@nvidia.com>
References: <1655345240-26411-1-git-send-email-longli@linuxonhyperv.com>
 <1655345240-26411-13-git-send-email-longli@linuxonhyperv.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <1655345240-26411-13-git-send-email-longli@linuxonhyperv.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?5ZF9O2VHCeJ1bIcBFimF0nm4oA3+OjEl8u8SGHEKiWAl4rFzvYf4uYpLUFxZ?=
 =?us-ascii?Q?l9TGkmnqwHkmJL2RmfNGpN9hG3ft7Uh6G+hXnHshFmHgyQuUPf7ApDEFhHkT?=
 =?us-ascii?Q?OQ50UGN4Ezzoi+1vN+gdFKmbnipx1Iclrl8szjosrmMC1kknVz4BrAgQ1l0T?=
 =?us-ascii?Q?ktTtQjQWXXfUk3rN+Uzx+cJ5baoFUfhdmwY1KmUHTYx/gyqm9r8+iPo+Z8vW?=
 =?us-ascii?Q?07siDN8d3IJOjcT+jlCx069bN+A1mHrYSEvSWyYQUX7LBHPAyjKWafFrrHUG?=
 =?us-ascii?Q?i45TTaHG5XmAbx+l5bkwmrEVIDFhwd0U4J3Wi08sM3rOiThE933uz4H20HUq?=
 =?us-ascii?Q?MHHS5QPX6pkbYplp9aihxN4qkZgV7pnrsqOJyfSLqfF0yXaTQ0Bp5U5DUd9n?=
 =?us-ascii?Q?gB+27J4uEBnlti5zVHOOR0cjg3SA4dD2KqCHH5i+9hLJpgw6kXAFMFdoWGmu?=
 =?us-ascii?Q?O9nz0HSLhyvxYETxGuwvB+Z+OvXze+M8GpVsm7548yvHLeMevlsV4dxUS7dO?=
 =?us-ascii?Q?+x7nJ8SlfH7m0QEFZAB337g/wBksDuVVfMQFawSYdLpYOQVnJ81c1BB5T/uZ?=
 =?us-ascii?Q?pwemZNguQ4KWiNeVxBTT/rclYNtOZ38N2JdrrTT/vzQF2BxseiDpwvERAWq3?=
 =?us-ascii?Q?zNf0MTWHejeaUjY87+tCmAbBD2WbVBkv4RKi/UFa0MGQ/gBY7mIuehAHgXUN?=
 =?us-ascii?Q?QnHH2b0jUc+w7OaUK2i18qmhTuJ3wfA1bYT1OPrq3i4nEPtUm6szdB2FOssc?=
 =?us-ascii?Q?hllbv2gK5S5Ndwiie6Tpo5mw3isjrot/kY3KcBHrshat2wcdmWB6dxjZ0VZp?=
 =?us-ascii?Q?LIQ/WpyxrgosVTGuWVviShA7UeFsLFnVwFuvSoJzNavAAViAfiCmTlSNJ4SJ?=
 =?us-ascii?Q?rnRWQpMtqgEsVGsD9m8nKY59nQC37tNwECme19RRhzzCiDvYzXXGUKtzdVof?=
 =?us-ascii?Q?P1zW9aVDXKJHDmcLxiVllWZxd6ITUC9+e/FPg/Z7VkvIcfuXkaFjBPPLL7su?=
 =?us-ascii?Q?Msls0/9OnFcvbW2wvOxb+wQ6pOU+BVTARyNovSMGDdgHrA/HG5ccDSNXAPY6?=
 =?us-ascii?Q?5XQI9MXW/rvsQNmSMJ1T+RjcYl8paZzupacoJ8e7WSf2bE7SXp0Hy4u5zrLh?=
 =?us-ascii?Q?U7tg6z5e/+fAeDX18pK9zqlHZnIb+HLDha6pHLaF2aEcFivCKz763Ovbg62O?=
 =?us-ascii?Q?Bzs7ZQ2R1zUm0/CHF3pqafPTlAUJ2jnpmxlAn/tF2otYRV+I6O3YkEF1xEvi?=
 =?us-ascii?Q?nWfb4WSyBeQqpskwXmVLZhyCdLTw3+q6IVrLuqmAgFjdkIl3GLS/GKsI8sBz?=
 =?us-ascii?Q?T315JuJKwIygy3sChc+j7aJVoJNCo5OWEap5KUGzuCsjHApels9IWeGVlRBZ?=
 =?us-ascii?Q?Fbv/8u83KguF2ImjiYTi72spZ3y+CEZv4SJX7AcJvgAYXGpa19ErdOrWUHwR?=
 =?us-ascii?Q?3EpkCSWN925gfHDfGnFTZXjYwEzUKhvzQLcRxzw5874p8/oxYqi+xQ/bU2x6?=
 =?us-ascii?Q?sAmHV+to86OkckX6Gl5Z5Rrjlh4UVgjFgv0IV0VOPUmY4YLQsWZ+AesTOrgf?=
 =?us-ascii?Q?nTSKsjcACiU+n5FU7O0Ea/jj3oBvTFF/S91BcjHf?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: aad1de6b-973a-408c-a890-08da6aaa7a22
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jul 2022 23:49:25.7266
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: pMJCAvJtqDJRw15bmwdtvSTw/nC5dEc2ls75c4ClaS2WHyNatiaCfCE/vfqHVwto
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5797
X-Spam-Status: No, score=-1.8 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Jun 15, 2022 at 07:07:20PM -0700, longli@linuxonhyperv.com wrote:

> +static int mana_ib_alloc_pd(struct ib_pd *ibpd, struct ib_udata *udata)
> +{
> +	struct mana_ib_pd *pd = container_of(ibpd, struct mana_ib_pd, ibpd);
> +	struct ib_device *ibdev = ibpd->device;
> +	enum gdma_pd_flags flags = 0;
> +	struct mana_ib_dev *dev;
> +	int ret;
> +
> +	dev = container_of(ibdev, struct mana_ib_dev, ib_dev);
> +
> +	/* Set flags if this is a kernel request */
> +	if (!ibpd->uobject)
> +		flags = GDMA_PD_FLAG_ALLOW_GPA_MR | GDMA_PD_FLAG_ALLOW_FMR_MR;

I'm confused, this driver doesn't seem to support kverbs:

> +static const struct ib_device_ops mana_ib_dev_ops = {
> +	.owner = THIS_MODULE,
> +	.driver_id = RDMA_DRIVER_MANA,
> +	.uverbs_abi_ver = MANA_IB_UVERBS_ABI_VERSION,
> +
> +	.alloc_pd = mana_ib_alloc_pd,
> +	.alloc_ucontext = mana_ib_alloc_ucontext,
> +	.create_cq = mana_ib_create_cq,
> +	.create_qp = mana_ib_create_qp,
> +	.create_rwq_ind_table = mana_ib_create_rwq_ind_table,
> +	.create_wq = mana_ib_create_wq,
> +	.dealloc_pd = mana_ib_dealloc_pd,
> +	.dealloc_ucontext = mana_ib_dealloc_ucontext,
> +	.dereg_mr = mana_ib_dereg_mr,
> +	.destroy_cq = mana_ib_destroy_cq,
> +	.destroy_qp = mana_ib_destroy_qp,
> +	.destroy_rwq_ind_table = mana_ib_destroy_rwq_ind_table,
> +	.destroy_wq = mana_ib_destroy_wq,
> +	.disassociate_ucontext = mana_ib_disassociate_ucontext,
> +	.get_port_immutable = mana_ib_get_port_immutable,
> +	.mmap = mana_ib_mmap,
> +	.modify_qp = mana_ib_modify_qp,
> +	.modify_wq = mana_ib_modify_wq,
> +	.query_device = mana_ib_query_device,
> +	.query_gid = mana_ib_query_gid,
> +	.query_port = mana_ib_query_port,
> +	.reg_user_mr = mana_ib_reg_user_mr,

eg there is no way to create a kernel MR..

So, why do I see so many kverbs like things - and why are things like
FMR in this driver that can never be used?

Jason