Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates
 216.228.112.34 as permitted sender) receiver=protection.outlook.com;
 client-ip=216.228.112.34; helo=mail.nvidia.com;
Subject: Re: [PATCH v2 3/3] IB/core: Obtain subnet_prefix from cache in IB
 devices
To:     Anand Khoje <anand.a.khoje@oracle.com>,
        <linux-rdma@vger.kernel.org>, <linux-kernel@vger.kernel.org>
CC:     <dledford@redhat.com>, <jgg@ziepe.ca>, <haakon.bugge@oracle.com>,
        <leon@kernel.org>
References: <20210603065024.1051-1-anand.a.khoje@oracle.com>
 <20210603065024.1051-4-anand.a.khoje@oracle.com>
From:   Mark Zhang <markzhang@nvidia.com>
Message-ID: <1e7ec500-223b-2cf5-9cf4-651e31d7b072@nvidia.com>
Date:   Thu, 3 Jun 2021 20:10:35 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101
 Thunderbird/78.10.2
MIME-Version: 1.0
In-Reply-To: <20210603065024.1051-4-anand.a.khoje@oracle.com>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2021 12:10:40.4283
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: d27cc667-8761-4e4b-1751-08d926889ae8
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.112.34];Helo=[mail.nvidia.com]
X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT003.eop-nam11.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4340
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 6/3/2021 2:50 PM, Anand Khoje wrote:
> External email: Use caution opening links or attachments
> 
> 
> ib_query_port() calls device->ops.query_port() to get the port
> attributes. The method of querying is device driver specific.
> The same function calls device->ops.query_gid() to get the GID and
> extract the subnet_prefix (gid_prefix).
> 
> The GID and subnet_prefix are stored in a cache. But they do not get
> read from the cache if the device is an Infiniband device. The
> following change takes advantage of the cached subnet_prefix.
> Testing with RDBMS has shown a significant improvement in performance
> with this change.
> 
> The function ib_cache_is_initialised() is introduced because
> ib_query_port() gets called early in the stage when the cache is not
> built while reading port immutable property.
> 
> In that case, the default GID still gets read from HCA for IB link-
> layer devices.
> 
> Fixes: fad61ad ("IB/core: Add subnet prefix to port info")
> Signed-off-by: Anand Khoje <anand.a.khoje@oracle.com>
> Signed-off-by: Haakon Bugge <haakon.bugge@oracle.com>
> ---
>   drivers/infiniband/core/cache.c  | 7 ++++++-
>   drivers/infiniband/core/device.c | 9 +++++++++
>   include/rdma/ib_cache.h          | 6 ++++++
>   include/rdma/ib_verbs.h          | 6 ++++++
>   4 files changed, 27 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/infiniband/core/cache.c b/drivers/infiniband/core/cache.c
> index b6700ad..724ac0e 100644
> --- a/drivers/infiniband/core/cache.c
> +++ b/drivers/infiniband/core/cache.c
> @@ -1624,6 +1624,8 @@ int ib_cache_setup_one(struct ib_device *device)
>                  err = ib_cache_update(device, p, true);
>                  if (err)
>                          return err;
> +               set_bit(IB_PORT_CACHE_INITIALIZED,
> +                       &device->port_data[p].flags);
>          }
> 
>          return 0;
> @@ -1639,8 +1641,11 @@ void ib_cache_release_one(struct ib_device *device)
>           * all the device's resources when the cache could no
>           * longer be accessed.
>           */
> -       rdma_for_each_port (device, p)
> +       rdma_for_each_port (device, p) {
> +               clear_bit(IB_PORT_CACHE_INITIALIZED,
> +                        &device->port_data[p].flags);
>                  kfree(device->port_data[p].cache.pkey);
> +       }
> 
>          gid_table_release_one(device);
>   }

Do we need to clear it in gid_table_cleanup_one()?